For the process of hydrographic surveying, the skipper of the survey vessel needs a variety of real-time navigational and hydrographic information to navigate the vessel safely and precisely. The survey data is typically displayed on several monitors on board. As part of a research project, we are exploring the potential of augmented reality – a technology that can extend reality by overlaying it with computer-generated visualizations – to assist the skipper in hydrographic surveys. We report an overview of the technology itself, our proof of concept and several challenges we focus on to create a usable and valuable prototype system.