Elsevier

Computers & Geosciences

Volume 147, February 2021, 104666
Computers & Geosciences

Toward real-time optical estimation of ocean waves’ space-time fields

https://doi.org/10.1016/j.cageo.2020.104666Get rights and content

Highlights

  • WASSfast is a sea-waves 3D reconstruction method optimized for speed.

  • Wave spectrum is continuously estimated from a sparse set of stereo matches.

  • Extensive tests with real-data show good accuracy and 10× faster processing.

Abstract

Stereo 3D reconstruction is continuously increasing its popularity in the study of mid-to small-scale sea waves. In the recent past, different approaches have been proposed to reconstruct the space-time sea surface elevation field from synchronized stereo frames. Usually, the reconstruction is performed by first recovering a dense and sparse 3D point cloud from stereo pairs and then by interpolating it into a regular grid. Even considering state-of-the-art methods, with typical image resolutions, it's unlikely to perform the first step in less than dozens of seconds (without paralellization), that can easily doubled if we include the subsequent interpolation step. This will limit the applicability of stereo based wave analysis for all the approaches (like monitoring or data assimilation) in which the time between the raw acquisition and the output of processed data is critical.

In this paper, we propose a new method to directly estimate the sea surface spectrum over time from a sequence of stereo frames. We exploit the frequency dispersion relation of gravity waves and the (Non-uniform) Fast Fourier Transform to continuously update the 3D surface and 2D wavenumber spectrum estimations given a sparse set of matching features between the two frames. This effectively combines the two aforementioned steps with a performance gain of more than an order of magnitude while keeping a reconstruction accuracy that is extremely close to the current state-of-the-art approaches. Code is available at https://gitlab.com/fibe/wassfast.

Introduction

The recent advances of imaging hardware and computing power are changing the paradigm driving the study of wind-driven ocean waves. As a matter of fact, a transition from traditional one-point temporal to spatio-temporal observational systems is a recent trend followed by many researchers and engineers. Indeed, spatial wave fields enable the analysis of many distinctive features that are simply not observable with legacy point-like buoys, wave-probes or similar instruments. To cite a few examples, Banner et al. (1989) analyzed wavenumber spectra of short gravity waves while Leckler et al. (2015) used the Fast Fourer Transform of 3D stereo data to study the shape of wavenumber-frequency spectra. For wave extremes, Fedele et al. (2013) compared space-time statistics of high-waves and Benetazzo et al. (2015) observed that rogue waves are more frequent than expected when leveraging from a point-based to an area-based surface analysis. These new insights were also recently included within the spectral wave model WAVEWATCH III by (Barbariol et al., 2017).

Spatio-temporal observational systems can be grouped in 3 broad categories, depending on the scale to which the observation is targeted. At a larger scale (10300 meters wavelength), radar is a commonly used system to estimate the spectral properties of 2D wave fields (Young et al., 1985; Nieto Borge et al., 2013). Despite its popularity, this technology does not allow the direct observation of the wave field, that instead must be inferred, from the radio wave backscatter, through an empirically defined Modulation Transfer Function (MTF) (Nieto Borge et al., 2004; Benetazzo et al., 2018). At a medium scale (0.250 meters wavelength) optical stereo-based 3D reconstruction techniques offer an excellent tradeoff between reconstruction accuracy, resolution and cost (Jähne et al., 1994; Benetazzo et al., 2012). Finally, at small scale (0.0011 meter wavelength), polarimetric imagery (Zappa et al., 2008) is the only technology able to capture capillary waves, but usually involves expensive hardware and complex calibration procedures.

In the context of 3D stereo reconstruction, Bergamasco et al. (2017) published WASS (Wave Acquisition Stereo System), the first open-source 3D reconstruction pipeline adopted nowadays by many research teams worldwide. Designed to take advantage of modern parallel architectures, WASS can efficiently produce dense 3D point clouds from sequence of stereo frames. However, point clouds are just half of the story. To be useful in practice, point-based data must be rotated to a geographical reference frame, and interpolated into a regular surface grid. Under the assumption that the system remains fixed, the first problem can be simply solved by averaging the 3D mean sea-plane estimated from multiple stereo frames. Instead, if cameras are mounted on a ship, a motion-correction approach is a mandatory step to enable the subsequent spectra analysis (Brandt et al., 2010; Benetazzo et al., 2016; Bergamasco et al., 2016). The second problem failed to find the attention it deserves by the community perhaps because the interpolation of scattered data is something related to data processing more than sea-waves reconstruction. However, general-purpose methods like Radial Basis Functions (Smolik and Skala, 2017) are not designed for sea-waves reconstruction and, for instance, do not allow to impose additional priors like the distribution of local slope gradient. More than that, they tend to be computationally intensive, especially when interpolating million point samples, as typically produced by WASS. For all these reasons, the resulting surface may contain spikes (hence needs to be filtered) and takes longer to be produced than the actual stereo reconstruction itself.

In this work we explore a new end-to-end approach to directly produce a 3D surface (and its 2D wavenumber spectrum) from stereo images, quickly enough to output the 3D data at a ratio close to the image acquisition speed. This would allow new observation systems to be able to continuously monitor an area and compute meaningful statistics of the directional wave field. Possible applications include the usage of 3D stereo data for wave model data assimilation, the collection of wave statistics over extremely long periods, and the real-time tuning of other instruments requiring some parameters of the current sea-state (like the MTF function of marine radars).

Section snippets

WASSfast

We have developed a completely new method, named WASSfast, to directly and efficiently obtain the space-time wave field from a sequence of stereo frames. We require a fixed stereo-rig observing the same area throughout the whole acquisition. The area extent, together with the mean sea-plane and the camera calibration parameters must be defined a-priori. These conditions are easily met in all ‘‘fixed-platforms” installations like oceanographic towers, lighthouses, shorelines etc. In return to

Experiments

To demonstrate the capabilities of the new fast reconstruction approach, we compared WASSfast with the results obtained by the WASS pipeline for different stereo sequences. We considered a total of 4 records (Table 1), one acquired at the Acqua Alta oceanic research platform (See Benetazzo et al. (2015) for a description of the setup) and the others at the Gageocho-ORS managed by the Korea Institute of Ocean Science and Technology (KIOST) after the passage of the typhoon Kong-rey on October

Conclusions

We presented a new method to estimate the space-time sea surface elevation field from a sequence of stereo images that can operate more than an order of magnitude faster than the current state-of-the-art method while keeping a reasonable reconstruction accuracy. It works by combining a sparse 3D point cloud reconstruction based on optical flow estimation and a spectrum prediction and update cycle exploiting the linear dispersion relation of gravity waves. One of the main contributions of this

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The study was partially supported by the project of Construction of Ocean Research Stations and their Application Studies funded by the Ministry of Oceans and Fisheries, Republic of Korea.

Authors also gratefully acknowledge the funding from the Flagship Project RITMARE - The Italian Research for the Sea - coordinated by the Italian National Research Council and funded by the Italian Ministry of Education, University and Research within the National Research Program 2011–15.

References (25)

  • A. Benetazzo et al.

    Observation of extreme sea waves in a space-time ensemble

    J. Phys. Oceanogr.

    (2015)
  • J.-Y. Bouguet

    Pyramidal Implementation of the Lucas Kanade Feature Tracker

    (2000)
  • Cited by (8)

    • Experimental investigation on three-dimensional structures of wind wave surfaces

      2022, Ocean Engineering
      Citation Excerpt :

      Inspired by their models, the following studies concentrated on improving the models for better prediction (Janssen, 2004; Li and Shen, 2022; Sullivan and McWilliams, 2010). Taking advantage of optical methods, the reconstruction of water waves has made great progress in the field measurement techniques (Benetazzo et al., 2012, 2015, 2018a, 2016; Bergamasco et al., 2017, 2021; de Vries et al., 2011; Mironov et al., 2012) and laboratory measurement techniques (Lev, 2018; van Meerkerk et al., 2020; Zavadsky et al., 2017) in the past ten years. Generally, the measurement principle of a wave surface can be categorized into (i) refraction-based techniques relating the slope change to a change in height, (ii) stereo-correlation–based techniques that determine the three-dimensional world position of particles or their naturally present features, and (iii) projection-based techniques that determine the free-surface height using a projected pattern.

    View all citing articles on Scopus
    1

    All authors contributed to the writing of the manuscript. F.Bergamasco conceived the presented idea. F. Bergamasco, A. Benetazzo, and A. Torsello designed the model and the computational framework and analyzed the data.

    View full text