Abstract
The processing of intensity data from terrestrial laser scanners has attracted considerable attention in recent years. Accurate calibrated intensity could give added value for laser scanning campaigns, for example, in producing faithful 3D colour models of real targets and classifying easier and more reliable automatic tools. In cultural heritage area, the purely geometric information provided by the vast majority of currently available scanners is not enough for most applications, where indeed accurate colorimetric data is needed. This paper presents a remote calibration method for self-registered RGB colour data provided by a 3D tristimulus laser scanner prototype. Such distinguishing colour information opens new scenarios and problems for remote colorimetry. Using piecewise cubic Hermite polynomials, a quadratic model with nonpolynomial terms for reducing inaccuracies occurring in remote colour measurement is implemented. Colorimetric data recorded by the prototype on certified diffusive targets is processed for generating a remote Lambertian model used for assessing the accuracy of the proposed algorithm. Results concerning laser scanner digitizations of artworks are reported to confirm the effectiveness of the method.
1. Introduction
An active area of research in computer graphics involves the processing of colorimetric data with established and innovative mathematical methods [1–5]. The development of the most widespread instrumental survey techniques (topography and stereophotogrammetry) has been considerable and has made both the acquiring and the processing of a large amount of data possible. Three-dimensional (3D) laser scanning represents one of the ultimate steps of the technological progress in the field of morphological survey and it offers new and important opportunities [6].
The intensity information from terrestrial laser scanners (TLS) has become an important object of study in recent years [7, 8], and there are an increasing number of applications that would benefit from the addition of calibrated intensity data to the topographic information.
Generally, laser scanner performance, such as accuracy and detection range, varies with distance, object reflectivity, and angle of incidence to the reflective surface. The scanner response may be subject to an input-output nonlinearity that needs to be modeled. The quality of a rendered laser image strongly depends on the accuracy of the calibration procedure used. Accurate calibrated intensity could give added value for laser scanning campaigns, for example, in generating faithful 3D colour models of real targets, in making object recognition, and in classifying easier and more reliable automatic tools. While studies of TLS radiometric calibration are sparsely available, to the best of the authors’ knowledge, research on remote colorimetric calibration started in the last few years with the RGB-ITR (Red Green Blue-Imaging Topological Radar) system [9], a patented 3D colour laser scanner prototype completely designed and realized at ENEA laboratories of Frascati (Rome) for cultural heritage purposes. The advantages of 3D imaging systems in the field of cultural heritage are now recognized and widely accepted [10–12]. The purely geometric output provided by the majority of currently available laser scanners is not enough for most cultural heritage applications [13]. A common practice is to superimpose textures derived from ordinary digital photos onto the 3D models generated by most devices. This technique has many drawbacks and limitations in terms of achievable visual quality and accuracy, since the generation and superimposition of texture images are realised via software and commercial cameras. Differently from commercial 3D scanners, the ENEA prototype can simultaneously record for each investigated point range and RGB colour data. Such distinguishing feature opens new scenarios and a new class of problems for remote colorimetry [14].
A standardization of colour information detected by RGB-ITR system, respecting CIE standards, was already performed in the past with colour calibrations by distance information and by MINOLTA spectrophotometer-CM-2600d [9]. These calibrations have assumed a linear relation between the red, green, and blue back-reflected signals and scene reflectances at each distance of measurement. The present study shows that, due to nonlinearities of the optoelectronic system, this assumption produces undesired effects in the remote colour measurement of certified diffusive gray targets (Labsphere Inc.) and proposes a nonlinear calibration for compensating the input-output characteristics of the device, providing meaningful and more accurate 3D RGB-ITR colour images. The improvement gives added value for laser scanning campaigns, providing intensity data much more suitable for cataloguing, dissemination, restoration, and remote diagnosis purposes. The effects of distance and target reflectance on the recorded intensity along with the development of the proposed correction method are analyzed. The model is capable of representing real, small, and big surfaces at short and large distances from the scanner. The implementation is performed with a fast algorithm that uses a quadratic model with nonpolynomial terms and piecewise cubic Hermite polynomials with first-order accurate derivative approximation method. A remote Lambertian model for assessing the accuracy of the algorithm and several experimental results that demonstrate the effectiveness provided by the novel calibration procedure are reported.
2. RGB-ITR Colour Detection
The RGB-ITR system is a tristimulus laser scanner prototype based on the amplitude modulation [15, 16] of three monochromatic sources (660 nm, 514 nm, and 440 nm) and is able to simultaneously collect colour and structure information for any investigated sampled surface point in a working range of about 3–30 meters. This capability makes colour information as important as range data.
Raw colour data returned by the RGB-ITR simply consists of a triplet of voltage values. Each value represents the red, green, or blue light power reflected by a particular surface, as collected by the receiving optics and revealed by the detector. The RGB triplets are then processed with a calibration procedure, which consists in illuminating a movable white target with the laser beam at fixed steps (e.g., 10 cm) along the scanning range. Finally, calibrated RGB data is merged with range information to produce high-quality 3D digital models.
2.1. Linear Calibration
Ideally, a correct colour calibration procedure should represent the content of a real target 3D model by reflecting the intrinsic properties of the target surface. Namely, two colorimetrically indistinguishable regions of the surface should be represented in the model as having the same colour, independently on the conditions under which those regions were sounded during the scan, that is, without being affected by scanning geometry measurement, ambient light effects, and so forth.
The RGB-ITR colour measuring capabilities are influenced by various factors, well described in [13]. In this study, the effects produced by the nonlinearities of the optoelectronic system are illustrated by calibrating certified Lambertian gray targets of and reflectances with raw colour measurements performed on a white diffusive target at different distances from the system. These raw detected data form the calibration curves. Figures 1 and 2 report the calibration curves and the detected gray target signals as functions of distance at different ranges from the scanner. The curves exhibit the same behaviour; that is, they all have a maximum that can be fixed by varying some optical parameters. The on-focus distance for these curves corresponds to = 4.4 m (curves in Figure 1) and = 20.8 m (curves in Figure 2), where approximately all the RGB curves have a maximum. For the collected power falls down mainly because of the misalignment of the optical axes and the typical trend [13]. For , this trend is counterbalanced by the fact that the receiver intercepts only part of the reflected light, so the collected power also falls down. As a consequence, all the calibration curves, their characteristic bell-shape, are the result of the optical parameters tuned up for maximizing the signal-noise ratio of the detected voltages and their shape is independent from distance. Note that, because of the optoelectronic system features, it is not possible to obtain the same RGB signals on certified Lambertian targets at all the distances of measurement. This implies that an accurate calibration with a certified reference target is necessary for producing RGB-ITR colorimetric models.
(a) Red channel amplitudes
(b) Green channel amplitudes
(c) Blue channel amplitudes
(a) Red channel amplitudes
(b) Green channel amplitudes
(c) Blue channel amplitudes
Linear calibration of gray targets is performed by calculating the ratio between the three RGB signals acquired by the instrument for each target and the amplitude value on the three calibration curves selected by the corresponding distance . For each channel the measured reflectance factor of a gray target is given by where is the measured reflectance factor at the distance and is the certified spectral reflectance factor of the perfect diffuser, that is, the white target used in this measurement. As shown by calibrated signals in Figure 3, linear calibration (1) does not return the same reflectance values for light and dark gray targets at each distance. This nonlinearity generates discontinuities in the colour representation of gray targets; that is, the punctual colour model of the same object at different distances is not exactly represented as having the same colour. Figure 3 shows the result of the linear calibration procedure applied to light gray and dark gray targets at 39 sampled distances.
(a) Linear calibrated colour images
(b) RGB reflectance plots as function of distance
3. Methods
This section presents the mathematical models introduced in the proposed calibration procedure. Four Spectralon (Labsphere Inc.) Diffusive Reflectance Standards (SRS) with , , , and nominal reflectance have been used as reference targets for the calculations.
3.1. Quadratic Calibration Model
As shown by linear calibration results in Section 2.1, the back-reflected RGB signals of light gray and dark gray targets are not constant with distance. A model that considers different certified Lambertian reflectances is needed for compensating this system nonlinearity and for producing a more accurate colorimetric representation of real scenes. A nonlinear term is introduced in the model in order to both eliminate inaccuracies in the remote colour measurement of gray targets and mimic the nonlinear perceptual response to luminance of human vision [17]. More accurate data is much more suitable for restoration and diagnosis aids, while adaptation to human perception makes RGB-ITR images more interesting for dissemination, cataloguing, and education purposes. To address the second constraint, the nonlinear term is defined to be comparable to the power function defined by the Munsell equation of lightness according to the standard CIE 1931 and 1964 [18, 19]: with . The term is called the luminance factor, that is, the ratio between the luminances of a specimen and of a perfect diffuser, when illuminated and viewed under specified geometric conditions. By definition, the tristimulus value for an object is the luminance factor and can be approximated as where is a CIE illuminant, is the object’s spectral reflectance factor, is one of the three CIE standard observer colour-matching functions (, , and ), represents summation across wavelength , is the measurement wavelength interval, and is a conventional normalizing constant defined as [19] For each distance of measurement and for each channel a curve is constructed with a quadratic model with nonpolynomial terms necessary for interpolating at each distance four points given by lightness values measured with the MINOLTA spectrophotometer-CM-2600d on four SRS. The use of four targets and the pursuit of the dual objective mentioned above require the model to be nonlinear in the coefficients and in the data, respectively. Namely, given generic data, the model should be in the form where is the nonpolynomial function of the model and , , , are the model coefficients. The diffusive target is assumed to be the zero (black) for the calibration measurements. The MINOLTA measurements were performed in a spectrum interval of 380 nm 740 nm with nm assuming the illuminant and the CIE standard observer. Since reflectance values of gray targets are almost constant in the visible spectrum, RGB-ITR lightnesses, related to 440 nm, 514 nm, and 660 nm, can be well approximated with the spectrophotometer measurements.
Given a set of calibration distances , the model used for each channel and for a generic distance can be defined with the following compact form: where , , , and correspond to the four Lambertian targets used in the calibration procedure, is the input measured voltage, is the imposed measured lightness, , , , and are the unknown model coefficients, and is the nonlinear term defined as comparable to the lightness measure in (2). The model coefficients are computed by constructing and solving a set of simultaneous equations. This is accomplished by forming a design matrix , where each column represents a variable used to predict the response (a term in the model) and each row corresponds to one observation of those variables. For generic target and distance, the matrix is given by The least-square solution of the following system returns the model coefficients at the distance : This procedure is repeated for all distances. Using piecewise cubic Hermite polynomials, coefficients are interpolated at the acquired scene distances. The evaluation of the design matrix at scene detected voltages for each RGB channel gives the nonlinear calibrated intensities , that is, the approximated lightnesses, of the scene:
As an example, the nonlinear model for the red channel, at a generic distance of measurement and with 25 mV of detected signal measured on the white target, is shown in Figure 4.
At this distance the black target detected signal is less than 10 μV (comparable to the instrument noise signal); thus it does not affect the calculation. Differently, near the maximum of the calibration curves, the RGB signals can be very high (tens of mV) and the black signal can be used in the calibration measure since it can be greater than or equal to about 1 mV.
The implementation of the proposed model is given in Algorithm 1.
3.2. Interpolation with Piecewise Cubic Hermite Polynomials
Robust and efficient interpolation of the model coefficients at scene distances is needed for an accurate calibration measure. Piecewise cubic Hermite interpolating polynomials have properties that meet these requests [20, 21]. Given an interval , a function , with derivative , and a set of partition points with , a cubic Hermite spline is defined by a set of polynomials with for . The spline formed by this collection of polynomials can be defined as The Hermite form has two control points and two control tangents for each polynomial. They are simple to calculate, in terms of time required to determine the interpolant and to evaluate it, but at the same time they are too powerful. The slopes at are chosen in such a way that preserves the shape of the data and respects monotonicity [20]. This means that, on intervals where the data are monotonic, is also monotonic; at points where the data has a local extremum, also has a local extremum. These properties make monotonic Hermite interpolation extremely local: if a single interpolation point changes, the approximation can only be affected in the two intervals and which share that partition point. An example of Hermite interpolation of generic data is shown in Figure 5.
(a)
(b)
Algorithm 1 (main algorithm steps of the model).
Consider the following.
Input. Scene intensity matrix , scene distance matrix , calibration distance vector , measured targets intensities , and imposed targets lightnesses at calibration distance are as follows:Output. Calibrated intensity matrix is as follows: (1)Form the design matrix at distance : (2)Compute model coefficients with least-square method at distance : (3)Repeat all for each of the sampling distances: (4)Interpolate model coefficients at scene distances corresponding to the elements of with piecewise cubic Hermite polynomials within and : (5)Evaluate the model for each element of the vectorized intensity channel: (6)Reshape the calibrated scene intensity matrix:
In this work, the derivative values of each polynomial are approximated in such a way that the interpolation could not be significantly affected by errors that can occur in the calibration procedure, that is, wrong distance measurements. Giving a set of calibration distances and detected voltages , the derivatives on the left and right edges are computed by taking the forward differences:On interior points with taking the central differences, This simple approximation is first-order accurate for evenly and unevenly spaced points. More accurate methods for finite difference approximation of derivatives that distinguish between evenly and unevenly spaced points do not work well with noisy data; thus, they are not useful in this context. In order to compare the robustness of this gradient computation method to errors occurring in remote measurements with respect to a more accurate one, noise is introduced in the calibration distance measurements. Hermite interpolations are performed with both approximation methods on data unevenly perturbated by adding normally distributed random numbers on acquired distances. Numerical simulations are performed on single channel data detected in a range of m for 500 different sets of random perturbations ranging from 0 to 5 cm. Three different sets of perturbations are shown in Figure 6. The more accurate approximation method performs Hermite interpolation by using (21) and (22) for evenly spaced points and replaces (22) for unevenly spaced points with the following formula: with . Contrarily to (22), this approximation is second-order accurate for both evenly and unevenly spaced coordinates. Some results of Hermite interpolation with both methods are shown in Figure 7. As can be noted, the less accurate approximation method for derivatives has a low mean interpolation error that is almost constant in all the simulations. Contrarily, in some cases, the more accurate one produces large interpolation errors as can be observed by Figures 7(b) and 7(c). Thus, Hermite polynomials with first-order accurate method keep good capabilities for interpolation and also exhibit low computational complexity.
(a)
(b)
(a)
(b)
(c)
3.3. Remote Lambertian Model for Experimental Validation
An accurate quantitative analysis for RGB-ITR colorimetric calibrations can be carried out by comparing prototype remote measurements with spectrophotometer measurements on certified Lambertian targets.
A remote Lambertian model is computed in laboratory for assessing the accuracy of the proposed algorithm (Algorithm 1). The presented laboratory test was processed with controlled conditions act to simulate all the possible power variations of the laser sources equipped inside the RGB-ITR scanner. The model is based on the RGB measures of the four SRS, placed at different distances and illuminated by RGB-ITR lasers. Each RGB signal is collected by moving the targets at sampling distances of about 10 cm in a range of m. Three RGB curves as a function of distance are computed for each target, that is, three RGB vector quantities, where is the number of sampled distances. Using Hermite polynomials and first-order accurate derivative approximations, remote colour models for light gray ( reflectance) and dark gray ( reflectance) SRS are estimated. More precisely, interpolating the detected RGB signals of the gray targets at acquired RGB-ITR real scene distances (Figure 8) and reshaping the computed vector quantities to scene matrix dimensions, colour texture images for both targets are generated. These interpolated matrix quantities represent the remote Lambertian models, which are then calibrated. Linear and nonlinear calibrations of light gray SRS raw model give the images shown in Figure 9. Differently from the linear calibrated image, the nonlinear one does not show discontinuities; that is, the same object is correctly represented as having the same colour at all scene distances. On the contrary, in Figure 9(a), it is possible to identify objects located at different distances. A quantitative result of this correction is given with the RGB histograms shown in Figure 10. The histograms are computed using the probability normalization function with bin width of 0.05 for both the Lambertian models.
(a) Linear calibrated remote Lambertian model
(b) Nonlinear calibrated remote Lambertian model
(a)
(b)
(a) Raw colour data
(b) Linear calibration
(c) Nonlinear calibration
A comparison of these results with MINOLTA measurements performed on light and dark gray targets is reported in Table 1. RGB linear calibrated values are more different between them with respect to the reflectances measured by MINOLTA. Nonlinear calibration ensures that RGB values measured on gray targets are much more similar to each other and to MINOLTA measurements. In this way, colour discontinuities, caused by nonlinearities of the system, can be reduced. This quantitative result demonstrates that the proposed model provides significantly better colour mapping in comparison with the conventional linear method when illuminating certified targets.
Figure 11 reports raw data normalized by the maximum RGB values and linear and nonlinear calibrated images for the real scene acquired in laboratory at distances shown in the range image (Figure 8) used for remote Lambertian models generation. The raw input image is not colorimetrically correct since each RGB sampled surface point is dependent on the artwork/scanner distance and on the level of the RGB signals that illuminate each point. In particular, the white statue appears greenish, while the small object in the lower left side of the scene is represented to be too dark and is difficult to recognize. The wrong gray shades on the white wall of the linear calibrated image are strongly reduced with the proposed correction method. The proposed scene with large whitish areas well fit the purpose of this validation, since the small variations of the collected back-reflected signals are more visible in a neutral (whitish or grayish) region than in a colourful environment and show that two objects having similar colour properties but located at different distances from the scanner, such as wall and statue, are represented with different colours in noncalibrated data.
4. Applications of the Model
The nonlinear model was tested with success on various RGB-ITR scans. In this paper, results concerning the application of the model on two different digitizations are reported:(i)The fresco of “Amore e Psiche” (Villa Farnesina, Rome) located at a distance of 7–9 meters from the scanning system.(ii)The Vault of the Sistine Chapel (Vatican Museums, Rome) located at 17-18 meters of distance from the scanning system.To confirm the effectiveness of the proposed colorimetric calibration, a comparison with the corresponding normalized raw data and linear calibrated models is shown in Figures 12 and 13. For illustrative purposes, only some portions of these huge digitizations are reported. It can be observed that the raw images are darker than the calibrated ones and have higher red and blue intensity components, related to higher red and blue ITR signals that illuminate the scene at those distances. The processed colour images, in union with the experimental validation in Section 3.3, demonstrate that the nonlinear model can be viewed as a colorimetric transformation that visibly reduces inaccuracies occurring when linearly calibrating grayish/whitish areas and eliminates the dependence on RGB signals’ level, that is, the position of the RGB scene signals in the calibration curves, as performed by linear calibrations.
(a) Raw colour data
(b) Linear calibrated
(c) Nonlinear calibrated
(d) Raw colour data
(e) Linear calibrated
(f) Nonlinear calibrated
(a) Raw colour data
(b) Linear calibrated
(c) Nonlinear calibrated
(d) Raw colour data
(e) Linear calibrated
(f) Nonlinear calibrated
It is important to underline that linear and nonlinear models correspond to two different colour spaces: approximated reflectance space, the linear one, and approximated lightness space, the nonlinear one. Both images appear to be visually pleasing to the observer, but, as quantitatively demonstrated on certified targets, the nonlinear one represents a much more accurate laser colour model.
Overall, these results show that the proposed nonlinear calibration works well for real scenes located at different distances from the scanner. As confirmed by experts in cultural heritage sector, the nonlinear calibrated models can be considered suitable colour images for cultural heritage purposes.
5. Conclusion
This paper has presented a nonlinear model for remote calibration of laser colour images. The problem of linear calibration for gray diffusive certified targets is discussed. The nonlinear model is introduced in the calibration procedure as a postprocessing step for the correction of raw colorimetric data, that is, for both reducing inaccuracies occurring in remote measurement of gray targets and providing meaningful colour images for cultural heritage purposes. The nonlinearity of the model is the consequence of the processing made on data collected by illuminating four different calibrated targets. An experimental validation of the proposed algorithm is performed with a remote Lambertian model computed by using monotonic piecewise cubic Hermite polynomials and certified diffusive targets. This quantitatively demonstrates that the nonlinear model is much more accurate than the linear one when performing remote colorimetric measurements on different certified Lambertian targets. Hermite polynomials with first-order accurate derivative approximation formula have shown robust and efficient interpolation performances in presence of added noise. Results that highlight the effectiveness of the proposed model in producing meaningful images for dissemination, education, and cataloguing purposes are reported.
In conclusion, this work has shown that the processing of colorimetric data permits generating more accurate texture images that if combined with 3D laser models can provide high-quality information. Although remote colour calibration procedure is still object of active research, the proposed nonlinear correction suggests that an optimal calibration is possible for the RGB-ITR. This represents an improvement especially in the cultural heritage domain, where the colorimetric characterisation of an artwork has the same significance of the structural determination of the artwork itself.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors thank the Vatican in the person of Mons. Nicolini, Professor Paolucci, Professor Santamaria, and Professor Di Pinto for their cooperation as cultural heritage experts and for having allowed working in the Sistine Chapel.