Skip to main content
Springer Nature Link
Log in

On-the-Go Reflectance Transformation Imaging with Ordinary Smartphones

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13801))

Included in the following conference series:

Abstract

Reflectance Transformation Imaging (RTI) is a popular technique that allows the recovery of per-pixel reflectance information by capturing an object under different light conditions. This can be later used to reveal surface details and interactively relight the subject. Such process, however, typically requires dedicated hardware setups to recover the light direction from multiple locations, making the process tedious when performed outside the lab.

We propose a novel RTI method that can be carried out by recording videos with two ordinary smartphones. The flash led-light of one device is used to illuminate the subject while the other captures the reflectance. Since the led is mounted close to the camera lenses, we can infer the light direction for thousands of images by freely moving the illuminating device while observing a fiducial marker surrounding the subject. To deal with such amount of data, we propose a neural relighting model that reconstructs object appearance for arbitrary light directions from extremely compact reflectance distribution data compressed via Principal Components Analysis (PCA). Experiments shows that the proposed technique can be easily performed on the field with a resulting RTI model that can outperform state-of-the-art approaches involving dedicated hardware setups.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    \(l_u\) and \(l_v\) range between \([-1 \ldots 1]\) respectively as they are the first two components of a (unitary-norm) 3D light direction vector pointing toward the light source.

References

  1. Ackermann, J., Fuhrmann, S., Goesele, M.: Geometric point light source calibration. In: VMV, pp. 161–168 (2013)

    Google Scholar 

  2. Ahmad, J., Sun, J., Smith, L., Smith, M.: An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region. Pattern Recogn. Lett. 50, 15–22 (2014)

    Article  Google Scholar 

  3. Ciortan, I., Pintus, R., Marchioro, G., Daffara, C., Giachetti, A., Gobbetti, E., et al.: A practical reflectance transformation imaging pipeline for surface characterization in cultural heritage (2016)

    Google Scholar 

  4. Coules, H., Orrock, P., Seow, C.E.: Reflectance transformation imaging as a tool for engineering failure analysis. Eng. Fail. Anal. 105, 1006–1017 (2019)

    Article  Google Scholar 

  5. Dulecha, T.G., Fanni, F.A., Ponchio, F., Pellacini, F., Giachetti, A.: Neural reflectance transformation imaging. Visual Comput. 36, 2161–2174 (2020). https://doi.org/10.1007/s00371-020-01910-9

  6. Earl, G., et al.: Reflectance transformation imaging systems for ancient documentary artefacts. In: Electronic Visualisation and the Arts (EVA 2011), pp. 147–154 (2011)

    Google Scholar 

  7. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)

    Article  Google Scholar 

  8. Giachetti, A., Ciortan, I., Daffara, C., Pintus, R., Gobbetti, E., et al.: Multispectral RTI analysis of heterogeneous artworks (2017)

    Google Scholar 

  9. Giachetti, A., Ciortan, I.M., Daffara, C., Marchioro, G., Pintus, R., Gobbetti, E.: A novel framework for highlight reflectance transformation imaging. Comput. Vis. Image Underst. 168, 118–131 (2018)

    Article  Google Scholar 

  10. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2003)

    MATH  Google Scholar 

  11. Kinoshita, S., Yoshioka, S., Miyazaki, J.: Physics of structural colors. Rep. Progress Phys. 71(7), 076401 (2008). https://doi.org/10.1088/0034-4885/71/7/076401

  12. Kinsman, T.: An easy to build reflectance transformation imaging (RTI) system. J. Biocommun. 40(1), 10–14 (2016)

    Article  Google Scholar 

  13. Kotoula, E., Kyranoudi, M.: Study of ancient Greek and Roman coins using reflectance transformation imaging. E-conservation Mag. 25, 74–88 (2013)

    Google Scholar 

  14. Malzbender, T., Gelb, D., Wolters, H.: Polynomial texture maps. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 519–528 (2001)

    Google Scholar 

  15. Manfredi, M., et al.: Measuring changes in cultural heritage objects with reflectance transformation imaging. In: 2013 Digital Heritage International Congress (DigitalHeritage), vol. 1, pp. 189–192. IEEE (2013)

    Google Scholar 

  16. Manrique Tamayo, S.N., Valcárcel Andrés, J.C., Osca Pons, M.: Applications of reflectance transformation imaging for documentation and surface analysis in conservation. Int. J. Conserv. Sci. 4, 535–548 (2013)

    Google Scholar 

  17. Mudge, M., et al.: Image-based empirical information acquisition, scientific reliability, and long-term digital preservation for the natural sciences and cultural heritage. In: Eurographics (Tutorials), vol. 2(4) (2008)

    Google Scholar 

  18. Mudge, M., Malzbender, T., Schroer, C., Lum, M.: New reflection transformation imaging methods for rock art and multiple-viewpoint display. In: Ioannides, M., Arnold, D., Niccolucci, F., Mania, K. (eds.) The 7th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, vol. 6, pp. 195–202. Vast (2006)

    Google Scholar 

  19. Mytum, H., Peterson, J.: The application of reflectance transformation imaging (RTI) in historical archaeology. Hist. Archaeol. 52(2), 489–503 (2018)

    Article  Google Scholar 

  20. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979). https://doi.org/10.1109/TSMC.1979.4310076

    Article  MathSciNet  Google Scholar 

  21. Palma, G., Corsini, M., Cignoni, P., Scopigno, R., Mudge, M.: Dynamic shading enhancement for reflectance transformation imaging. J. Comput. Cult. Heritage (JOCCH) 3(2), 1–20 (2010)

    Article  Google Scholar 

  22. Pintus, R., Dulecha, T.G., Ciortan, I., Gobbetti, E., Giachetti, A.: State-of-the-art in multi-light image collections for surface visualization and analysis. In: Computer Graphics Forum, vol. 38, pp. 909–934. Wiley Online Library (2019)

    Google Scholar 

  23. Pistellato, M., Albarelli, A., Bergamasco, F., Torsello, A.: Robust joint selection of camera orientations and feature projections over multiple views, pp. 3703–3708 (2016). https://doi.org/10.1109/ICPR.2016.7900210

  24. Pistellato, M., Bergamasco, F., Albarelli, A., Torsello, A.: Dynamic optimal path selection for 3D triangulation with multiple cameras, vol. 9279, pp. 468–479 (2015)

    Google Scholar 

  25. Pistellato, M., Bergamasco, F., Albarelli, A., Torsello, A.: Robust cylinder estimation in point clouds from pairwise axes similarities, pp. 640–647 (2019). https://doi.org/10.5220/0007401706400647

  26. Pitard, G., et al.: Discrete modal decomposition: a new approach for the reflectance modeling and rendering of real surfaces. Mach. Vis. Appl. 28(5), 607–621 (2017)

    Article  Google Scholar 

  27. Ponchio, F., Corsini, M., Scopigno, R.: Relight: a compact and accurate RTI representation for the web. Graph. Models 105, 101040 (2019)

    Article  Google Scholar 

  28. Porter, S.T., Huber, N., Hoyer, C., Floss, H.: Portable and low-cost solutions to the imaging of paleolithic art objects: a comparison of photogrammetry and reflectance transformation imaging. J. Archaeol. Sci. Rep. 10, 859–863 (2016)

    Google Scholar 

  29. Rainer, G., Jakob, W., Ghosh, A., Weyrich, T.: Neural BTF compression and interpolation. In: Computer Graphics Forum, vol. 38, pp. 235–244. Wiley Online Library (2019)

    Google Scholar 

  30. Ren, P., Dong, Y., Lin, S., Tong, X., Guo, B.: Image based relighting using neural networks. ACM Trans. Graph. (ToG) 34(4), 1–12 (2015)

    Article  Google Scholar 

  31. Schuster, C., Zhang, B., Vaish, R., Gomes, P., Thomas, J., Davis, J.: RTI compression for mobile devices. In: Proceedings of the 6th International Conference on Information Technology and Multimedia, pp. 368–373. IEEE (2014)

    Google Scholar 

  32. Shen, C.: Analysis of detrended time-lagged cross-correlation between two nonstationary time series. Phys. Lett. A 379(7), 680–687 (2015)

    Google Scholar 

  33. Smys, S., Chen, J.I.Z., Shakya, S.: Survey on neural network architectures with deep learning. J. Soft Comput. Paradigm (JSCP) 2(03), 186–194 (2020)

    Article  Google Scholar 

  34. Suzuki, S., Be, K.: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985). https://doi.org/10.1016/0734-189X(85)90016-7, https://www.sciencedirect.com/science/article/pii/0734189X85900167

  35. Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. Adv. Neural. Inf. Process. Syst. 33, 7537–7547 (2020)

    Google Scholar 

  36. Uribe, M.D.G., Wheatley, D.W.: Rock art an digital technologies: the application of reflectance transformation imaging (RTI) and 3D laser scanning to the study of late bronze age Iberian stelae. Menga: Revista de prehistoria de Andalucía (4), 187–203 (2013)

    Google Scholar 

  37. Vieira, M., Guimarães, P.V., Violante-Carvalho, N., Benetazzo, A., Bergamasco, F., Pereira, H.: A low-cost stereo video system for measuring directional wind waves. J. Marine Sci. Eng. 8(11), 831 (2020)

    Article  Google Scholar 

  38. Watteeuw, L., et al.: Light, shadows and surface characteristics: the multispectral portable light dome. Appl. Phys. A 122(11), 1–7 (2016)

    Article  Google Scholar 

  39. Xu, Z., Sunkavalli, K., Hadap, S., Ramamoorthi, R.: Deep image-based relighting from optimal sparse samples. ACM Trans. Graph. (ToG) 37(4), 1–13 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DAIS, Universitá Ca’Foscari Venezia, 155, via Torino, Venezia, Italy

    Mara Pistellato & Filippo Bergamasco

Authors
  1. Mara Pistellato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Filippo Bergamasco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mara Pistellato .

Editor information

Editors and Affiliations

  1. IBM Research AI and MIT-IBM Watson AI Lab, Haifa, Israel

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

1 Electronic supplementary material

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pistellato, M., Bergamasco, F. (2023). On-the-Go Reflectance Transformation Imaging with Ordinary Smartphones. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25056-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25055-2

  • Online ISBN: 978-3-031-25056-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics