Reflectance Transformation Imaging (RTI)
In progress: Seeking additional comments and images to develop this section
Please be patient, this part of the site is under development. We are starting to build out the Imaging Wiki.
Reflectance Transformation Imaging (RTI) is a Computational Photography assisted technique, which uses multi-lighting conditions to capture a set of images, from a fixed camera position, with the aim of virtually and interactively revealing the characteristics of an imaged surface.
Interested in contributing to this page? Visit the Contributors' Toolbox or reach out to one of our Team Leads:
Wiki Team Leads: Emily Frank, Hendrik Hameeuw, Bruno Vandermeulen
Wiki Editors: Christopher Ciccone, Caroline Roberts, Jessica Walthew
Wiki Contributors: Amalia Siatou, Alexander Dittus, Bruno Vandermeulen, Carla Schroer, Caroline Roberts, Christopher Ciccone, Emily Frank, Eve Mayberger, Hendrik Hameeuw, Jessica Walthew, Kurt Heumiller, Paige Schmidt
You can also leave comments or make quick suggestions about the content you see on this page using the RTI Wiki Suggestions form.
What is RTI[edit | edit source]
RTI (Reflectance Transformation Imaging) is a user-friendly, non-invasive imaging technique for the examination and documentation of cultural heritage object surfaces. In this technique, a source image set is processed into an interactive file. It allows the viewer to examine the visual appearance of an object in various lighting conditions with a range of computational enhancements, highlighting and revealing characteristics of the imaged object. Applications using this technique range from simple, accessible tools to highly calibrated scientific systems. RTI can be used for a variety of activities including documentation, access, condition monitoring, interventive conservation treatment, interactive museum displays, and research. The distinctive feature of this method is the ability to virtually relight the imaged surface from any raking angle in a viewer interface. A processed source image set is therefore often referred to as a relightable file or image.
Key to this imaging method is the acquisition of the source image set; a series of images are captured with the object and camera static relative to each other, while knowing or recording the spatial location of the light sources.
The following steps/conditions are essential to this method:
- A. A camera is mounted on top of a physical or imaginary hemispherical umbrella/dome facing the object.
- B. A series of individual images are captured, each lit (with a flash or continuous light source) from varying directions following the shape of a physical or imaginary umbrella/dome. These images can be many of the same light sources, all with fixed positions, homogeneously distributed, or a single light source to be manually repositioned.
- C. Throughout the recording process, both the camera and object stay 100% motionless.
- D. The distance from the light source to the imaged object is consistent throughout all recordings.
- E. Throughout the acquisition process, the spatial position of the applied light sources is registered (highlight method) or it is predetermined and incorporated into the processing software (some of the dome methods).
- F. The obtained source image set is processed into a relightable image with appropriate software.
Other more general terms for this acquisition method are also in use, i.e. Single Camera Multi Light (SCML) and Multi-light Reflectance (MLR). See the Setup section for a more detailed overview on the various types of acquisition systems (dome, highlight, rotating arm, and microscopic methods), equipment, and setups.
SCML methods such as RTI use of multiple angles of illumination to understand the reflectance and topographical information of the surface of interest. The most common implementation of RTI is via Polynomial Texture Mapping (PTM) invented by Tom Malzbender of HP Labs in 2000 (Maelzbender et al. 2000, Maelzbender et al. 2001) and HSH (Hemispherical Harmonics) RTI introduced for heritage applications by 2008. Since, these types of interactive imaging have been applied to a range of art and cultural assets in a wide variety of situations (see also the History section) (Mudge et al. 2008, Earl et al. 2010, Klausmeyer et al. 2012, Mudge et al. 2006).
Key to the computer assistance in SCML/MLR techniques is that the position of the light source in each image be known. When the camera/multi-light combination system is aligned in a fixed setup (such as a dome, with a rotating arm, and/or under a microscope), the information about the light positions can be included in the processing software beforehand. In other cases (methods relying on the highlight method), each photograph in the source image set needs to include a reflective sphere. It allows the processing software to keep track of the position of the light source in each shot; this crucial information to be extracted and used during the processing phase. This lighting information is used to generate a mathematical model of the surface of interest. More technically, the RTI method derives a per-pixel reflectance model represented by a set of coefficients that defines the fitted function that takes into account the direction of the lighting (Manfredi et al. 2014, LOC RTI Format Description Properties). Other SCML/MLR methods follow similar calculation methods. See the Other Multi-light Reflectance Technologies section for the functionality and differences of each of these methods.
The resulting relightable images are processed from a source image set by custom built software (e.g. RTIBuilder) to calculate and establish the interactive value (function) for each pixel. Missing illumination angles are interpolated from the available incident lighting angles and registered in the source image set to fully model the surface’s interaction with light. For each pixel, the surface reflectance is approximated. The result is what can be described as a virtual relightable object, which can mimic any raking light condition. Secondly, various rendering modes reveal, accentuate, and document details on the imaged surface. These outcomes can be consulted in viewer interfaces, which can be stand alone or web-based (e.g. RTIViewer, WebRTIViewer, Pixel+ viewer). When uploaded, all pixel information is mapped with the same resolution as the original photographic captures. The viewers allow the operator to move the virtual light sources over the imaged surface providing unique diagnostic illuminations that would be difficult to obtain by inspecting individual or standardized still-images, and often even surpass visual inspections of the original objects. Furthermore, processed relightable files can be embedded in web pages or combined with additional imaging methods to present multi-faceted investigations of surface topography and other features of art of cultural assets ( Artal-Isbrand et al. 2011, Caine et al. 2011, Hanneken 2016, Watteeuw et al. 2020). Advanced viewer interfaces allow annotations and layered disseminations. Some have the option to generate high-quality screen captures of any particular visualisation at the desire of the operator, to be used for dissemination. See the Viewers section for an extensive overview.
Since its introduction early/mid 2000s, and thanks to the launch of the open access processing and viewing software by Cultural Heritage Imaging (see History below ), RTI and other SCML/MLR methods have been implemented by a large number of cultural institutions, conservators, conservation scientists, research units, and individuals around the world. Many digital repositories include relightable files, and countless applications and workflows have been developed using this imaging technology. In conservation practice, these methods have been used to examine and track surface condition (Hughes-Hallett et al. 2020), document and understand methods of manufacture, (i.e. via study of tool marks, etc.) (Harris and Piquette 2015, Artal-Isbrand 2010, Serotta 2014), and record ephemeral and in situ phenomena (i.e. rock art, archaeological textile impressions, etc.) (Duffy 2018, Frank 2017).
History of RTI[edit | edit source]
Setups[edit | edit source]
Applying RTI or other SCML imaging methods all starts with the creation of the source image set. The image set can be obtained in many ways, as long as the following criteria are met: every image in the set must depict the same surface, the surface in each image of the set must be lit from a different angle, and there must be a method to derive the angle of the light in each image. Some approaches require unique specialized lighting and/or camera devices (sub-sections A, C, D); some can be performed only with more standard photographic equipment (sub-section B).
While methods A, B, and C can be adapted to a variety of applications and it may be possible to create image sets for a given project with multiple setups, each method is best suited for particular project parameters, discussed in the Practical Considerations and Recommended Applications sections. Factors to consider when choosing the best method for a project and subsequent equipment acquisition include (but are not limited to):
- Reproducibility/uniformity of results
- In situ technical infrastructure and working environment
- Accessibility of imaging location and portability requirements of equipment
- Size of the objects to be imaged
- Number of objects to be imaged (and consistency of object sizes)
- Cost of equipment vs. time
A. Dome Method[edit | edit source]
The Dome Method for acquiring the source image set uses a stationary dome structure with mounted light sources at pre-established intervals that can be activated individually according to an automated sequence (Figure 2 in ‘What is RTI’). In this method of image capture, both the camera and lighting structure are stationary, which is in contrast to the mobile light or rotating arm methods, in which single or multiple light systems rotate around the subject in relation to a fixed camera position. Multiple dome systems are available for purchase or can be constructed by users (a useful Affordable DTI Dome toolkit can be found on GitHub, for example). In order to establish lighting positions in each image, the dome method can employ the ‘highlight method’ (see subsection B) or a lighting array whose lighting positions in relation to the camera are known and programmed into the software.
Equipment lists[edit | edit source]
- Camera (essential)
- Lens(es) (essential)
- Dome with fixed lights (essential)
- Reflective sphere targets (some models, other have programmed lighting and camera positions)
- Camera or dome mount (system dependent and/or for non-downward-facing setups)
- Computer for capture (some models, other can operate via camera only)
- Computer for processing (essential)
Workflows[edit | edit source]
Each dome system has its own workflow. They vary from fully automated (a click-and-go acquisition and image processing approach) to semi-automated (positioning reflective targets, automated acquisition, calculating light positions and processing images); some of them make use of the RTI image processing procedures. In general, the dome methods keep the necessary workflows simple.
Field/Low/High-Tech Options (see also examples under section case studies)[edit | edit source]
Dome systems are most commonly used in labs, studios or other ‘inhouse’ locations, including collection spaces. Almost all domes have means to transport them: the smaller versions as a single unit, and larger systems with some (dis)assembly.
Some dome systems can run on batteries and thus, can be used in the field. For those cases, these domes are also made more robust to withstand rough conditions.
Standard domes are positioned on a table or ground and are oriented downwards. Some models have the flexibility to be oriented in many directions (to image vertical or even overhead positioned surfaces).
Some dome devices are modular, and any chosen camera (DSLR or others) can be mounted on top of the dome structure. That gives flexibility towards the image definition and field of view.
Some dome systems are equipped with various or varying types of lighting. In addition to visible white light, sets of ultraviolet, red, green, blue and/or infrared light sources can be installed. These systems expand functionality and combine the abilities of Multi Light Reflectance (MLR) with Multispectral (MS) imaging.
The more acquisition and processing are fully integrated and precisely attuned to each other, the more high-tech features can be added to the toolbox of the viewer software. Examples are xyz-measurements on the surface and combining multispectral recordings.
Practical considerations[edit | edit source]
Dome systems keep the acquisition of the source image set simple. The capture procedure is controlled, consistent and reproducible, which is important for reliable monitoring and scientific applications.
The controlled nature of the dome systems setup allows for relatively fast file generation (± 5 minutes) and serial work. This is important when many objects are being imaged, and is especially useful when the objects’ surfaces and sizes are uniform. The standardized approach of the dome system allows for uniform repetition, which opens strategies to monitor surfaces over time, including before/during/after treatments and as a tool for tracking surface condition.
The surface area that can be imaged with dome devices is limited, depending on the size of the dome and the applied camera and lens combination. Stitching several recordings together is possible, but is still in an experimental phase.
Fully integrated, all-inclusive dome systems can be very expensive (+ $20.000). The more modular devices (for example, to be combined with your own camera and lenses) can be relatively cheap (- $5.000).
Common applications[edit | edit source]
There are purely commercial and more academic/scientific oriented dome systems available. The latter, especially in combination with tailored software, allow for applications which go beyond the visual aspects typical for the RTI/MLR imaging methods (See the RTI in combination with other imaging technologies and Viewers sections)
Tips & Tricks[edit | edit source]
Think carefully about the recording strategy in advance; when the relative position of the dome and the camera (including its lens and zoom settings) stay the same, and the light positions are calibrated with the highlight method, this calibration has to be calculated only once and the obtained Light Position File can be reused.
Make sure the dome is installed in a stable position on a likewise stable work surface. When a laptop is used, position it on a seperate support to avoid vibration of the dome system during image capture.
As most domes create with their structure a black room above/in front of the imaged surface, and the dome method allows a very stable recording process (assuming it is positioned on a stable surface), a relatively small camera aperture can be used to obtain an optimized depth of field and keep more extreme variations in the relief across the surface in focus. The latter is crucial to calculate accurate reflectance characteristics per pixel.
Reusable batch process scripts in image editing programs can be applied to the source image set (recommended to be captured in RAW) to create the jpg derivatives; e.g. for file renaming, rotation, white balance.
Related multi-light technologies[edit | edit source]
Terminology[edit | edit source]
PS (Photometric Studio)[edit | edit source]
HSH RTI (Hemispherical Harmonics)[edit | edit source]
PTM (Polynomial Texture Mapping)[edit | edit source]
RBF RTI (Radial Basis Function)[edit | edit source]
DMD (Discrete Modal Decomposition)[edit | edit source]
Difference between these[edit | edit source]
Data[edit | edit source]
Creation (capture data)[edit | edit source]
Processing[edit | edit source]
Metadating[edit | edit source]
Quality[edit | edit source]
File types (consultation data)[edit | edit source]
Dissemination / RTI Viewers[edit | edit source]
Stand-alone/desktop viewers[edit | edit source]
Web-based viewers[edit | edit source]
Embedding webpages and online databases[edit | edit source]
Technical Support / Obsolescence[edit | edit source]
Main obsolescence issues facing PTM and HSH RTI[edit | edit source]
Processing/Builder[edit | edit source]
The software provided via http://culturalheritageimaging.org/ is reaching its technological expiration date. It is written in java, which was a good choice +10 years ago. However, as of 2020, java and the implemented java-code of the ‘builder’ causes issues on most updated Mac and Windows platforms (e.g. 64-bit operating systems); it no longer runs on these platforms or it is identified as a virus and is automatically deleted upon download.
The RTIBuilder from Cultural Heritage Imaging does two main things:
- It creates a light position (lp) file (based on the information extracted from the reflective spheres in each image in the source image set.
- It applies a processing algorithm (fitter) to the dataset, with the help of the lp file, to generate a finished relightable image.
A number of alternative solutions for step 2 are available, but the RTIBuilder is the primary used path to date.
Viewer[edit | edit source]
At present (December 2021), the two mainly used RTI viewers (desktop and webviewer) work on the most recent Windows and Mac platforms, which means that fully processed PTM and HSH RTI files can still be viewed.
Solutions to address known software issues[edit | edit source]
Mac OS compatibility issues with RTI software[edit | edit source]
Possible solutions:[edit | edit source]
- Maintain a computer with an old OS
- Remote desktop into a computer with an old OS
- Use this https://www.parallels.com/products/desktop/resources/ or virtualbox to run a guest operating system on computer and install the RTI software in the vm (while this link says run Windows on Mac, but you should also just be able to run other Mac os-es)
- Install and run an old OS and the RTIBuilder on an external drive
Running RTIBuilder on Windows 10 Platform[edit | edit source]
Running the RTIBuilder on the current Windows 10 platform can be problematic. In case you can not download RTIbuilder on your computer via http://culturalheritageimaging.org/, try here (same version, different packaging). Important: JAVA needs to be installed on your computer!!!
To run the RTIbuilder, double-click on RTIbuilder.jar. If nothing happens with a double-click on RTIbuilder.jar try:
- Step 1: Click on the “Start” button on your desktop and type “Command Prompt” in the search field. Right-click on the result and select “Run as Administrator.” (You'll need admin rights on your computer.)
- Step 2: In the “Command Prompt” window, type the following command line and hit “Enter”: ftype jarfile="C:\Program Files\Java\jre1.8.0_311\bin\javaw.exe" -jar "%1" %*
- NOTE 1: the \Program Files\ can also be \Program Files (x68); it depends where your Java is installed.
- NOTE 2: the \jre1.8.0_311\ depends on the version of java you have installed, go to the "C:\Program Files\Java\" or "C:\Program Files (x86)\Java\" folder on your computer and define which version of Java you are running; copy that exact version!
- Step 3: Go back to you RTIBuilder folder structure, and double click RTIbuilder.jar
All of these solutions have one element in common: they are trying to find work-arounds to prolong the lifespan of +10 year old processing software. At some point, this will no longer be reasonable; and new software will need to be developed.
The good news, new open processing software interfaces are being produced (or updated) by several stakeholders across the globe. (when fully functioning, they will be discussed in this section)
Case Studies[edit | edit source]
General[edit | edit source]
Specific for Conservation Sciences[edit | edit source]
RTI in combination with other imaging technologies[edit | edit source]
UV[edit | edit source]
IR[edit | edit source]
3D[edit | edit source]
Bibliography / Suggested Reading[edit | edit source]
Artal-Isbrand, P. et al. 2010. “An Evaluation of Decorative Techniques on a Red-Figure Attic Vase from the Worcester Art Museum using Reflectance Transformation Imaging (RTI) and Confocal Microscopy with a Special Focus on the ‘Relief Line’.” MRS Proceedings 1319.
Artal-Isbrand, P., P. Klausmeyer, and W. Murray 2011. “An Evaluation of Decorative Techniques on a Red-Figure Attic Vase from the Worcester Art Museum using Reflectance Transformation Imaging (RTI) and Confocal Microscopy with a Special Focus on the ‘Relief Line’.” in: MRS Online Proceeding Library Archive 1319.
Avgoustinos A., A. Nikolaidou, and R. Georgiou. 2017. "OpeNumisma: A Software Platform Managing Numismatic Collections with A Particular Focus On Reflectance Transformation Imaging". Code4Lib Journal, 37.
Caine, M. and M. Magen. 2011. “Pixels and Parchment: The Application of RTI and Infrared Imaging to the Dead Sea Scrolls.” In Electronic Visualisation and the Arts (EVA 2011), edited by Stuart Dunn, Jonathan P. Bowen and Kia Ng, 140–146. London: BCS.
Duffy, S. 2018. Multi-light Imaging for Cultural Heritage. Swindon: Historic England.
Earl, G.P., K. Martinez, and T. Malzbender. Archaeological Applications of Polynomial Texture Mapping: Analysis, Conservation and Representation. J. Archaeol. Sci. 2010, 37, 2040–2050.
Frank, E. 2017. “Lights, Camera, Archaeology: Documenting Archaeological Textile Impressions with Reflectance Transformation Imaging (RTI).” Textile Specialty Group Postprints 25. The American Institute for Conservation, 11-42.
Hanneken, T. 2016. “New Technology for Imaging Unreadable Manuscripts and Other Artifacts: Integrated Spectral Reflectance Transformation Imaging (Spectral RTI).” In Ancient Worlds in a Digital Culture, edited by Claire Clivaz and David Hamidovic, 180–195. Digital Biblical Studies 1. Leiden: Brill.
Harris, S. and K. Piquette. 2015. “Reflectance Transformation Imaging (RTI) for visualising leather grain surface morphology as an aid to species identification: a pilot study.” Archaeological Leather Group Newsletter 42: 13-18.
Hughes-Hallett, M., C. Young, and P. Messier. 2020. “A Review of RTI and an Investigation into the Applicability of Micro-RTI as a Tool for the Documentation and Conservation of Modern and Contemporary Paintings.” Journal of the American Institute for Conservation.
Klausmeyer, P., R. Albertson, M. Cushman, and P. Artal-Isbrand. Applications of Reflectance Transformation Imaging (RTI) in a Fine Arts Museum: Examination, Documentation, and Beyond. Available online: http://ncptt.nps.gov/wp-content/uploads/klausmeyer.pdf (accessed on 13 June 2014).
Malzbender, T., D. Gelb, H. Wolters, and B. Zuckerman. Enhancement of Shape Perception by Surface Reflectance Transformation. Available online: http://www.hpl.hp.com/techreports/2000/HPL-2000-38R1.pdf (accessed on 9 July 2014)
Malzbender, T., D. Gelb, and H. Wolters. Polynomial Texture Maps. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA, 12–17 August 2001; pp. 519–528.
Mark M., T. Malzbender, A. Chalmers, R. Scopigno, J. Davis, O. Wang, P. Gunawardane, M. Ashley, M. Doerr, A. Proenca, and J. Barbosa. 2008. Image-based empirical information acquisition, scientific reliability, and long-term digital preservation for the natural sciences and cultural heritage. In (Eurographics’08) Tutorials, M. Roussou and J. Leigh, Eds., Eurographics Association.http://culturalheritageimaging.org/What_We_Do/Publications/eurographics2008/eurographics_2008_tutorial_notes.pdf (accessed on 10 March 2021)
Manfredi, M., G. Bearman, G. Williamson, D. Kronkright, E. Doehne, M. Jacobs, and E. Marengo. “A New Quantitative Method for the Non-Invasive Documentation of Morphological Damage in Paintings Using RTI Surface Normals.” Sensors 14, no. 7 (2014): 12271–84. doi:10.3390/S140712271.
Mudge, M., T. Malzbender, C. Schroer, and M. Lum. New Reflection Transformation Imaging Methods for Rock Art and Multiple-Viewpoint Display. Available online: http://culturalheritageimaging.org/What_We_Do/Publications/vast2006/index.html (accessed on 13 June 2014).
Serotta, A. 2014. “An Investigation of Tool Marks on Ancient Egyptian Hard Stone Sculpture: Preliminary Report.” Metropolitan Museum of Art Studies in Science and Technology 2. New York: The Metropolitan Museum of Art, 197-201.
Watteeuw, L., M. Van Bos, T. Gersten, B. Vandermeulen, and H. Hameeuw. 2020. An applied complementary use of Macro X-ray Fluorescence scanning and Multi-light reflectance imaging to study Medieval Illuminated Manuscripts. The Rijmbijbel of Jacob van Maerlant, in: Microchemical Journal 155 (June 2020), 104582. (DOI: 10.1016/j.microc.2019.104582)
RTI Format Description Properties: https://www.loc.gov/preservation/digital/formats/fdd/fdd000486.shtml (accessed on 29 March 2021)
http://culturalheritageimaging.org/What_We_Offer/Downloads/Process/ (accessed on 29 March 2021).
http://culturalheritageimaging.org/What_We_Offer/Downloads/View/index.html (accessed on 29 March 2021).
http://vcg.isti.cnr.it/rti/webviewer.php (accessed on 29 March 2021).
https://www.heritage-visualisation.org/viewer/ (accessed on 29 March 2021).
http://vcg.isti.cnr.it/PalazzoBlu/ (accessed on 4 November 2021)
https://cdli.ucla.edu/?q=rti-images (accessed on 4 November 2021)