D-80 Report on Protocols and Procedures to Verify the Performance of Reference Radiance Sources used by FRM OCRs for Satellite Validation
Comparisons are the main tool used by the National Metrology Institute (NMI) community to ensure the decadal (century) long stability of the SI, to ensure international consistency and to enable international trade. NMIs have always performed international comparisons. In 1999 this process was formalised through the “Mutual Recognition Arrangement” (MRA) which means that customers can obtain traceability from any NMI as the NMIs are equivalent within their internationally accepted “degrees of equivalence”. For this process to work there is a formal process of “key comparisons” in all SI units. These happen at the international level (generally a few laboratories from each world region who have particularly low uncertainties and stable references) and then at the regional level (for all European laboratories, or all Asian laboratories, for example).
NPL participates in such NMI comparisons at the highest level in almost all disciplines. Within the area of radiometry, comparisons are operated through the Consultative Committee on Photometry and Radiometry (CCPR) which meets every two years at the International Bureau of Weights and Measures (BIPM). NPL piloted the first CCPR k1.a comparison on spectral irradiance from 250 nm to 2500 nm – for this we were responsible for establishing the protocol, performing the reference measurements (each participant had different lamps) and analysing and reporting the results. This comparison was the first CCPR comparison under the MRA rules and many of the processes NPL introduced there have formed the basis of the protocols and analysis procedures of later comparisons. NPL also piloted the EURAMET.PR-K1.a comparison of spectral irradiance which provided a link to the CCPR k1.a for all European laboratories (Goodman et al, 2015; Woolliams et al., 2006).
As the first comparisons under the MRA were carried out, it became clear that different pilots were using different analysis methods and that comparison reports were stalled because of disagreements on the analysis process. Because of this, the CCPR established guideline documents on the establishment, running, analysis and reporting of comparisons. NPL was heavily involved in the development of these guidelines and in particular chaired the CCPR Working Group on comparison analysis to develop formal approaches to the rigorous analysis of comparisons and to understand how best to link the regional comparisons to the original CCPR comparison. NPL chaired this working group from 2008 to 2015. Furthermore, in the establishment of the QA4EO guidelines for the Earth Observation Community, NPL used this
expertise and adapted it to meet the needs of Earth Observation. The QA4EO guideline “QA4EO-QAEO-GEN-DQK-004: A guide to comparisons” is based on the CCPR comparison analysis guidelines.
The report on Protocols and Procedures to Verify the Performance of Reference Radiance Sources used by FRM OCRs for Satellite Validation will therefore be compiled by NPL as the “master guide” for implementing the first Laboratory Calibration Experiment (LCE-1). It will also serve as a reference handbook for any future round robin follow-on experiments for irradiance and radiance sources for ocean colour radiometry. In fact the aforementioned NPL expertise will act as a solid basis for the preparation of protocols and procedures for all three intercomparison experiments of this project (LCE-1, -2 and FICE). The NMI intercomparisons are carried out to the highest possible SI-traceable standards using the NPL state of the art radiometric laboratories and these standards will be applied in formulating the protocols and procedures for LCE-1. Furthermore, NPL is offering the same laboratories for the LCE-1 of FRM4SOC.
At the NMI level only irradiance calibration sources are compared and even though the similarities in experimental approach mean that many details will be easily translatable, there will still need to be some adjustment for the intercomparison of radiance sources to suit the measurement requirements for particular types of instruments. For radiance source intercomparisons for ocean colour radiometers there have been the SIREXX experiments (Hooker et al., 2002), the work of the MERIS Validation and AERONET-OC teams (Zibordi and Bialek, 2015; Johnson et al., 2015) and the SI-traceable calibrations of radiance sources (e.g. Spectralon panels) carried out by NPL over the years for many different application areas of both laboratory based and field radiometry, including those used for ocean colour (Woolliams et al. 2003; Pegrum et al., 2004; Bialek et al., 2013). These will also provide basis documentation for setting the protocols and procedures necessary for the radiance source intercomparisons and the part of the best practice guide dealing with radiance rather than irradiance.
D-90 Implementation plan for LCE-1
NPL is planning to extend a global open invitation for LCE-1 to attract as many participants as possible who provide radiometric calibration for the OC community. This will potentially include key instrument manufacturers of ocean colour radiometers commonly in use as well as any other interested parties that have in-house radiometric laboratory facilities.
Having confirmed a list of participants, a review of their radiometric standards will be performed starting with their irradiance sources and power supplies. For example, there are different types of mounts for 1000 W FEL lamps and these have different alignment procedures. It is also important to determine which types and which alignment procedures participants use. The stability and accuracy of each individual participant’s power supply will also be important following the LCE-1 intercomparisons when a verified irradiance source will go back to their home lab and be used as a working standard there. In addition the review will allow the preparation of the laboratory at NPL sufficiently in advance of LCE-1 to accommodate all identified lamps during the intercomparison exercise and provide the level of detail required to prepare a full implementation plan for LCE-1. Training in absolute radiometric calibration and uncertainty evaluation will be given by NPL and any remaining differences between participant’s standards will be accounted for in the uncertainty budget evaluations of WP303.
Following the irradiance and power supply review, similar information on the partners’ radiance source set ups (e.g. shield position and laboratory stray light estimation) will be detailed as an input to the implementation plan for this part of the intercomparison. Further detailed practical considerations will also be included in the implementation plan. For example, for source stability through the duration of the experiments, NPL will recommend hand carrying lamps to and from LCE-1 where participants are able to attend in person. This does not preclude remote participation, although it is possible that a lamp can change during transportation and therefore, if possible, 3 lamps from each participant should be transported with an appropriate means to and from the comparison.
D-100 Host LCE-1 at NPL
For the irradiance source inter-comparisons all FEL lamps submitted to the LCE-1 exercise will be measured at NPL against NPL primary standards. A training session on absolute radiometric calibration will be held at NPL at the beginning of LCE-1 for all participants and this will include detail on how to conduct uncertainty evaluation. Following this the participants will be shown the NPL Spectral Radiance and Irradiance Primary Scales (SRIPS) facility (please see below) and this will include training on the alignment process. The actual measurements of each of the participant’s irradiance sources will then be run on SRIPS.
As a result of this comparison, irradiance values will be obtained from each lamp as measured under the carefully controlled conditions of SRIPS at NPL and these will be compared with their certificate values and with each other.
The second phase of LCE-1 concerns the inter-comparison of radiance sources for OCRs. As a radiance source is formed by combining an FEL lamp and a diffuser panel, comparing several reflectance panels using one lamp provides only a partial solution for the comparisons. The most appropriate method for carrying out a complete radiance source comparison between labs, and one that follows the suggestions of the ITT is using an NPL fully characterised transfer radiometer. This will involve an NPL calibrated transfer radiometer sent back and forth in turn to each participant to perform radiance measurements. The transfer radiometer in this configuration will be used to compare the participant’s in-house radiance sources with the NPL derived radiance scale. As NPL do not own an OCR, and to purchase a set of transfer radiometers dedicated solely to the LCE-1 is out of the scope of the FRM4SOC budget, we have arranged with an external collaborator (JRC – see letter of intent attached to the proposal) to use one or more of their stable OC filter radiometers as the transfer radiometer(s) for the radiance source comparisons.
lthough we believe that the above arrangements will work well for FRM4SOC and LCE-1, if there are any unforeseen problems NPL can follow one of a number of additional options to complete the radiance source comparisons. For example, there is a radiometer being built in the METEOC-2 project for NPL for terrestrial applications that may be suitable as a stable transfer radiometer as it covers the necessary wavelength range for ocean colour. Alternatively, we could use a radiometer from each participant, i.e. multiple transfer radiometers. This proposed solution involves each participant calibrating their own radiometer against their radiance scale, sending it to NPL so we can calibrate it against our radiance scale, then re-measuring at the participant’s lab. The radiance scale at NPL is the common reference against which all the radiometers are compared, the before and after measurements at the participant provide confidence the radiometer hasn’t changed during the course of the comparison. The main drawback with this alternative is that we will not have fully characterised each participant’s radiometer and therefore will not have a complete idea of the source of all the uncertainties in the measurements of the radiance sources. A final alternative would be for NPL to visit each participant with an NPL radiance standard and another non-OC transfer radiometer. Radiance measurements would be taken at each participant’s lab using both the labs and the NPL radiance standards. This solution would be a last resort as the cost involved is out of scope of the project’s budget, including the 50% contribution supplied by NPL.
The radiance measurements at each participant’s lab using the transfer radiometer will be scheduled after the irradiance source comparison at NPL. Therefore, as well as the training course and the irradiance lab visit, focus for LCE-1 will be placed on the characterisation and absolute radiometric calibration of the transfer radiometer(s) at a previously prepared NPL laboratory with full traceability to NPL and SI standards.
At NPL all optical radiation measurement is traceable to the cryogenic radiometer. The cryogenic radiometer is used to calibrate trap detectors, which in turn calibrate other detectors including filter radiometers. The filter radiometers are used to measure the temperature of the blackbody and thus determine its spectral radiance. In this way the source emission scales, based on the blackbody, are linked to the detector scales. The SRIPS facility is used to transfer the scale from the blackbody to lamp and integrating sphere sources. These sources are then used as spectral radiance and irradiance standards further down the chain.
SRIPS will be the main facility used at NPL for LCE-1 of FRM4SOC. It is based around a monochromator that defines the wavelength for the comparison. In front of this monochromator input optics collect the light and, after the mono- chromator, detectors measure that light. Figure 4-5. presents a schematic diagram of the SRIPS. The monochromator is operated in turn with each of three different gratings and there are four different detectors used to cover the full spectral region.
Figure 4-5. Diagram of the NPL SRIPS facility layout.
At additional cost, if requested by the LCE-1 participants, NPL can also offer the use of the National Reference Reflectance (NRR) facility to measure the radiance/reflectance factor for each of their panels. This is with an illumination at 0 degrees and a viewing angle at 45 degrees that gives more precise information than the diffuse reflectance provided by the panel manufacturer. Although not necessary for the intercomparisons this has the advantage of providing further data on possible biases due to differences in the panels themselves.
D-110 Organize results data from LCE-1
NPL has many decades of experience in the collection and quality control analysis of many different laboratory based optical measurement datasets. The NPL laboratories and facilities involved have dedicated custom software that automates and facilitates this data gathering and organization process critical to the success of an intercomparison experiment. This NMI expertise in what is needed in terms of the data that needs to be collected and how they should be organised for all the experiments in FRM4SOC, will be an invaluable aid to the success of the project. The detail on this, for example the uncertainty data needed during each step of the experiments, will be provided by NPL for each protocol and procedure document. Additionally for LCE-1 this will not only include training on the correct measurement protocols to be followed but also on the data that should be gathered by the participants back at their home laboratories. Particular emphasis will be placed on providing correct uncertainty data.
Figure 4-6. Typical uncertainty data output from an absolute radiometric calibration of an OCR.
D-120 Prepare report on LCE-1 results
An NPL report on LCE-1 will be published online and supplied to ESA as one of the deliverables of this activity. However, as the results may prove valuable as a reference to the wider scientific community, in particular the ocean colour community, NPL will also look to publish a summary of the experiments and the most important results in a relevant peer reviewed journal.