QUESTIONS AND ANSWERS REGARDING FULL SPECTRAL IMAGING

 

This page contains some commonly asked questions regarding Full Spectral Imaging and their answers.

For the research projects proposed to provide more details to the answers to these questions, see: Full Spectral Imaging Research Projects

Q. How is curve fitting better than simply compressing the data?
A. It may turn out, after investigating the matter, that curve fitting is not any better than data compression. The primary issue is capturing the information. If capturing the information in the format of bytes per band and then transmitting it to the ground is just as effective as on-board curve fitting, then that is the way to do it. The spectral curves can then be generated on the ground. The curve fitting approach offers many options however, and should be checked to see if it can be effective.

Q. Do all of the features of FSI have to be implemented for it to work?
A. While is is desirable to try to incorporate as many features as possible, for developmental systems at least this is not necessary. The primary idea of the end-to-end system approach is to be sure that they are no "weak links" in the system that will degrade the performance.

Q. How is it possible to process/compress all that data in real-time?
A. We will rely heavily on the expertise of people who actually know what they are doing for this technology. The most promising source may be the theater quality video compression schemes using wavelet, or more specifically, real-time JPEG-2000 3-D compression. The output of a hyperspectral imager is similar to the output of a video camera.

Q. How is it possible to achieve data compression ratios of a factor of 100 or more?
A. End-to-end system optimization is the key to overall performance, which includes the performance capability of the data compression system. A data compression system works best with "clean" data. Clean data requires the removal of instrument artifacts and a good signal-to-noise ratio. The non-linear A/D converter will also help as it will minimize the number of low range or noise counts.  The only information required in each frame is the pixel location which represents the spectral and spatial information, and the "counts" representing the signal level at each pixel.

Q. Are there enough vicarious calibration sites or invariant targets to make empirical reflectance retrieval possible?
A. The FSI approach will significantly broaden the number and nature of targets that can be used for calibration and characterization. Variations in spectral reflectance or "invariant" targets may be taken into account using statistical methods, and by utilizing spectral curve shapes and features rather than absolute reflectances. By making the calibration and characterization process a part of normal data acquisition, large data sets will be available, making the application of statistical methods feasible.

Q. How does empirical reflectance retrieval work in areas where ground truth is not available?
A. Generic "invariant" targets should provide the information necessary. Investigations of "unique" targets may pose a challenge.

Q. Can FSI be applied to staring sensors as well as to pushbroom sensors?
A. Yes; but it may be more difficult with staring sensors to secure adequate registration of the spectral bands.

Q. Can FSI be applied to commercial systems as well as to research systems?
A. Yes. FSI will work best with any high spatial resolution system that collects large amounts of data. Some kind of sensor web system might be advantageous.

Q. The use of spectral curve fitting as a means of extracting and transmitting "all information" from a scene. I assume this means all spectral, spatial and radiometric information at the inherent resolutions (spatial, spectral and radiometric) of the sensor. For current multispectral imagers where we are trying to measure 40-50 parameters from the land, oceans and atmosphere, this appears to be very ambitious. Developing the curves on-board will probably require more computing power than has been previously used. As I indicated in the attachments, the curve development will require calibration (or flat-fielding) prior to the curve fitting. There are probably a host of additional problems, but these will do for now.
A. “Curve fitting” may have become a bit of a red herring as it seems that the best way to do the data compression may be to compress the entire hyperspectral hypercube in one fell swoop. This will eliminate the conception that the fitted curves are tailored to any particular application. The technology to do this is being developed by people who really know what they are doing for a “killer app” other then remote sensing, so it makes sense to take advantage of this work. The computing power required to do this compression will be developed alongside the algorithms and will probably not be much of a strain on currently available computing resources. This development will have to be done in close collaboration with the remote sensing applications developers to be sure that the information they need is preserved. From what I have heard from the pros in the field (commercial remote sensing software developers) this should not be a problem.

 

Q. Empirical reflectance retrieval as a means of bypassing on-board radiometric calibration. My comments in the attachment try to point out that surface features are overlaid with a very dynamic atmosphere and that correcting for this atmosphere is the major challenge in much of earth remote sensing. I do agree with you that much of our data is highly over sampled in the spatial domain, but the community has been unwilling to-date to contemplate this problem.
A. Clearly (sorry about that bad pun) the atmosphere is going to be a major complicating factor in empirical calibration. I am counting on the very large amount of information that will be at our disposal to overcome this problem. Even though the atmosphere is dynamic, it is not entirely unpredictable. Empirical reflectance retrieval is going to be a major exercise in statistics, sprinkled with a bit of artificial intelligence. I am also hoping that the empirical reflectance retrieval system will produce much better information in the cases where the atmospheric contribution to the signal is critical. Hearing the constant moaning and groaning of the ocean types about the difficulty of retrieving accurate reflectances at the blue end of the spectrum is one of the reasons I thought of this idea in the first place.

Q. I talk frequently with Kurt Thome and the group at the University of Arizona. Although they are doing a good job, theirs is only one of several methods used to validate the MODIS radiometric calibration. I believe they would be the first to say that trying to extend their effort globally is impractical. I also mention that most of the newer imagers are producing TOA reflectance as their output (that pesky atmosphere is still in the way).
A. Actually, the key to vicarious calibration or empirical reflectance retrieval as I refer to it, is the global aspect of the endeavor. It relies on the huge amount of information that is available during the normal course of data acquisition. The more information the better.

Q. In my talks with the blue-sky technologists where we are planning for the future, they typically tell me that band-width to the ground is not a problem. Since they are closer to the technology edge than I am, I have to take them at their word. Therefore, a system that is primarily aimed at minimizing the down-link requirements is not attractive to them.
A. The bandwidth to the ground for a FSI system will not be a problem, but I am sure that even the bluest of blue sky technologists would have a seizure if we were to talk about transmitting all of the raw data generated by a high spectral and spatial resolution hyperspectral system.

Q. “Information” is in the eye of the beholder. You need to define what you mean (and once you have a good definition, I suspect that transmitting “all information” will be very nearly equivalent to transmitting all bytes.
A. While I can agree with you that many remote sensing scientists have differing opinions about “information”, there is in fact a rather rigorous definition of Information set forth by Shannon and colleagues many years ago. This is currently the basis of Information Theory and the principle upon which data compression algorithms are based, in general. It is this definition of Information that I am using for my FSI work. I could go into a long discussion of remote sensers definitions of information and how to alleviate most of their concerns (decent signal-to-noise ratios, for starters) but will leave that for another time.

Q. MODIS, VIIRS and others have separate VIS, SWIR and IR focal planes (after the beamsplitters)
A. The FSI system would separate the spectral ranges using a technique (not beamsplitters) that would permit significant wavelength overlap between the spectrometer systems (focal planes). This would provide an advantage over beamsplitter technology. An FSI system would have lots of focal planes, not only to cover the spectral ranges, but also to provide spatial coverage. Compact spectrometers and detectors are relatively cheap, so it pays to use a bunch of them.

Q. [No on-board calibration system] Not an attribute. Vicarious calibration is not good enough for current needs
A. This is a major topic for discussion, and as with Information, I could go on, and on, and on….. Clearly, it is going to take a lot of work to demonstrate that relying solely on vicarious calibration is feasible. One thing that I should mention immediately is that FSI is really intended for high spatial resolution systems like LANDSAT rather than for a moderate resolution system like MODIS. The much greater number of “unmixed pixels” would contribute significantly to the feasibility of vicarious calibration. This topic alone is probably worth several Ph. D. dissertations, and it would probably not be remote sensers who would write them.

Q. Non linear (bi-linear) is being used in VIIRS. Various schemes have been considered during the design process, but the match of performance to requirements has been lacking
A. Square root A/D conversion has been proposed before (I proposed it for the MODIS-T instrument more than 10 years ago, for example) but the technological capability to do it was lacking. Now we have the technology to do it, more or less off-the-shelf. Most people who have reviewed this approach agree that it is well suited for remote sensing. In case you did not see the table that illustrates the utility of square root compression, it is in the Barcelona paper at: http://carstad.gsfc.nasa.gov/topics/JBRESEARCH/BarcelonaFSI.htm

Q. In most multispectral imagers, the cloud data is just as important as the clear data. However, with modern onboard processors, filtering out the cloud data would not be a problem.
A. Exactly. The FSI system would not filter out cloud data, it would simply provide the information that is in cloud data, which in most cases is a whole lot less than is in land data, but more than is in ocean data.

Q. [Flat fielding] Could be done onboard, but would require onboard calibration as each detector has a different radiometric response.
A. Flat fielding and vicarious calibration would be the same process. Both would be done continuously using reference imagery acquired during the normal course of operation. I am hoping that the quality (stability) of the detectors will reduce the need for rigorous monitoring of the flat-field characteristics.

Q. [Flat spectral response] Only across the band. The gain can be set for each band.
A. Flat spectral response is very important for a spectral dispersion type of hyperspectral system. If you do not have a flat spectral response you will have problems with signal-to-noise in some parts of the spectrum. The use of an all-reflective optical system and multiple spectrometers tailored to the wavelength ranges will help to alleviate this problem. Other tricks remain “up the sleeve”.

Q. VIIRS, ABI and others do use both lossy and lossless compression.
A. The data compression algorithms currently employed give compression rations of factors of 2 to 3 and sometimes a little more. I am talking about system data volume reductions of orders of magnitude. This is where there is a significant difference between a FSI system and current designs. Achieving this reduction requires an end-to-end system approach.

Q. Since the atmosphere is dynamic, empirical approaches to achieve surface measurements are difficult without atmospheric data/models. SeaWiFS (and to some extent MODIS) use an empirical fit to buoy data for ocean color parameters, but they also have a major atmospheric correction model. Measuring atmospheric parameters is even more difficult.
A. As I mentioned above, the ocean color guys were one of the inspirations for the FSI concept. Modeling is a very tricky business. What I am proposing would eliminate modeling completely. I realize that this is going to upset some people, but it seems like a better idea.


Comment: I feel that you have bitten off too big of a chunk of this problem. You need to cleanly define "information" and demonstrate the ability to retrieve all of it from a simple image data set using a limited set of curve parameters. This would be done first in a laboratory environment and then (if successful) in an airborne system.
A. I agree that I have bitten off a big chunk, but as I explained earlier, I feel that this is really the best way to approach the problem. The “bits and pieces” approach normally used is not going to produce any significant breakthroughs in remote sensing technology. There is a lot of work to do, but I feel that it is well defined and can be tackled by people who have the appropriate expertise. As I mentioned above, the curve fitting idea will probably turn out not to be a viable approach. I also agree with you completely that the development should be done first in the laboratory, then on airborne systems, and finally flight qualified. This is the way we used to do it and that system worked very well. Along the way, we might even get some useful commercial spin-offs.

Question Sources

FSI Data Rate Calculation

 

This page was last modified on Wednesday, April 04, 2012

 

HOME