John Bolton's Initiatives 1985 - Present
The following is a summary of the most significant projects
that I have initiated during the more than 25 years that I have worked at
Goddard. Though the initiatives are, for the most part, directly related to the
Flight Projects that I was supporting, the work was usually started and
executed on my own initiative. In most cases I collaborated with colleagues at
Goddard, and frequently with people outside Goddard.
The projects are listed in, more or less, chronological order. Appendix A contains copies of DDF Proposals. Appendix B contains selected proposals and related papers.
- Development of optical analysis techniques and integration of thermal, structural, and optical (STOP) analysis capabilities
When working on my first assignment at Goddard, the Solar Optical Telescope (SOT) Project (a precursor to one of the instruments in the ISTP SOHO Project) an analytical procedure was needed for dealing with the hot solar telescope mirrors. A makeshift stand-alone program had been developed, but it was difficult to use and insufficient for the demanding analysis required by the SOT Project. In collaboration with Goddard’s NASTRAN experts and with a commercial vendor of optical analysis software we developed the first integrated structural, thermal, and optical (STOP) design and analysis capability. A paper on the SOT STOP analysis was presented at the SPIE Lens Design Conference, in New Jersey. The following year a paper on telescope mirror STOP analysis was presented at the SPIE Large Telescope Conference, in Tucson. These tools were used extensively at Goddard for several projects. Within a few years, any commercially available full-featured optical design and analysis program contained this capability.
- MODIS end-to-end design and performance analysis
In collaboration with Bill Barnes’ old Code 925 (Remote Sensing Instrument Systems Branch) I modified and extended the FORTRAN code that had been written by Harry Montgomery for passive optical remote sensing instrument performance analysis. The code was adapted to the more convenient spreadsheet format so that entries could be made more easily and so that the results could be readily displayed in graphical format. This program was versatile enough to be used on many instrument systems design projects, and a version is currently being used by the ISAL.
Code 925 went on to contract to have the code “improved”. This eventually led to a completely unusable program that required the purchase of an expensive mathematical analysis program. As far as I know there is no longer any trace of the “improved” program, but the spreadsheet version is still popular.
- Collaboration with Space Telescope Science Institute on Next Generation Space Telescope Design
As a result of work that I had done before coming to Goddard, I had a strong connection with the Hubble Space Telescope organization and many of its scientists. In addition to working on several of the original instruments, I also became involved in the development of what was then called the Next Generation Space Telescope (NGST). At the time, the NGST Science Working Group was calling for a 30-meter full aperture telescope. To come up with a conceptual design that would fulfill this requirement, I worked with several people from the Space Telescope Science Institute (STScI), Goddard optical people, and several contractors. The conceptual design we came up with proposed a 30-meter ultra lightweight mirror that would have been assembled on-orbit, cooled to the operating temperature, and then given its final figure by an ion beam deposition process. This figuring process was under development by one of the contractors at the time. The ultimate figure correction would have been done by a deformable secondary mirror and real-time wavefront measurement system, which was also being developed at that time for ground-based telescopes. A paper "Conceptual Design for a Second Generation Space Telescope" was presented at the SPIE Conference “Space Astronomical Telescopes and Instruments” in Orlando.
Another notable feature of the conceptual NGST design was a separation of the “clean” (quiet) and “dirty” (noisy) parts of the observatory. The telescope and its instruments were effectively decoupled from the data processing and transmission, power and thermal control systems. The results of this work were presented at an SPIE meeting on large optical telescopes.
A considerable number of the concepts developed during this exercise have been incorporated into the JWST. Unfortunately, we are not getting the 30-meter aperture that the scientists originally asked for.
- Develop polarization analysis technology with Russell Chipman (“Mr. Polarization”)
One of the analytical features missing from optical programs at the time was the capability to handle polarization. This is important for both astronomical and for Earth remote sensing instruments. Dr. Chipman had developed stand-alone polarization analysis programs. Working with him and with the same commercial vendor of optical analysis software that we collaborated with on the STOP Project, we developed the first integrated polarization analysis capability. Within a few years, all commercially available full-featured optical design and analysis program contained a polarization analysis capability.
- Manage PRA contract for CODE 900 (Earth scene simulation)
To test instrument models, some kind of scene simulation software was needed. Fortunately, there were several vendors who had developed programs similar to what we needed for the Department of Defense (DoD). A West Coast company, Photon Research Associates (PRA) was willing to collaborate with us to do this development. The product that we came up with was rather crude by today’s standards, as it was limited by the available computing power. Similar programs that are commercially available today can provide extraordinarily detailed scene simulations.
A spin-off benefit of this work is that the software that we developed was also useful for the people developing the science data processing algorithms. Now, in addition to the real data that they had, they could generate artificial input data, which might be more suitable for modeling their analysis and algorithms.
- MODIS-T (Hyperspectral Transfer Radiometer) conceptual design and analysis
One of the facility instruments proposed for the EOS Program was a Hyperspectral Transfer Radiometer (HTR). This would have been a high spectral and spatial resolution instrument that could have been pointed off nadir. Its purpose was to transfer the “characterized” (ground truth) portions of the MODIS-N (now simply known as MODIS) instrument’s swath to the unknown portions. Its primary function was to be calibration. Its secondary function was high spectral and spatial resolution spot (small area) imaging.
The conceptual design that I came up with, in collaboration with a contractor, would have been the first spaceborne hyperspectral imager. It was a simple design, utilizing the current state-of-the-art technology that would have met the science team’s requirements. The conceptual design was thoroughly described in an extensive paper, “A Conceptual Design For An Imaging Spectrometer”. This paper is too long (62 pages) to be incorporated in this document.
The Phase A conceptual design was turned over to a design team that proceeded to add a lot of “features”, including a calibration system (for an instrument for which calibration transfer was the primary function). The increasing cost and complexity of the HTR project finally led to its cancellation.
- Select engineering support team, formalize review guidelines, and complete EOS instrument proposal review (81 proposals)
I joined the EOS Program shortly after it started at Goddard. After participating in reviews for the ISTP Project and non-advocate reviews with Marty Donohoe, Marty asked me to lead the technical review of instrument proposals for the EOS Program. This was a big and important job as there were 81 proposals, and hundreds of scientist’s and engineer’s immediate career plans would depend on our evaluations.
To do the evaluations I assembled a team of Goddard and contractor (Swales) specialists in all of the instrument subsystems. The first thing we did was to develop an evaluation procedure and report format that would be applied to all the proposals. This preparation was done from scratch, as at that time there were no guidelines available for proposal evaluation.
The result of the evaluation was a set of reports that were unprecedented in their scope and credibility. The depth and quality of the information provided amazed the NASA Headquarters instrument selection team, which used the reports as a basis for making several critical selection decisions. Years after the selection process was complete, I met EOS Program instrument managers who were still using the evaluations as guidelines for their work.
- Goddard Research and Study Fellowship with Remote Sensing Group at VTT, Otaniemi, Finland (July, 1991 – July, 1992). Initiate development of the AISA airborne imaging spectrometer
Hoping to utilize some of the experience that I had gained during the ill-fated MODIS-T exercise, I proposed to take advantage of the Goddard Research and Study Fellowship with the Remote Sensing Group at VTT (The Technical Research Center of Finland) in Otaniemi, Finland. The Head of the Remote Sensing Group had told me that they were interested in developing hyperspectral technology. They were one of the few organizations interested in the technology at that time.
With the assistance of a very helpful person in the Goddard personnel office, I managed to sail through the Fellowship application process and made all the arrangements with no hassles. (This in distinct contrast to a similar exercise that I attempted in 2004. It got so bogged down in the bureaucracy, involving everyone from the local ITAR Office to the State Department, that I had to abandon it.)
During the one year that I was in Finland, we developed the plan, made the proposal, secured funding (involving a commercial partnership), found the right people to do the work, built, tested and actually flew and got good results from the prototype airborne hyperspectral instrument. I repeat; all of this was done in one year.
From this experience I learned a lot about what you have to do to actually get something done. Fortunately, before coming to Goddard I had experience in both academia and in industry, so I already had some idea of how to get a job done on time and within budget. One critical thing that I learned in Finland was that in order to get something done in the remote sensing business, it may be necessary to involve people who know nothing whatsoever about remote sensing. The guys who did the “heavy lifting” on the Finnish project were from the Opto-Electronics Division of the VTT and knew nothing about remote sensing before we stated the project. After the successful completion of this first phase of the project, all of the VTT engineers involved decided to leave their positions at VTT and start a small company to build the AISA instrument that we had developed. Their company, SPECIM, is now quite successful, having expanded their product line well beyond the original AISA instrument. They currently employ about 30 people.
When I returned to Goddard after the Fellowship I, and others at Goddard had hopes that the AISA instrument that had been developed might be brought to Goddard. The developers and their commercial sponsor visited Goddard and presented the AISA instrument. While there was considerable enthusiasm among the Goddard technical people, Earth Science management decided that we would not follow-up with this technology. Thus we missed, in my opinion, another extraordinary opportunity to get into the business of hyperspectral imaging at the very beginning. You may recall that the first time we missed this opportunity was when the MODIS-T Project failed.
- Precipitate "Tiger Team" Review of MAMA detectors
Upon my return from Finland I was assigned to work on one of the instruments in the SOHO program that was having technical difficulties. The Science Team was a joint Italian-American (Harvard Smithsonian) group, and Ball Aerospace was building the instrument. As the largest instrument by far on SOHO, we had plenty of mechanical problems, but the problem of most interest to me concerned the detectors. In previous jobs I had gained a lot of experience with this type of detector called the multi-channel multi-anode array, or MAMA. There were also several people at Goddard at that time who knew a lot about the detectors. The problem was that Ball was having a lot of trouble coming up with workable detectors; and they were not particularly open to any suggestions.
In collaboration (collusion might be a better word) with the Dick Fisher, the Head of the Solar Physics Branch we came up with a strategy to break the impasse. This solution was based in part on an experience that I had while serving in the U.S. Army during the Viet Nam war. The strategy was for me, a lowly instrument system engineer, to stand up in the middle of one of the design reviews out at Ball and suggest that we were never going to get a usable detector out of Ball and that we ought to start looking for another vendor.
The result of this action was, fortunately, exactly what we had hoped for. The top managers from Ball went “non-linear” and came to Goddard to berate the Center Director for allowing somebody to come out to a design review and question Ball’s capabilities, much less to suggest that the job might be done better by somebody else. John Kleinberg’s immediate response was to ask Alan Sherman to appoint a “Tiger Team” to look into the situation. I was not a participant on the Team. To make a long story short, the Tiger Team reached the conclusion that Ball was not going to be able to build the detectors. In addition, they agreed that the alternative vendor (that turned out to be the one that I had suggested) would be the best choice. The instrument was built with the suggested alternative detectors (at no additional cost to the Project) and has operated successfully for many years.
- Begin collaborating with Oceans Branch at Wallops (arrange for test flights of AISA)
Rather than be discouraged by the rejection of the work that I had done in Finland by Earth Science management, I decided to see what I could do in collaboration with some of the NASA people who had been looking forward to exploiting the technology that had been developed. Several of these people were in the Oceans Branch at Wallops. Coincidentally, I had also met the president of a small GIS and remote sensing company located in Easton, Maryland who was interested in expanding his business. We quickly developed collaboration between the new Finnish company, (SPECIM, the developers of the AISA instrument), the Easton company (3-D Imaging), and Wallops. At that time it was easy to arrange for “piggyback” flights for experimental instruments at Wallops. This collaboration evolved into a very successful (for the commercial interests at least) development of hyperspectral technology and applications. More details of this collaboration will be described below.
- Propose establishment of CARSTAD in conjunction with Wallops
When Dan Goldin became NASA Administrator, one of the first things he did was to propose to consolidate NASA’s aircraft at Dryden. This would have effectively shut down some of the most useful activities at Wallops. In collaboration with the folks at Wallops, and certain concerned politicians, we developed a plan that we had hoped might help to justify the maintenance of Wallops flight programs. CARSTAD (Center for Airborne Remote Sensing and Technology and Applications Development) was proposed to be located at Wallops and to provide critical services for the development of remote sensing technology and applications. CARSTAD was proposed to:
* Be a source for technological resources for airborne instrument development
* Be a site for testing airborne instruments
* Be a site for developing and maintaining calibration standards
* Provide instrument calibration services
* Maintain a database of remote sensing service providers
* Maintain a database of remote sensing instrumentation
* Provide facilities for outfitting aircraft for remote sensing operations
* Arrange for aircraft/instruments/data processing for experimental remote sensing activities
* Be a source for downlinking remote sensing data (modeled after the Code 935 RAC)
* Provide training in remote sensing image processing
* Serve as an outreach center for remote sensing technology and applications education
* Provide a central site for East Coast remote sensing resources
* Provide specialized or prototype airborne remote sensing services for Government Agencies
* Provide a site for underflight characterization of spaceborne sensors
* Serve as a model for other underflight characterization sites
* Coordinate underflight characterization activities
* Coordinate remote sensing activities with other centers
* Serve as a model for other remote sensing centers
Though several attempts were made to actually establish CARSTAD, in practice CARSTAD never progressed beyond a comprehensive web site that provided a lot of information related to airborne remote sensing. (The CARSTAD web site was recently taken down because of new NASA regulations and requirements). There are some people who still believe that the technology for remote sensing instruments should be developed via airborne platforms. This became the theme of the CARSTAD web site. Further information on CARSTAD can be found at: http://fullspectralimaging.net/CARSTAD.htm
- DDF on Image Correlation Tracker (ICT) stabilized pointing system
Pointing stabilization is a serious problem with airborne pushbroom imaging systems. The typical hyperspectral imager, like the AISA instrument, is a pushbroom system. It acquires data by collecting sequential cross-track scans as the instrument moves over the terrain. If the instrument does not maintain a steady nadir view as it moves, the data will be corrupted.
The solution to this problem adopted by many vendors of pushbroom imaging systems is to record the instrument motions along with the data and then, through post-processing, properly register the data. If this is done in conjunction with spatial location information, such as GPS, the data can also be geolocated. This is not the best solution as resampling of the data is both a tedious process, and it does not necessarily provide full spatial coverage, and it may not be radiometrically correct. The best solution is to control the pointing of the instrument, so that it maintains accurate nadir orientation.
This solution requires both an instrument pointing system and an orientation sensing system. Neither is simple. The instrument pointing system will be discussed later. There are several options for the orientation sensing system. The usual method for sensing orientation is with some kind of gyrosensing system or Inertial Navigation System (INS). The simplest INS is expensive, and if you need high orientation stability, it can get very expensive if only the INS is used.
One way to improve the orientation sensing capability significantly is to incorporate an auxiliary system that uses the scene as a reference. Such a system is known as an Image Correlation Tracker (ICT). The technology for image correlation tracking is highly developed as a result of target tracking demands by the military. This technology is used to lock a weapon system on to a target. A typical application is in a fire control system for a tank, which has to keep its weapon pointed at a target as it moves along uneven terrain.
The same technology can be applied to an instrument stabilization system. Instead of correlating the target with a fixed reference scene, as in the case of the tank, the reference scene can be shifted corresponding to the motion of the scene before the pushbroom imager. If, for example, a small airplane carrying the instrument is flying at 50 meters per second, the reference scene can be shifted at the equivalent of 50 meters per second. Any difference between the acquired scene and the reference scene will then be the pointing error.
The system that was developed for this DDF project consisted of an ICT and an INS. The outputs of each were passed through a Kalman filter to provide a synthesis of the pointing information.
The proof of concept for the ICT system was successful. A simple mechanical system (using a children’s construction set) was built that was able to track a target, thus demonstrating the basic capability of the ICT system.
(Further Information on the Image Correlation Tracker Project)
- Write A Proposal for an Alternative Method for the Development of Earth Observation Science and Technology
This is the initial draft of a paper that I wrote (“And Now for Something Completly Different: A Proposal for an Alternative Method for the Development of Earth Observation Science and Technology”) to open the discussion of an alternative way of doing NASA’s Earth Observing Science. It is based on my experience working for the EOS Project, and on some years of instrument development experience. It is also based on the many conversations I have had regarding alternative methods for the development of technology and applications in connection with my proposal to establish a Center for Airborne Remote Sensing and Technology and Applications Development (CARSTAD). This plan tries to fit in with the stated goals of the NASA Administration and with the strategic plan of the GSFC.
- Develop Hexapod image stabilization system concept
Once we had developed a good orientation sensing system, we needed a system to actually point the instrument under the control of the ICT/INS system. Fortunately, our collaborator on the project, the Goddard Controls Branch, recently had experience with the Stewart Platform or Hexapod. This is the six degree-of-freedom actuator that is commonly seen on flight simulators. For our application, the hexapod was a perfect solution. A three degree-of-freedom system would have been an obvious choice, but it had practical problems. When mounted in a small aircraft, for example, the instrument cannot normally be placed right at the window or viewport. It is therefore a good idea not to move the instrument about its own axis, but to move it about the center point of the window or viewport in the airplane. This assures that the instruments view is not obscured by the edges of the viewport, and, if there is a window on the viewport, that it is always looking through the same area of the window. Mirror systems were also considered, but because of the small size of the instrument, and optical considerations (polarization and distortion), mirrors were not seriously considered.
With some funding that was left over from the ICT Project and additional funding from Goddard Technology Transfer Office, we were able to buy a small hexapod and build the required control systems and computer interfaces. Unfortunately, the original contractor that we got to work on the Project was not able to complete it. Some additional funding was obtained which enabled us to have another contractor demonstrate the feasibility of the ICT/Hexapod system. The Technology Transfer Office made a considerable effort to find a commercial partner for further development of the system, but they were not successful. The possibility for commercialization of the system is still open. (Further information on the Hexapod Project)
- DDF proposal, "A Digital Terrain Mapping (DTM) System Feasibility Study"
Having developed highly stable instrument pointing system and controller, one obvious application (in addition to the original imaging spectrometer) was a stereo imaging system. While in Finland working on the AISA instrument development, I had the opportunity to make the acquaintance of some highly skilled photogrammetrists. These fellows had developed a commercial system for positioning car bodies on an assembly line, using multiple cameras (which, incidentally, they had sold to General Motors Europe, making each of them quite wealthy). I discussed with them the possibility to use two of our imaging spectrometers, one tilted forward and the other aft, to acquire a stereo image. They agreed that if the images from the two instruments could be accurately registered, then they could produce a digital terrain map on the fly (in real time).
Rather than use two imaging spectrometers, I proposed to use two simple line-scan cameras. In practice, these would have been two linear arrays in the focal plane of the same fore-optics. This project started out very well, but several problems led to the failure to complete the development of the system. First, the Controls Branch, with whom I have been working was re-organized and the people with whom I had been collaborating were assigned to new tasks. Second, the Israeli engineer who was at Goddard on an NRC Grant and had provided a lot of the technical guidance for the Project went home and finally, the failure of the search for a commercial partner for the hexapod system, decreased the opportunity to find a commercial partner for the DTM system.
Within a year after the DTM project started, a commercial vendor of aerial imaging systems announced a DTM system very similar to what I had proposed. They (a Swiss company) had collaborated with the German Space Agency (DLR) to utilize the technology developed for a Mars camera. This system is now commercially available, but it is interesting to note that it does not have a good instrument pointing stabilization system.
- Collaborate with commercial remote sensing company and with AgriBusiness companies in the development of hyperspectral applications
A further result of the collaboration between the Finnish company SPECIM with their AISA instrument, the Easton company 3-D Imaging, and Wallops (see “Begin collaborating with Oceans Branch at Wallops”) was a project to evaluate the feasibility of a hyperspectral imaging system for precision farming/agriculture applications. This is an area in which some of the “Big Names” in aerospace (Boeing and TRW to be specific) had attempted to do some business, and had managed to lose multiple millions of dollars. We were looking at a much simpler and limited scope approach. The results of the project were very informative, and more or less defined what you can and cannot do with hyperspectral remote sensing and agriculture. Having the advantage of working with people who really knew what they were doing in the agriculture business (Dupont and Monsanto to be specific) and several researchers from the Beltsville Agricultural Research Center (BARC), we managed to avoid the mistakes that the Big Names in aerospace had made.
One of the conclusions that we reached was that it was not economically, or even technologically feasible to do absolute determinations of crop conditions using a hyperspectral system. We did, however, come up with a technique that was very useful for the AgriBusiness companies. The first step was to do what is called an “unsupervised classification” of the crops. This technique simply divided up the crops into areas with similar spectral characteristics. When this was done, the AgriBusiness companies could send somebody out to see what crop condition was represented by these spectral characteristics. Once the condition was identified for one area, it was done for the entire area that was surveyed. This saved the AgriBusiness company an enormous amount of manual labor. With a little experience with this technique, and a good knowledge of local crop conditions, the crop inspectors could often come to detailed conclusions about crop conditions without even going out into the field.
NOTE: At this time I also worked with several people in the weather monitoring business. The goal was to acquire information, rainfall data in particular, that could provide auxiliary information for crop monitoring. At that time several “weather radar” systems had been installed that had the capability to monitor rainfall rates and amounts quite accurately. Unfortunately, all of this information was employed only in “real-time” and the data was not saved.
It is interesting to note that only very recently has a system for the collection, archiving, and distribution of rainfall data been put into place by the National Weather Service.
- Develop new procedures for characterization and calibration of hyperspectral instruments in conjunction with AISA applications development
One of the biggest problems with any remote sensing system is calibration and characterization of the instrument. This is typically accomplished through the use of sophisticated optical laboratory equipment including NIST traceable illumination standards. This process is expensive and requires the services of calibration experts. It is worth noting that hyperspectral instruments are much easier to characterize than multispectral instruments. This is particularly true of the spectral response characteristics. The reason for this is that all of the light across the reflectance spectrum is collected and simply divided up into “bins”. This eliminates the need for the very tedious characterization of spectral bandpass that is required for multispectral systems. Incidentally, this also makes interpretation of the data products considerably easier as the details of spectral bandpass do not have to be taken into account.
An alternative to the traditional expensive and time-consuming calibration and characterization process is one that is more practical and more directly related to the actual measurement or data acquisition process. For example, instead of using sophisticated optical laboratory test equipment, natural or man-made features such as shorelines and airport runways or roads, could be used to characterize the optical distortion. A complete system for characterization and calibration of the AISA hyperspectral system using real data and inexpensive off-the-shelf components was developed in collaboration with the commercial partners. A significant benefit of this approach is that it could be accomplished without using dedicated characterization and calibration “campaigns”. This function could be accomplished using the data that was acquired during normal operation (though some efforts were made to include optimal targets, such as airport runways). This protocol for characterization and calibration was adopted by the commercial partners with whom I had been working. Once the company had achieved some commercial success, they did acquire some traditional calibration equipment such as a small integrating sphere and wavelength standards. This was done primarily to satisfy the demands of some of their customers for calibration “traceability”.
A paper on new techniques for hyperspectral imager characterization was presented at the ERIM’99 Airborne Remote Sensing Conference, in Ottawa.
NOTE: At the same time as these methods were being developed, we also developed a “downwelling radiance monitor” for the hyperspectral imager. This was based on the simple principle of a “ratioing radiometer” where the ratio of the incoming spectral radiance is compared to the spectral reflectance. This method gives the absolute spectral reflectance directly.
The AISA imaging spectrometer was fitted with a fiber optic illumination collection system between a collector on the top of the airplane and a small spatial portion of the instrument’s spectrometer. This made it possible to normalize measurements that were made under different illumination conditions, or even on overcast days. The technique is now widely used in airborne hyperspectral systems.
- Chair panel discussion on commercial airborne remote sensing at ERIM ’99 (Ottawa, Ontario, Canada)
For several years, the Environmental Research Institute of Michigan (ERIM) sponsored semi-annual conferences on airborne remote sensing. The last one was in 2002. Though the primary audience for these conferences was the photogrammetric and GIS community, there was always a good representation from the environmental remote sensing community. As I had recently done a lot of work with many companies interested in commercial applications of airborne remote sensing when developing hyperspectral technologies and applications, I suggested to the Conference sponsors that we have a session in the form of a panel discussion on the commercialization of remote sensing. ERIM thought that it was a good idea and made me the session chairman.
I convinced seven representatives of the remote sensing community to be panel members. The discussion was very interesting, as at that time several companies were just getting started in the remote sensing business, and none of them were yet making any money. The discussion during that session was very useful for anybody interested in getting into the commercial remote sensing business. Many of the suggestions offered then by remote sensing professionals, are just as valuable today as they were then.
- DDF proposal, "Hyperspectral Imaging Radiometric Transfer System (HIRTS)"
This DDF proposal was a follow-up to the work that I had done previously on the MODIS-T Hyperspectral Transfer Radiometer (HTR). Though this would have been an airborne instrument, its function would have been similar to that of the HTR, with a few added features. As mentioned earlier, one of the primary concerns for any remote sensing mission is calibration. One very useful calibration source is ground truth. Ground truth is simply a target site that has been accurately characterized so that its spectral reflectance properties are well known. The problem is, of course, accurately characterizing the target site. Typically, this is done by tedious measurements using ground based sensor systems. A good airborne hyperspectral system, carefully calibrated, would have made the task much simpler. This would have been the primary function of the HIRTS.
This proposal intended to take advantage of recent developments in sensor technology that would have utilized off-the-shelf components to develop the system. Again, most of the subsystems that would have been required for this project were available from people, companies, and organizations that had nothing to do with remote sensing.
This DDF Proposal was rejected.
- Collaborate on development of NASA/Goddard knowledge capture system and establish TECHDATA web site
With the impending retirement of many of the engineers and scientists who has ‘invented’ the space business, and with the shortage of new hires for them to mentor, the need to capture their expertise became obvious. Several attempts had been made to set up databases into which people could enter information. At best, this was a cumbersome process and not likely to be attractive to busy scientists and engineers. The alternative that I proposed was to set up an internal Internet system or Intranet, that would simply provide general access to on-line documentation. The on-line documentation could be in virtually any form. The only requirement was that it be available to the Intranet. This would have made the task of entering information into the system much easier. The only problem remaining was that of confidentiality and proprietary documentation. Recent developments in file sharing systems and search capabilities would have made this system feasible.
To document this effort and to provide a starting point for the effort, the TECHDATA web site (http://techdata.gsfc.nasa.gov) was established. This web site contained links to the work on file sharing and search engines that could have been employed. It also contained some ‘strawman’ formats for the proposed knowledge capture web site, and introduced the concept of “working on-line”.
After discussion with several parties at Goddard, the initiative was dropped. The Goddard Library did secure some funding to contract with a commercial database provider to try to set up a knowledge capture system. This project started but proved impractical and was eventually dropped. The issue continues to come up at regular intervals, but there is still no good knowledge capture system.
Today, systems for Intranet knowledge capture and retrieval are readily available. Documentation in virtually any form can be entered into or linked to the system. Using off-the-shelf but sophisticated search tools, this information can be easily retrieved.
- Submit DDF Proposal, "A Geolocation System for Remote Sensing Instruments"
One of the most difficult tasks in the processing of remotely sensed data is geolocation. With imagery, this is often done using “tie points” or known location points within a scene to geolocate the entire scene. The obvious problem with the method is that tie points are not always available. When this DDF was proposed, relatively inexpensive GPS systems were becoming available. During the work on the Image Correlation Tracker (ICT) stabilized pointing system DDF, a commercial GPS/INS system was acquired. Though the INS part of this system was of primary interest for that project, we soon realized that by using both the GPS and the INS we could actually geolocate the data without any need for tie points.
This DDF Proposal was rejected.
Not long after this proposal, the Finnish company, SPECIM, started the development of their own GPS/INS based geolocation system that was virtually identical to the system I had proposed for the DDF. Within a year they had developed the system and incorporated it into their AISA hyperspectral imager. This geolocation system is now a standard part of their product line.
- Submit IR&D 2000 proposal on "A Remote Sensing Instrument Suite with Geolocation"
This IR&D proposal was an expansion of the previously mentioned DDF Proposal. The primary addition to the proposal was the “instrument suite”. At that time it was becoming popular to develop groupings of instruments, often in an instrument “pod” that could be easily mounted on an aircraft. The idea was to use the suite of instruments to obtain complementary information. To make best use of this information, the data from each of the instruments would have to be overlaid spatially. To overlay the data, each of the instruments would have to be accurately co-aligned. Finally, all of that co-aligned data would have to be geolocated. This proposal addressed those issues and proposed several innovative ways to accomplish both the co-alignment and the geolocation.
This IR&D proposal was rejected.
- Submit DDF Proposal, "Development of a Remote Sensing Education and Outreach Laboratory"
One of the problems with remote sensing that I recognized shortly after getting into the business was that in order to do anything in remote sensing you had to be some kind of a remote sensing expert. Even though remote sensing is not a particularly intellectually challenging business, one has to be a member of the “inner circle” to even obtain data with which to work. It seemed to me that if remote sensing were to become more successful, and to receive more widespread support from more diverse users, access to remote sensing data and practices would have to be made easier.
In collaboration with the Goddard Education Office and Mike Comberiate’s Coolspace Project (a part of the Special Projects Initiative Office), I proposed the establishment of a Remote Sensing Education and Outreach Laboratory (RSEOL). The Laboratory was intended to provide a location for teachers and students to work on remote sensing projects, to provide access to remotely sensed data, and for the development of new technologies and applications for the processing of remotely sensed data. All of the functions and activities of the laboratory were to be documented on the RSEOL web site.
Even though this DDF proposal was rejected I continued to pursue methods to make access to remotely sensed information more accessible to the “man on the street”. Some of these are described below.
- Submit DDF Proposal, "Development of a Remote Sensing On-Line Web Site"
One of the problems with remote sensing, as mentioned above, is that it is a field that is only well understood by and accessible to a select few people. There is no good reason for this as remote sensing is not exactly “rocket science”. One of the primary problems is that it is very hard to get one’s hands on remotely sensed data. The data is generally not available, and the data that is available is in a format that can only be decoded by those who are experts in the field.
To address this problem, collaboration between Goddard’s EOS Direct Broadcast Project and a commercial vendor of remote sensing software, RSI, was established. The idea was to set up a simple point-and-click interface so that users could select the geographical area in which they were interested. When that area was selected, they could see what Direct Broadcast MODIS data was available. Then, they could either go ahead and use the on-line tools to process the data (assuming that they had the expertise to do that) or they could go to the on-line tutorials and guides that would have showed them how to use the data.
This DDF Proposal was rejected, but because of the collaboration that has already been formed, the EOS Program Office provided some funding to do a feasibility study. This study was completed successfully and the RSOL web site was established. Unfortunately, with the exception of one person at the DAAC who was interested in using the RSOL technology as a front-end to the DAAC data, nobody at Goddard showed any interest in further developing the RSOL. Without any interest being demonstrated, the commercial collaborator decided that they had better things to do.
As a footnote, it is interesting to note that collaboration between Google and NASA/Ames was recently announced. One of the intentions of this collaboration was to put NASA’s remote sensing data on-line. The very popular Google Earth program is virtually identical to RSOL, the difference being that Google Earth uses high spatial resolution imagery while RSOL used high spectral resolution and moderate spatial resolution imagery (MODIS data). It does not take too great a leap of imagination to figure out what Google and Ames intend to do with our remotely sensed data.
- Submit DDF Proposal, "Remote Sensing Collaborative Computing and Web Server System"
Most remote sensing laboratories are small and underfunded operations. One reason for the establishment of the RSEOL was to provide these laboratories with resources that they might not have known were available, or that were too expensive to purchase. Another problem for most small remote sensing laboratories is a lack of computing power. A robust RSOL system would have provided a partial solution to this problem, but there are always specialized (and sometimes proprietary) tasks that have to be done in house. An approach was needed to enhance and optimize a laboratory’s computing capability without simply going out and buying more powerful computers.
When this DDF was proposed, the Gnutella file sharing system was wildly popular, and the first collaborative computing systems were being developed (SETI On-Line being one example). The combination of collaborative computing and file sharing is often referred to as “Grid Computing”. This DDF proposed to do a feasibility study of the potential to build a “mini-grid” for remote sensing laboratories.
This DDF proposal was rejected.
These days, grid computing is all the rage, and has been widely adopted, particularly for very large database systems. Though there has been some talk of developing small-scale grid systems, there are still none available for small remote sensing laboratories.
- Establish and obtain lab space for Remote Sensing Education and Outreach Laboratory (RSEOL)
Even though the RSEOL DDF Proposal was rejected, there was enough interest in the Laboratory to support its establishment. The Goddard Education Office needed a laboratory that could be used by teachers coming in during the summer. At that time advanced high school students regularly visited Goddard, and there was a very active coordinator for these activities in the Education Office. Functional computers were obtained through Goddard Excess Property, and the RSEOL lab and web site (http://rseol.gsfc.nasa.gov) were set up.
The laboratory had a number of networked computers and accessory equipment for acquiring and processing remotely sensed data. One of the computers was used as the RSEOL web server. Due to a lack of continuing interest in the Laboratory and due to recent changes in Goddard IT security policy and in its web site policies, the lab and its web site have been shut down.
NOTE: An enquiry was recently received from the NGO (National Geospatial (and Intelligence) Organization) regarding the RSEOL. The NGO had reviewed the RSEOL web site and was interested in obtaining training for their people in the use of NASA’s remotely sensed products. While the original intention of the RSEOL was provide resources for organizations that could not afford or did not have the capability to obtain them, there is no reason that we could not have provided the same services for another Government agency. When I inquired of Goddard management about the possibility to set up a program to provide the support that the NGO requested, I was told that there was no funding for that kind of activity. After pursuing the matter through several channels, I always received the same response, occasionally with regrets.
- Work with Dr. Nicholas Short Sr. to continue development of the Remote Sensing Tutorial (RST)
In conjunction with the establishment of the RSEOL, I also took over the responsibility for managing the Remote Sensing Tutorial (RST). The Tutorial was developed by Dr. Nicholas Short, as an extension of his LANDSAT Workbook, after he retired from Goddard (Code 935). Code 935 had been maintaining the Tutorial, buried in their own web site. When the RSEOL was established, I approached Code 935 with the proposal to incorporate the Tutorial into the RSEOL, and to give it its own web site. The RST was then established as http://rst.gsfc.nasa.gov.
When I assumed responsibility for the RST, I also established an excellent relationship with Dr. Short who was still actively keeping the RST up-to-date. The amount of work involved with keeping the Tutorial up-to-date was one of the reasons that Code 925 was not reluctant to let it go. For several years I maintained the RST web server and worked with Dr. Short to keep the Tutorial up-to-date. The Tutorial had been incorporated into many college remote sensing courses, and is linked to by many other web sites. Dr. Short and I frequently receive messages of appreciation for the web site.
The original RST server that I established and maintained was recently shut down because of changes in Goddard IT policies, but we have managed to move the web site to one of the LANDSAT Project servers. We are working on bringing it back with all of its features, and are keeping it up to date with Dr. Short’s frequent additions and modifications.
- Develop concepts for "Full Spectral Imaging" remote sensing systems
The basic principles of Full Spectral Imaging (FSI) are quite simple. The whole concept is based on the fundamentals of reflectance spectrometry. FSI is what passive optical remote sensing would have been in the very beginning, if the technological capability to do it were available. FSI was developed in response to all of the problems that have come up in remote sensing, particularly with the recent development of Hyperspectral Imaging (HI). It is important to note that FSI is an end-to-end remote sensing system concept. It deals with everything from the instrument front-end to the data processing, and distribution systems.
FSI is based on the experience that I have acquired working in remote sensing, and particularly in Hyperspectral Imaging, since joining the EOS Program. FSI is the result of the merger of this experience with my “outsider’s” view of remote sensing. As a Physicist and Physical Chemist with a great deal of practical experience in Electro-Optics (Photonics), and instrument system engineering, my view of remote sensing is somewhat different than the typical “remote senser”.
The basic ideas of FSI were presented at the SPIE 10th International Symposium on Remote Sensing in Barcelona, Spain in a paper, “Full Spectral Imaging; A Revisited Approach to Remote Sensing”. The paper was very well received and there were a lot of questions, plus a number of skeptical comments from the established remote sensing community. Since that paper was presented I have had the opportunity to discuss FSI with many people, both inside and outside the traditional remote sensing community. Most of the useful criticisms and suggestions and technological realizations have come from people and organizations that have little to nothing to do with remote sensing. Since Barcelona, Goddard has not supported my work on FSI, and I did not have the opportunity to make a follow-up presentation at the SPIE 11th International Symposium on Remote Sensing. I have, however, been continuing to develop the concepts and have been in touch with most of the people, worldwide, who are involved in hyperspectral imaging and in the development of new technologies for passive optical remote sensing.
The conceptual development is now at a stage where I feel confident that it could be tested by the development of a prototype airborne remote sensing system. I am currently looking for an organization that would be interested in developing a collaborative effort to pursue this work.
- DDF proposal "A Full Spectral Imaging Feasibility Study"
This is the first of two DDF proposals for a Full-Spectral Imaging Feasibility Study. While writing the paper for the Barcelona Symposium, I also wrote this DDF proposal that, if accepted, would have provided some official backing from Goddard for my work on FSI. The proposal described the basic principles of FSI and outlined the work that would be needed to prove its feasibility.
This DDF proposal was rejected.
- Develop conceptual design for next-generation LANDSAT/SPOT instruments
Immediately after LANDSAT-7 was ‘off the drawing boards’ I, and several others at Goddard suggested that we form a small engineering team to begin the conceptual design of LANDSAT-8. For many years is has been apparent that to continue the LANDSAT program the U.S. Government is simply going to have to build, launch, and operate a sequence of LANDSAT instruments. Attempts to acquire “LANDSAT data” from the private sector are doomed to failure. Unfortunately, the initiative to start the design of LANDSAT-8 was not accepted, and the LANDSAT Data Continuity Mission was established.
Rather than simply give up on the design of a next-generation LANDSAT, I continued to follow relevant technological developments and work on a conceptual design. The result was to develop the concept that hyperspectral, and eventually full spectral technology be used for the next-generation LANDSAT. In addition to proposing an advanced technology LANDSAT instrument, I also proposed a complete end-to-end system for remote sensing. As with the FSI conceptual design, the LANDSAT concept is based on the experience that I have gained, and the problems that I have encountered in the more then 15 years that I have been working in the field of remote sensing. Also as with the FSI conceptual design, the LANDSAT concept is and end-to-end system design, dealing with issues all the way from the instrument’s fore-optics to the data acquisition, processing, and distribution systems.
It is very important to remember that this LANDSAT follow-on proposal is for an end-to-end remote sensing system. While the LANDSAT instrument is a key component of the proposal, a lot more is needed to make a successful system. The proposal takes advantage of technology that is currently available “off-the-shelf’, and of technology that has been developed for applications other than remote sensing. It also takes advantage of the infrastructure that has been developed worldwide for the acquisition, processing, and distribution of data.
Though I have frequently proposed the initiation of an in-house LANDSAT technology development project, no progress has been made. There have been some efforts to develop next-generation LANDSAT subsystems. There is an occasional mention of a next-generation hyperspectral LANDSAT by some Goddard scientists, but this has nothing (other than a name) to do with the end-to-end system that I have proposed.
Recent initiatives by organizations outside NASA (and usually outside the U.S.) indicate that there is serious interest in the development of next-generation land remote sensing systems. It is therefore likely that these concepts will eventually be further developed and used.
- Initiate studies of Geostationary Earth Imaging System
After the collapse of STAAC, I was assigned to the EOS Program Office. This is the first time that I had actually been working for a Flight Project. All work that I had done since coming to Goddard was in support of flight projects from the Engineering Directorate. The one ongoing project of interest to me when I came to EOS was the development of a conceptual design for a geostationary Earth imager. This imager was intended to make diurnal measurements at moderate spatial resolution and high spectral resolution.
The scientific requirements for this system were very demanding. Most people do not realize how much more difficult it gets to do remote sensing the farther you are from your target. Fortunately, with my experience in both the “uplooking” and “downlooking” sides of Goddard, I was able to come up with a reasonable conceptual design for the geostationary system. This design took advantage of recent developments in detector technology. The design also minimized the demands on the optical system by only requiring that the optics be large enough to provide adequate spatial resolution. In addition to meeting the scientific requirement, the design that I proposed had the additional advantage of providing a constant time-of-day acquisition of the data. This feature significantly simplified the interpretation of the data, as changes in the solar illumination with time did not have to be taken into account.
The Geostationary Earth Imaging project came to an end when the anticipated ESSP call for proposals was not announced, and was totally abandoned upon the sudden, untimely death of the Project leader, Jan Gervin.
- DDF proposal "Vicarious Calibration of Hyperspectral Sensors"
One of the biggest problems with any remote sensing system is calibration. Calibration is very important in a traditional remote sensing system because absolute radiances must be measured. Absolute radiances must be measured because the whole science of passive optical remote sensing with satellites has developed around the measurement of the radiance at the top of the atmosphere. What you really want to measure is the reflectance at the target. To get reflectance at the target from radiance at the top of the atmosphere, one has to develop a model of the processes that occur between the target and the sensor. This is very difficult. All of these models require data that is radiometrically accurate.
This DDF proposal was a spin-off of the previous work (described above) to ”Develop new procedures for characterization and calibration of hyperspectral instruments in conjunction with AISA applications development”. As mentioned earlier, calibration is a critical aspect of remote sensing. On of the best methods for calibration is the use of ground truth sites, or “vicarious calibration”. In the earlier work, we had used features that appeared during the normal course of data collection, such as airport runways. There were also a lot of features for which we already had a record of their spectral reflectivity. By utilizing this information, calibration of the sensor could be done with information acquired during the course of normal operation. This method is particularly effective with a hyperspectral system.
This DDF proposal was rejected.
- DDF proposal "A Full-Spectral Imaging Feasibility Study"
This is the second of two DDF proposals for a Full-Spectral Imaging Feasibility Study. As a result of the considerable amount of practical experience that I had gained since presenting the Barcelona paper on FSI, there was a lot more to propose for the second FSI feasibility study. In addition, I had got to know a lot of people who were interested in contributing to the effort. Some of the most significant advances had been made by people associated with and funded by the DoD and various branches of the military. Other contributions came from people who had nothing to do with remote sensing, but were engaged in development work on technologies for image processing and storage. Both of these sources of information had significantly better sources for funding than NASA.
This DDF proposal was rejected.
- Develop Empirical Reflectance Retrieval concept
The development of the Empirical Reflectance Retrieval concept evolved from the work that I had done on the Vicarious Calibration of Hyperspectral Sensors, described above. As mentioned earlier, when using the traditional methods for remote sensing, instrument calibration is critical as absolute radiances must be measured. Absolute radiances must be measured so that they can be plugged into the models and so that the reflectance at the target can be “retrieved”. The concept of Empirical Reflectance Retrieval is based on the basic principles of reflectance spectrometry. To measure the spectral reflectance of a sample in the laboratory you simply compare the reflectance of the sample to the reflectance of a known standard sample. Empirical Reflectance Retrieval operates similarly.
Empirical Reflectance Retrieval requires FSI. FSI deals with spectral features rather than absolute radiances. FSI extracts the spectral information from the data. Empirical Reflectance Retrieval is based on spectral features rather than absolute radiances.
Empirical Reflectance Retrieval is made possible by the enormous amount of information that is available from a high spectral and spatial resolution remote sensing system. Imbedded in this wealth of information that is acquired during the course of normal operation, are a lot of targets for which we already know the reflectance characteristics (spectral features). These known targets can be used as the reflectance standards for the retrieval process. This approach has the added advantage of being independent of any atmospheric or illumination considerations.
If adopted, Empirical Reflectance Retrieval would eliminate the need for modeling. It would be particularly effective at the shorter wavelengths where atmospheric effects can provide a majority of the signal at the top of the atmosphere, and where modeling is very difficult. The tools to implement Empirical Reflectance Retrieval currently exist. A practical demonstration of the concept will probably prove difficult, as it would require that a complete end-to-end remote sensing system be devoted to the project.
- DDF Proposal "Empirical Reflectance Retrieval Feasibility Study"
In order to try to obtain some support and develop some interest in the work that I had been doing, I applied for a DDF Proposal to further develop the principles of Empirical Reflectance Retrieval. This proposal suggested a way that the process could have been modeled so that its feasibility, on paper at least, could have been demonstrated. This would have served as the first step in the demonstration of the Empirical Reflectance Retrieval concept.
This DDF proposal was rejected.
- Propose development strategy for the FLORA project
When the Geostationary Earth Imaging System project fell apart some of the science team became involved with the FLORA Project. FLORA was to be the successor to EO-1, the only quasi-operational hyperspectral imager. The science mission was to be the exploration of the Global Carbon Cycle.
I became involved with the FLORA Project by my own invitation. As it was the only project at Goddard that had anything to do with hyperspectral imaging, it seemed appropriate to participate.
Early in the Project is became obvious that decisions had been made by the Science Team that would cause problems with the instrument development. Unfortunately, in my unofficial capacity with respect to the Project, I was not in any position to change any of these decisions. Even though I sought the advice of senior managers at Goddard, and received encouragement to present my ideas, the FLORA Team did not accept my suggestions. It is safe to say that if the proposal development approach and the technology that I had recommended would have been adopted, the Project would have had a much greater chance of a favorable outcome and would have avoided some of the unpleasant situations that occurred. The FLORA Project was not even able to obtain approval from Goddard management.
One of the unfortunate side effects of my participation in the FLORA Project is that the Science Team “cherry picked” some of my ideas on Full Spectral Imaging. In fact, it became more or less standard practice to refer to the FLORA instrument as a “Full Spectral Imager”, using the terminology more or less interchangeably with “Hyperspectral Imager”. The approach proposed by the FLORA Team had virtually nothing to do with my concept of FSI. In addition the Team also started to suggest that the FLORA Mission might be a precursor to a “Hyperspectral LANDSAT”. As mentioned above in the section “Develop conceptual design for next-generation LANDSAT/SPOT instruments”, this is an idea that I have been promoting for more than five years. Again, their “hyperspectral LANDSAT” has nothing to do with my conceptual design for next-generation LANDSAT/SPOT instruments.
- Develop Autonomous Remote Sensing concept
The goal of the autonomous remote sensing system is to provide remotely sensed information to people who do not have the capability to do the analysis that is currently required to utilize the data that is available today. The autonomous system would have the capability to determine what it does not know based on what it does know.
Work that I had done several years before on neural networks as applied to remote sensing systems was revisited after attending a Goddard Engineering Colloquium presented by the developer of a new neural network algorithm. Some work has been done to utilize neural networks for remote sensing, primarily for real-time military target identification.
By taking advantage of the ideas presented in a “conceptual design for next-generation LANDSAT/SPOT instruments” one can readily develop an autonomous system that incorporates neural networks features. Once again, the technology is not the problem. The inter-communication infrastructure exists, neural network algorithms have been developed, and the auxiliary information that would be needed is available.
The key to the autonomous system is the knowledgebase. The knowledgebase consists of all information that is relevant to the remotely acquired information. This includes ground truth, verified remotely sensed information, and auxiliary information. The auxiliary information can include a wide range of items ranging from maps to rainfall totals. This idea was first explored in the initiative, “Collaborate with commercial remote sensing company and with AgriBusiness companies in the development of hyperspectral applications”. In that project auxiliary information pertaining to crops was included in the knowledgebase to provide a better product for the farmers.
- Combine "Full Spectral Imaging", "Empirical Reflectance Retrieval, and "Autonomous Remote Sensing" to describe a new end-to-end passive optical remote sensing system
- Propose Global, Real-time Disaster & Environmental Monitoring System (GRDEMS)
- Collaborate in the development of the Near Realtime Imagery For Disasters Satellites (NEREIDS)
- NASA Retirement
- Promote the Global Real-time Disaster and Environmental Monitoring System (GRDEMS)
- Develop the design for a Next Generation Landsat ( Landsat for the 21st Century )
This page was last modified on 19 January, 2015