Researchers at Purdue have developed HADAR, or heat-assisted detection and ranging:
Because thermal waves constantly scatter, infrared cameras capture “ghostlike” images with no depth or texture:
For their experiment, the researchers chose an outdoor space in a marshy area, far from roads and urban illumination. They collected thermal images in the infrared spectrum across almost 100 different frequencies. And just as each pixel in RGB images is encoded by three visible frequencies (R for red, G for green, B for blue), each pixel in the experiment was labeled with three thermal physics attributes, TeX—temperature (T), material fingerprint or emissivity (e), and texture or surface geometry (X). “T and e are reasonably well understood, but the crucial insight about texture is actually in X,” says Jacob. “X is really the many little suns in your scene that’s illuminating your specific area of interest.”
The researchers fed all the collected TeX information into a machine-learning algorithm to generate images with depth and texture. They used what they call TeX decomposition to untangle temperature and emissivity, and recover texture from the heat signal. The decluttered T, e, and X attributes were then used to resolve colors in terms of hue, saturation, and brightness in the same way humans see color. “At nighttime, in pitch darkness, our accuracy was the same when we came back in the daytime and did the ranging and detection with RGB cameras,” Jacob says.
The biggest advantage of HADAR is that it is passive, Jacob adds. “Which means you don’t have to illuminate the scene with a laser, sound waves, or electromagnetic waves. Also, in active approaches like lidar, sonar, or radar, if there are many agents in the scene, there can be a lot of crosstalk between them.”
As a new technology, HADAR is in a fairly nascent stage, Jacob says. At present, data collection requires almost a minute. By comparison, an autonomous vehicle driving at night, for example, would need to image its surroundings in milliseconds. Also, the cameras required for data collection are bulky, pricey, and power hungry: “Great for a scientific demonstration, but not really for kind of widespread adoption,” according to Jacob. The researchers are currently working on these problems, and Jacob predicts another few years of research will be directed to address them.
As computing power and probabilistic processing ascend in power and percolate in distribution, every “sensor problem” begins to look more and more “cryptographic” in that a particular “trick” of computing may more and more readily extract a comprehensible, useful signal from any source not emanating purely random or pseudorandom noise.