High-Definition Time-of-Flight Imaging
Time-of-flight (TOF) three-dimensional (3D) imaging provides a complimentary fit for the LCPG steering technology. TOF cameras and flash lidars use a focal plane array (FPA) to simultaneously detect the return from thousands of locations in the receiver’s field of view (FOV). Large FOVs typically require diverging beams and wide-angle optics that reduce the amount of signal collected relative to background noise. Meanwhile, the angular resolution is limited by the resolution of the TOF FPA.
Using LCPGs, a TOF camera can concentrate illumination and signal collection over a narrow angle for high signal-to-noise ratio (SNR) and angular resolution, then non-mechanically scan both transmitter and receiver to regain a large FOV and high effective pixel count. We demonstrated this approach with a commercial TOF camera to boost range and resolution while reducing power consumption.
- Automotive Sensors
- Manufacturing Inspection and Factory Automation
- Aerospace + Defense
The Challenge
A TOF 3D imaging system generally illuminates a sizable area with modulated laser light and uses a FPA to simultaneously detect the return from thousands of locations in the receiver’s FOV. Thousands of points, from a QVGA (320x240) TOF detector for example, produce a dense 3D point cloud, but the 3D image’s spatial resolution is low if the points are distributed over a reasonably wide field of view. High-resolution point cloud generation over a wide angle presents problems for a TOF camera (or flash lidar), since more spatial resolution generally means more pixels within the array and thus less return signal per detector element. Also, wider view angles cause more ambient (background) light to be collected along with the TOF signal, which increases shot noise even if the ambient noise is rejected through electronic processing.
These signal-to-noise issues along with size, weight, power, and cost constraints usually force a tradeoff of resolution for coverage. Fortunately, a TOF camera (or flash lidar) can provide higher resolution and better SNR by narrowing the FOV of the camera, and the LCPG scanner can step both the transmit beam and receiver FOV to gain back area coverage.
A native QVGA TOF camera using 4 VCSELs covers a 60° × 45° FOV with 320 × 240 pixels.
Using LCPGs, the same TOF camera covers a 44° × 5.1° FOV with 2080 × 240 effective pixels. The modification boosts angular resolution by nearly 10×, reduces background noise by up to 78×, and reduces required laser power by 4×.
The Outcome
We modified a commercial QVGA TOF camera by changing the receiver optics to a larger aperture collection lens with a ~5.5° FOV. The transmitter was modified to use one of the four available vertical cavity surface emitting lasers (VCSELs), which was condensed to a ~5.5° divergence and converted to a single polarization state. For this simple proof-of-concept, a 1-D linear LCPG scanner was integrated to provide a non-mechanical steering of both transmit and receive paths over 44° in 5.5° steps.
- 9.5x
improvement of point cloud resolution after modifications - 4.5x
improvement of depth accuracy - 4x
improvement of VSCEL after reducing VCSELs from 4 to 1
Using the LCPG scanner, the TOF camera could now capture a series of individual 5.5°×5.1° FOV point clouds spanning a 44° field. Stitching together the point clouds provides a continuous 2080×240 resolution over a 44°×5.1° FOV. This represented an improvement of nearly 10× angular resolution. Put another way, the original camera had roughly the equivalent of 20/200 visual acuity and could be considered legally blind, whereas the LCPG-equipped camera captured the scene with 20/20 vision. In addition, the reduced background noise collection and increased illumination irradiance increased depth accuracy by ~4.5×, while only using 1/4th the original illumination power.
Our Approach
To examine this concept, we modified a commercial-off-the-shelf (COTS) QVGA TOF camera (EVK75123-60-850-1, Melexis) to reduce the FOV of the illumination and receiver to boost angular resolution and SNR. We then incorporated a 3-stage (i.e., 8-angle) LCPG non-mechanical scanner to steer both illumination and receiver out to recover a wide FOV. By characterizing the system at each step, it was possible to quantify the impact of the LCPG scanner on the TOF camera.
-
Characterize the Native TOF Camera
Baseline images were acquired of a variety of indoor targets to establish the starting performance of the native camera. Using a variety of indoor objects with different absorption and reflection characteristics, we acquired images with the native TOF camera. The camera captured a 60°×45° FOV with 320×240 resolution, corresponding to roughly 0.19°×0.19° angular resolution. The SNR and depth accuracy were measured at an imaging distance of approximately 2 meters.
-
-
Modify camera optics to narrow the illumination beam and sensor FOV
The illumination field and sensor FOV were reduced to ~5.5° by modifying the camera optics. For later integration with the LCPG scanner, the illumination was polarized and a linear polarizer was incorporated into the receiver path. The modified camera was then characterized with the same indoor targets. To narrow the illumination angle, first 3 of the 4 VCSELs were covered. This made it simple to incorporate a condenser lens to narrow the illumination field to 5.5°, though illumination was reduced by slightly more than 4×. A series of polarizing beam cubes and a halfwave plate were used to vertically polarize the unpolarized light from the remaining VCSEL without discarding half of it. This polarization of the emitted light is required for efficient operation with LCPGs.
For the receiver path, a new narrow-angle (~6.75°) one-inch receiver lens was used to replace the original wide-angle (60°) lens. The new narrow-angle lens provided approximately ten times the area of the original wide-angle lens.
When imaging the indoor targets, we observed a 6.4 dB increase in SNR with the narrow-field optics. This roughly corresponds to a 10x increase in aperture area, 4x decrease in output power, 2x decease in signal return due to polarization, and 4x decrease in shot noise due the polarizer and the smaller detector angular subtense (DAS). Therefore, in this implementation, the increased aperture more than offsets the use of less illumination and the polarization-related losses, while the reduction in background noise presents a net benefit to the system.
-
-
Integrate the LCPG scanner
The LCPG scanner was incorporated to steer both the transmit and receive paths, and custom software was implemented to synchronize TOF image acquisition with the non-mechanical scanner. The resulting stitched TOF point clouds of the indoor targets were then analyzed for comparison. We built a simple single-axis LCPG scanner to step the 5.5°×5.1° FOV out over a 44°×5.1° total field. The LCPG scanner consisted of three “stages”, each consisting of one active liquid crystal variable retarder cell and one passive polymer LCPG. The steering angles of each stage were ±2.75°, ±5.5°, and ±11° at the VCSEL’s center wavelength of 850 nm.
We wrote custom software to coordinate the LCPG scanner and ToF camera, resulting in stitched point clouds with 2080×240 resolution, and then imaged the same indoor targets. Compared to the native camera, the non-mechanically scanned camera had nearly 10× better angular resolution, 4.5× better depth accuracy, and used 4× less illumination power.
-
The Final Results
To demonstrate one of the compelling applications of LCPG non-mechanical beam steering, we equipped a COTS TOF camera with a LCPG scanner and characterized the impact on camera performance. By replacing the wide-angle optics with narrow-angle optics and an LCPG scanner, angular resolution was vastly increased. Furthermore, SNR was improved despite using ¼ the illumination power and relying on polarized light.
| Native ToF Camera | HD Modification using Non-mechanical Scanner | |
|---|---|---|
| Point Cloud Resolution | 320×240 | 2080×240 |
| Angular Coverage | 60°×45° | 44°×5.1° |
| Spatial Resolution | 0.19° per pixel ~1 pixel/cm @ 3 meters |
0.02° per pixel ~9 pixel/cm @ 3 meters 10× Improvement |
| Depth Accuracy | ~5.5 cm @ 2.7 meters with target reflectivity of ~50% | ~1.26 cm under same conditions 4-5× Improvement |
| Background Noise | 120 klux sunlight rejection per datasheet Detector Angular Subtense (DAS): 0.19°×19° | DAS reduced to 0.02°×0.02° Up to 78× improvement in outdoor conditions Narrow FOV enables further 20× reduction (i.e. 1560× total) by reducing the bandwidth of the spectral bandpass filter to ~10 nm |
| Tx Power Reduction | 4 synchronously pulsed VCSELS (≤ 25W) | 1 VCSEL 4× Improvement |
The power of this approach stems from the ability of LCPGs to scan both the transmitter and receiver paths of the sensor. This is enabled by the large apertures and angles provided by LCPG steering. Meanwhile, the tiling of a monochromatic sensor FOV provides a good match for the discrete steering nature of LCPGs.
