Quality cameras, as we’ve written before, are essential to any self-driving car. There are certain crucial tasks, including lane line detection, reading street signs, and accurately interpreting street lights, that only a camera can make possible. Until now, companies working on autonomous vehicles have mostly been limited to using either small cameras similar to those in smartphones, or standard industrial cameras designed for other applications. Now Sony has produced what might be the first commercially available sensor that is designed specifically for self-driving vehicle projects.
HDR is a must-have, and this sensor makes it a lot simpler
The contrast in a street scene — especially at night — can be more than a standard camera’s sensor can handle. In particular, the contrast between the large amount of illumination coming from a stoplight and the low amount of light on a street sign or emitted from a pedestrian wearing dark clothes can be extremely large. Entering and exiting tunnels is another situation that creates high-contrast scenes. Typically, either multiple cameras, multiple exposures, or potentially some extensive computational imaging inside the camera are needed. Sony claims that its new sensor has 120dB dynamic range and low light sensitivity down to an amazing .1 lux (basically moonlight), which means that it should be able to capture the entire scene in each frame.
Sony describes the sensor as having an “original pixel structure and exposure method.” This strikes me as very similar to the Digital Pixel System, pioneered by Stanford spinout Pixim. Since Sony bought the assets of Pixim a few years ago, that is entirely possible, but I’ve been unable to confirm it. If it is based on the Pixim technology, it works by allowing each pixel to have an independent shutter speed (time when it is read out). So pixels that are gathering more light can shut off sooner, so that the color of the light can be read accurately. The camera can then integrate the various pixels and shutter speeds to create a final HDR image.
LED flicker is another problem unique to vehicles
Many street and other lights are now using LEDs that actually flicker at a rapid rate, one that is too fast for our eyes to notice. Traditional sensors can be fooled by that flicker — misinterpreting the color or intention of the light. You can see the impact of both the HDR capability and LED flicker reduction in this demonstration video from Sony:
The new sensor, officially the Sony IMX390CQV, doesn’t have resolution specs that would impress anyone used to typical cameras. At 2.45MP, it is lower-resolution than even most selfie imagers. However, the limited resolution allows for reasonable size pixels despite the sensor’s small 1/2.7 form factor.
In any case, today’s car computers are limited in how much video they can process, so most systems in development currently don’t employ much more than 2MP (roughly 1080p) cameras. Along with its advanced features, the new sensor is also designed to meet the stringent reliability and safety standards required for deployment in cars. You won’t see one on the road right away though, as while Sony plans on sampling the sensor soon, it won’t be in production until early in 2018.
Source: https://www.extremetech.com
