Sony’s new image sensors will make cameras smarter with onboard AI
For years, the evolution of image sensors has been about packing in more pixels and slimming down their size to fit into smaller devices. Now, they’re about to get smart. Today, Sony is unveiling the IMX500 and IMX501, two 12.3-megapixel sensors with onboard AI processing chips. They’ll be able to handle “light” machine learning tasks — like recognizing if a stray dog or cat enters your backyard — on their own, without sending any video to the cloud or another system. Instead, they can deliver anonymous metadata pings to alert you about what they’ve seen.
This may not sound very exciting as first, but as Mark Hanson, Sony’s VP of Technology and Business Innovation told me, it could make cameras significantly more useful. For one, it’s a way to ensure better privacy, since the sensors can handle AI tasks entirely on-device. That’s something that could be particularly useful in Europe, where the new GDPR guidelines severely limit how organizations can use surveillance video. The closer data can stay to their source device, the better.
Onboard AI also means you can get information processed in real-time. A camera at the front of a store could count the number of people entering, and even alert security if someone isn’t wearing a face mask (a major health risk as we’re still dealing with the COVID-19 pandemic). Another camera could keep track of item stock on store shelves, while others could monitor the flow of foot traffic to determine shopping “hot spots.”
To be clear, these image sensors won’t replace the work of sophisticated cloud-based machine learning tools, which can go far beyond just recognizing objects. But they’re a step towards a future where cameras can function more like self-contained computers.
And they can also assist more complex computer vision systems. For example, in a cashier-less store like Amazon’s, where tons of video data is being captured and processed by a slew of cameras, Sony’s sensors could detect exactly where customers are located. If nobody is by the milk aisle, there’s no reason to flip those 3D cameras on, right?
Hanson tells us samples for the IMX500 have already been sent out to Sony’s partners, while the IMX501, which is packaged to fit onto devices like smart boards, will start shipping in June. We can expect to see devices using the new hardware by the end of the year and throughout 2021.
(12)
