My flight got delayed. Already the type to arrive criminally early, I found myself falling down a rabbit hole after a friend called to catch up and we ended up on the topic of research interests. Now I’m on the plane, somewhere over the midwest, and I can’t stop thinking about a system I’ve recently learned about for the first time. It’s not groundbreaking news. It’s not an undiscovered gem. I just can’t shake the feeling that I should have heard about this years ago: event-based cameras.
Let me back up.
The Need for Peripherals
In my early computer organization classes, I learned about how computers talk to the outside world. A CPU on its own is a very isolated thing that performs arithmetic, reads and writes memory, and not much else. To actually be useful, it connects to peripherals: keyboards, mice, displays, cameras, sensors, bluetooth modules. Anything that attaches to the CPU and exchanges information with it. The interesting question is how that exchange happens. There are two main approaches: polling and interrupt-driven I/O.
Polling
Polling is exactly what it sounds like. The CPU periodically asks a peripheral “do you have anything for me?” on a fixed schedule, regardless of whether the peripheral actually has anything to say. The peripheral has no say in the timing; it either has data ready when the CPU comes knocking, or it doesn’t.
The analogy I find most useful here is a bus system. Think of the peripheral as a passenger waiting at a bus stop, and the CPU as the bus organization. Buses run on a fixed schedule. A bus will pull up to the stop at regular intervals, and it will not wait for passengers who haven’t made it there yet. From the bus company’s side, you’re burning fuel and paying drivers to run every route on schedule, regardless of whether anyone is actually waiting. From the passenger’s side, you might have had something ready to send the moment the last bus pulled away, and now you’re standing at the stop waiting for the next one.
This exposes two fundamental problems with polling:
- Wasted compute: The CPU must spend resources issuing requests on schedule, like a kid in the back seat asking “are we there yet?” over and over for the entire car ride.
- Wasted temporal resolution: Data might be ready the moment the last poll ended, but it won’t be picked up until the next scheduled interval.
Fixing one problem tends to worsen the other. Poll more frequently and you get fresher data but burn more compute. Poll less frequently and you save resources but your data gets stale. Some systems try to split the difference with dynamic polling intervals. If the bus keeps arriving at an empty stop, space out the buses; once passengers reappear, send more buses again. This works reasonably well for non-timing-critical systems but still suffers when sensors go quiet and then suddenly have something important to say.
Interrupt-Driven I/O
The alternative flips the model. Instead of the CPU constantly asking peripherals for data, peripherals tell the CPU when they have something to send. The device raises an interrupt, essentially tapping the CPU on the shoulder and saying “hey, pay attention, I’ve got something”, and the CPU acknowledges the request and services it.
To stay in the transit analogy: instead of a bus organization, the CPU is now a taxi company. The peripheral hails a cab when it has something to send, and as long as a taxi is available, it gets picked up quickly. If all the taxis are already in use, the peripheral has to wait a beat, but taxis free up quickly, so even in the worst case the wait is short and the data is still far fresher than it would have been sitting at a bus stop. The CPU is no longer burning resources sending empty vehicles on a fixed loop: it only dispatches when asked.
Modern Peripherals
Interrupt-driven I/O sounds like the obvious choice for timing-critical inputs like keyboards and mice, but modern USB devices actually use polling. The USB host controller asks each device for data at a fixed interval, and the device responds when asked. This is exactly what gaming peripheral manufacturers advertise when they tout high polling rates: a mouse running at 1000 Hz is being queried for its position 1000 times per second. At that rate, the worst-case latency is 1 ms. At 8000 Hz, it is 0.125 ms.
The key insight is that polling fast enough makes the distinction mostly academic. Human reaction times sit in the range of 150 to 250 milliseconds. Even at a relatively modest 125 Hz, the 8 ms polling latency is a rounding error in the context of total system latency. At 1000 Hz or above, it is genuinely imperceptible. The cost of polling at these rates is manageable, and the simplicity of a host-driven bus with predictable, bounded intervals makes it far easier to implement reliably across the enormous variety of hardware combinations in the real world.
It is worth noting that the CPU is not the one doing the polling. The USB host controller handles that work autonomously, querying each device on its schedule without involving the CPU at all. When a device actually has data, the host controller raises an interrupt to the CPU to hand it off. The CPU never burns cycles on empty polls; it only pays attention when something is there. The host controller acts as a middleman, absorbing the overhead of polling and converting it into the interrupt-driven delivery the CPU prefers.
The Rabbit Hole
None of this is new to me. I’ve known about polling and interrupts since my first systems course. What I somehow had never made the connection to until recently is cameras.
For whatever reason, I’ve always just accepted cameras as streaming devices. They send frames. Every fixed interval, the sensor captures the scene and ships a frame of data. This is, functionally, polling: the camera is sampled at a set rate, and whatever the world is doing between samples simply isn’t captured. I never questioned it.
This topic was still rattling around in my head from the phone call at the airport, and somewhere over the midwest it spilled out into conversation with the woman sitting next to me. She hadn’t heard of event-based cameras either. I’ll have to ask around more to figure out whether this is a niche gap in my own knowledge or just a genuinely obscure technology in the broader public consciousness.
Event-Based Cameras
Traditional cameras operate like the polling model: every frame interval, regardless of what’s changed in the scene, the entire sensor is read out and transmitted. Want higher fidelity? Increase the frame rate. Higher frame rate means more data, more bandwidth, more compute, even in scenes where almost nothing is moving. Think about a videoconference call with a mostly static background and a talking head, where the vast majority of pixels in each frame are identical to the last. Codecs do a remarkable job compressing this down, but it’s worth being precise about what a codec actually is: a compression and decompression workflow designed to handle the transport of bulky video data by minimizing bandwidth and compute on the streaming end. The codec is optimizing the output, squashing the representation of what was already captured down to something manageable for delivery. The underlying capture mechanism is still sampling the whole scene at every interval regardless.
Event-based cameras take the interrupt-driven approach. Each pixel in the sensor operates independently. When a pixel detects a change in brightness, it fires an event immediately, timestamped to the microsecond. Pixels that aren’t changing don’t send anything. The result is a sparse, asynchronous stream of change events rather than a dense sequence of full frames.
The advantages are significant. Latency drops dramatically since events are reported the moment they occur rather than at the next frame boundary. Bandwidth drops too since you’re only transmitting data for what actually changed. And instead of being limited to whatever frame rate you’ve budgeted bandwidth for, temporal resolution increases to the point where you’re resolving events at microsecond granularity.
The Cons
For consumer use cases, this is basically irrelevant. If you’re video-calling your family or recording a vlog, the difference between a frame boundary and a microsecond-level event is imperceptible. The whole point of that interaction is for a human to see it, and humans don’t resolve microseconds. The latency of traditional cameras is not on the critical path. Polling, as it happens, is totally fine when you’re designing for human perception since humans won’t notice the improvement and the overhead is trivially manageable.
The same logic from USB peripherals applies here: polling fast enough is sufficient when the consumer is a human. But cameras reveal an important asymmetry that keyboards and mice do not. Pushing polling rates higher is relatively cheap for a mouse, and the cost scales gradually. For a camera, cranking up the frame rate gets expensive quickly. Doubling the frame rate roughly doubles the data, the bandwidth requirements, the storage, and the processing load, and you still do not escape the fundamental constraint that you are sampling the whole scene every interval regardless of how much has changed. Event-based capture is not a free lunch either. A fast-moving, high-activity scene can generate a dense flood of events that demands its own fast interconnects and capable downstream processing. The tradeoff is not polling overhead versus nothing; it is predictable bulk data at fixed intervals versus variable, potentially bursty data with lower latency. For latency minimization specifically, event-based cameras win, but they shift the cost rather than eliminate it. The reason that shift is worth making is the same reason the whole argument exists: the consumer is a computer, not a human, and computers are well-suited to handling fast, sparse, asynchronous data in a way that humans watching a screen simply are not.
The other obvious con is price. Event-based cameras are expensive, and they will stay expensive until something drives investment in manufacturing optimization. No commercial success means no pressure to improve the production process, means no cost reduction, means no commercial success. It’s a loop.
Why Computers Should See at a Computer’s Rate
Here’s where I can’t let it go. For human-facing applications, yes, traditional cameras are fine. But what about computer vision? Autonomous vehicles, robotics, industrial inspection, anything where the point of the camera is not for a human to watch the footage but for a computer to process it in real time?
Computers can execute billions of operations per second and react to inputs orders of magnitude faster than any human reflex. So why are we feeding them video at 30 or 60 frames per second, a rate chosen because that’s roughly what the human visual system finds smooth, and asking them to do high-speed, latency-sensitive work within that constraint? A fast-moving object between frames is simply invisible to the system until the next sample, which is a structural limitation we’ve inherited from cameras designed for human viewing and then applied wholesale to systems that have no reason to be bound by human perceptual limits.
Event-based cameras let the computer see at the rate the scene is actually changing, not at the rate a person would find comfortable to watch. For computer vision, that distinction matters enormously.
Whether it Sticks
The history of technology is full of good ideas that went nowhere and mediocre ideas that took over the world. Modular phones are a clean example of the former: Project Ara had a compelling pitch, real engineering effort, and essentially no commercial traction. The manufacturing and ecosystem complexity never justified the value proposition for most consumers, and it died before it could get cheap. On the other end, the transistor is perhaps the defining example of a technology that found its footing, attracted investment, and was optimized so relentlessly over seventy years that we now put billions of them on a chip the size of a fingernail for a few hundred dollars. Flash memory is another: early SSDs were comically expensive, then they weren’t, because enough people cared enough to push the manufacturing. LEDs followed the same arc, and so did touchscreens.
Event-based cameras are somewhere near the beginning of that curve, if they make it at all. The technology is sound and the use cases are real, but what’s missing is the commercial pull to justify the manufacturing investment that would make them cheap enough to be everywhere. Without a killer application, some robot or vehicle or device that absolutely has to have microsecond-level visual response and can pay the current price premium, there’s no forcing function. Whether autonomous vehicles get there, or industrial robotics, or something nobody has named yet, is genuinely hard to predict. It could just as easily stay a research curiosity: technically elegant, commercially inert, one of the many things that made total sense on paper and never quite found its moment.
I don’t know. It’s a long flight.