How we got AI classification under half a second on a budget Android

The benchmark that mattered to us wasn't accuracy on a server. It was: how long does a driver stand in the rain holding a phone before they get an EWC code back? On a cheap Android, our first version took four seconds. That's three and a half too many.

The constraint: the device in the van

Waste carriers don't issue flagship phones. The real-world device is a budget Android with a slow modem, a mediocre camera, and patchy signal at the yard gate. Optimising for a fast laptop would have been optimising for a customer who doesn't exist.

• Image capture and compression on-device, before anything is sent
• A small on-device model for the obvious cases, server inference only for the hard ones
• Aggressive caching of the EWC lookup so the description renders instantly

Compress before you transmit

The biggest win was the least glamorous: resize and compress the photo on the phone before upload. A 12-megapixel skip photo is wildly more detail than classification needs. Dropping to a sensible resolution cut upload time on a slow connection from seconds to a fraction of one.

Render optimistically

We show the plain-English description and EWC code the instant the model responds, then confirm the hazardous determination a beat later if it needs a second pass. The driver perceives the whole thing as instant because the first useful answer arrives fast.

“Perceived speed is the product. A driver doesn't care about your p99 latency chart — they care whether the screen answered before they got bored.”

— Olivia James, LoadSnap

Where we landed

Median classification on our reference budget handset is now comfortably under half a second from shutter to EWC code on a normal connection, and it degrades gracefully — queuing offline and syncing when signal returns — rather than spinning forever.