Efficient ANPR
Automatic number-plate recognition for Indian traffic video.
Reads plates off traffic video by running OCR exactly once per vehicle instead of once per frame — the tracker, not the detector, is what makes the pipeline affordable. The plate detector reaches 98.3% mAP@50; the tracker is what decides how often it has to matter.
- Two YOLOv8n models trained separately: 20,800 IDD images for the road-object detector, 1,618 for plates.
- Vehicle boxes feed the plate model, plate crops feed PaddleOCR — a cascade, not one big model.
- DeepSORT assigns a persistent ID per vehicle, so a plate is read once and reused across every later frame it appears in.
The problem
The obvious build — run a plate detector and OCR on every frame — is wasteful and wrong at the same time. A car is visible for dozens of frames, so you pay for dozens of OCR calls to answer one question. Worse, OCR is the noisiest stage in the pipeline: motion blur, plate skew, and glare mean consecutive frames of the same plate disagree with each other, and you have no principled way to pick a winner. The interesting problem isn't detection accuracy. It's deciding how often the expensive, unreliable stage needs to run at all.
Constraints
- Free-tier compute only — Colab and Kaggle sessions, no persistent GPU.
- Indian plates: varied fonts, spacing, and state formats, with no clean off-the-shelf dataset.
- Video-rate inference on consumer hardware, not a server rack.
- Two detection problems with wildly different dataset sizes — 20,800 road-scene images against 1,618 plate images.
- Colab kept dropping the session mid-run, so training had to be resumable — the road detector was restarted from `last.pt` eight times.
How it fits together
Video frame
1280×720 @ 29fps
YOLOv8n · road objects
conf 0.40
DeepSORT
persistent track ID
Seen before?
skip if ID resolved
YOLOv8n · plates
conf 0.50, inside vehicle box
PaddleOCR
accept above 0.75
Plate + timestamp per ID
CSV
Decisions and what they cost
Every one of these had a cheaper or more obvious alternative. These are the reasons I didn't take them.
Two cascaded detectors instead of one model with two classes.
Instead ofA single YOLO model trained to detect vehicles and plates as separate classes.
A plate is a small object inside a large one. Searching for it across the full frame wastes capacity on regions that cannot contain a plate. Cropping to the vehicle box first means the plate model sees a much larger effective object at a much better scale — and it let me train the two models on datasets of very different sizes independently.
YOLOv8n — the nano variant — for both stages.
Instead ofYOLOv8s or YOLOv8m for higher mAP.
Two models run per frame, so model cost is paid twice. Nano keeps the cascade inside a real-time budget on free-tier hardware. The tracker also absorbs some of the accuracy loss: a vehicle missed in one frame is usually caught in the next and re-associated to the same track.
DeepSORT tracking, used as a cache key rather than for analytics.
Instead ofFrame-by-frame OCR with deduplication on the output strings; or IOU-only tracking.
Deduplicating strings afterwards means you have already paid for the OCR. Tracking moves the deduplication upstream of the cost. DeepSORT's appearance embedding — rather than pure IOU — is what survives the brief occlusions that are constant in traffic footage.
PaddleOCR over Tesseract.
Instead ofTesseract, EasyOCR, or a custom CRNN trained on plate crops.
Tesseract is built for documents and degrades badly on short, skewed, high-contrast strings. PaddleOCR's detection-plus-recognition pipeline handles the crop-level case without needing me to build and label a character-level dataset I did not have.
What broke
Track IDs churn. A vehicle passing behind another comes out the far side as a new ID, and because the OCR cache is keyed on that ID, the same car gets read again.
FixNot fixed yet, and worth being straight about it: max_age is still at deep-sort-realtime's default of 30 frames. The real fix is either a longer max_age or keying the cache on the recognised string as well as the track ID, so a re-issued ID collapses back onto the plate it already read.
The plate model, trained on tight crops, misfired when handed a full frame during early integration testing.
FixMade the vehicle crop a hard precondition rather than an optimization — the plate model is never called on anything but a vehicle box, which is what its training distribution actually looks like.
Small and distant plates produce nonsense strings, and a track that resolves to one is stuck with it — the cache never revisits an ID it has already answered.
FixA 0.75 confidence floor on the PaddleOCR result. Below it nothing is written to the cache, so the track stays unresolved and waits for a closer frame — which tracking guarantees will arrive, because the vehicle is moving toward the camera.
Results
- Plate detector mAP@50
- 98.3%
- Road detector mAP@50
- 42.4%
- Plate detector recall
- 98.2%
- Detector inference
- 1.5 / 1.7 ms
- Training images
- 1,618 / 20,800
- Plate training run
- 100 epochs
What I'd do next
Two things, in order. First, filter the road detector's output by class before it reaches the tracker — it was trained on all 15 IDD classes and the pipeline currently feeds every detection through, so pedestrians and traffic signs get tracked and searched for number plates they cannot have. That is free compute back for a one-line change. Second, finish training the road detector: 42.4% mAP@50 is seven epochs of a hundred-epoch schedule, cut short by Colab disconnects, and the gap between it and the plate model at 98.3% is entirely a budget artefact rather than anything about the problem. After both, the per-track majority vote — OCR on the best few frames of a track, confidence-weighted — is what I would add before claiming an end-to-end accuracy number at all.