Assistive Vision for Independent Living

The Problem

Supermarket shelf labels are dense, inconsistent, and full of secondary numbers — unit prices, per-kilo rates, discount badges, promotional stickers. For people with cognitive impairments, low-vision conditions, or aging-related difficulties, picking out the actual price is harder than it looks. Existing assistive apps are either built for smartphones (hard to operate for the target user) or sold at €2,000–4,000 per unit (out of reach for most care institutions).

The founder — a trained social worker with years of field experience — set out to build a purpose-built device that any care client could use with a single button. Offline. No subscription. No account. No personal data leaving the device.

What I'm Building

• Label-layout detection — OpenCV pipeline that identifies shelf-label regions, separates main price from secondary elements (per-unit rates, promotional overlays, barcode strips), and handles the multiple format conventions used across German supermarket chains.
• OCR engine benchmarking — comparative accuracy testing across Tesseract and PaddleOCR engines, with hardware-cost analysis. Tesseract's layout-blindness on dense multi-price labels is a known limit; PaddleOCR handles it cleanly but demands stronger silicon. Recommendation engineering: when to stay on a €20 Pi Zero, when to step to a €45 Pi 4B, when the accuracy ceiling justifies the hardware bump.
• Superscript cents extraction — German retail labels notoriously display cents in smaller superscript form (e.g. 3⁷⁹ for €3.79). Independent ROI extraction for the superscript region, isolated OCR pass, deterministic reassembly — bypasses the engine's tendency to misread the composite glyph group.
• On-device inference tuning — model selection, threading, image-size caps, and memory-budget constraints calibrated to keep inference under the latency budget of a button-press interaction.
• Ground-truth validation harness — per-image ground-truth dataset with a reusable accuracy-scoring framework, so every pipeline iteration can be measured against the same baseline without guesswork.

Why It's Not a Smartphone App

Smartphones can technically do this — Microsoft Seeing AI, Google Lookout, Be My Eyes, Envision Glasses all exist. But the target population doesn't operate smartphones reliably. Multi-step app launches, permission prompts, OS updates that break the flow, notifications pulling attention away — these are failure modes for users with cognitive support needs. A dedicated single-button device is the pattern that works: proven across other assistive categories (medication reminders, GPS trackers, communication boards). The founder is applying that same lesson to the grocery-shopping independence use case, where no purpose-built option currently exists at an affordable price point.

What's Been Delivered

• Multiple OCR pipeline iterations covering dense-grid label detection, close-up single-label processing, and paper-label vs. ESL (electronic shelf label) branching — each measured against ground truth.
• Honest hardware recommendation memo: documented where Tesseract hits a ceiling on multi-price labels and why a hardware bump + PaddleOCR is the pragmatic next step rather than iterating further on a capped engine.
• End-to-end test scaffolding and label-type taxonomy so the founder can continue field-testing independently against a controlled baseline.

Tech Stack

Distribution Path

The product is designed to reach its users through care institutions and social-service distribution channels — not retail shelves. The founder's background in pedagogical and psychological work is the lead-generation engine; my part is shipping a technical pipeline robust enough that those introductions translate into deployed units. The long-term goal is a device that qualifies for assistive-technology reimbursement, so it reaches the people who need it without becoming a luxury good.