TwinStack built a mobile app that finds box labels with the phone camera and has Claude read them into a CSV file. Reading costs about $0.002 per label on Claude Sonnet 5.5, measured on real crops. The working Android prototype was delivered on 20 October 2026.
At a glance
| Industry | Warehousing and logistics |
|---|---|
| Delivered | Working Android prototype, 20 October 2026 |
| Claude products | Claude API (Messages API with structured output) |
| Models | Claude Sonnet 5.5 (default), Claude Opus 5.5 (most accurate) |
| Tech stack | Flutter (Android and iPhone from one codebase), Google ML Kit, Python test tool |
| Cost per label | About $0.002 on Sonnet 5.5 |
The challenge
The client needed to record the details printed on warehouse box labels: item number, barcode, description, quantity, PO number, lot or date, and case position. Typing these by hand while walking along a rack is slow and error-prone.
Proving the idea first
Before building the app, TwinStack built a small Python test tool that sent frames from a walk-through video to Claude. Claude returned structured data for each box it saw, and repeat sightings of the same box were merged. On two sample videos (24 and 17 seconds), the tool found 15 boxes for about $0.27 in total API cost.
The test also showed that moving video is a poor source, because labels are small and often blurry. That finding shaped the app: capture a sharp, close crop of each label instead of sending video.
The solution
The app splits the work so that Claude only ever sees good pictures of labels:
| On the phone (free, offline) | Claude (per call, online) |
|---|---|
| Watches the camera and finds label-shaped areas | Reads the printed text on each label crop |
| Saves a label only when it is in frame, large enough, steady and sharp | Returns item number, barcode digits, description, quantity, PO, lot or date, and case info |
| Straightens and crops the label, and skips labels already saved | Says "unreadable" instead of guessing when a crop is bad |
| Reads QR codes and barcodes exactly (Google ML Kit) | Fills extra columns the user defines, and suggests new ones |
The user walks the rack, reviews the captured crops, taps Read with Claude, confirms the results and exports a CSV file.
How Claude is used
- Batched, structured requests. Crops are sent in batches (8 per call by default) with a strict structured output format, so results always come back as clean data.
- Strict rules against guessing. Claude is told to read exactly what is printed, never to guess or complete digits it can't see, to leave a value empty if it isn't legible, and never to copy values from one image to another.
- Confidence and cross-checks. Each result carries a confidence level (high, medium or low). When Claude's barcode reading matches the phone's own scan, the card is marked "Matches scan", which is strong confirmation.
- User-defined columns. Users can add a column such as "expiry date" for Claude to fill. Claude can also suggest new columns for useful facts it finds, such as weight or country of origin.
- Cost in view. The app shows an estimate before sending, asks for confirmation above $0.50, and shows the real cost afterwards.
- Model choice. Sonnet 5.5 is faster and cheaper, and Opus 5.5 is the most accurate.
Quality
- 162 automated tests pass, covering label detection, the Claude request and reply handling (against a mock server, including retries and bad replies), CSV export and every main screen.
- The app was installed and run on a real Android phone (Redmi Note 11), where it analysed about 10 to 16 frames per second.
- All photos and data stay on the phone, apart from the crops the user chooses to send to Claude.
Results
- Label details are captured into a clean CSV without typing.
- Claude reads each label for about $0.002, because the phone filters out everything except sharp label crops.
- Barcode readings can be verified against an exact on-device scan.
Next steps
The prototype still needs to be validated in a real depot, on a full rack. A production rollout would also move the Claude API key from the app to a server.
Want Claude working inside your business?
TwinStack builds Claude agents, skills and integrations that fit how your team already works, with people kept in the loop where judgement matters.
Talk to Us