Following the previous experiment on grading answers, I thought I’d play around with even more local VLMs by running one directly in clients’ browsers.
With the help of Fable, I built a purely client-side pipeline that uses a mix of classic computer vision techniques and a VLM to grade multiple-choice questions. Try it: gauravmm.github.io/answer-sheet.
The pipeline is simple. Classical computer vision:
- deskews the image,
- locates the grid from the answer boxes’ own printed edges, then
- classify each cell by how much ink is inside it.
- infers the coloration.
The only ML in the product is a small handwriting model for the three header fields.
At first, Fable (at my direction) attempted to use Granite-Docling. It could read the structure perfectly but couldn’t reliably say which column each mark was in. VLMs in general are known to be weak at exactly the spatial-grounding job a dense grid demands. After testing it, Fable switched to a classical pipeline. On its test cases, it scores every mark correctly, in milliseconds, at a fraction of the download.
If I had to use a VLM, I would likely label each cell uniquely, removing the spatial-grounding requirement entirely.
The source, the sheet, and the full spec are on GitHub.
 and feed it to the app.](demo-sheet.webp)