Back to the work
AI/ML · Builder

ContextBox.

CLI-first personal knowledge assistant. Captures screenshots with OCR, pulls content from the web, embeds everything with sentence-transformers into SQLite, and answers questions over it from the terminal via GitHub Models.

Role
Builder
When
2024
Stack
Python, Click, SQLite, Sentence-Transformers
Scale
27 modules Python package
View live demo ↗GitHub ↗
ContextBox previewOCR · embeddings · Q&A
27 modulesPython package
all-MiniLM-L6-v2embeddings
3 extrasoptional feature groups
Local-firstSQLite store

The problem

The things you reference while working, a screenshot, a web page, a Wikipedia entry, a YouTube transcript, scatter across apps and are gone the moment you close the tab. The goal was a single command-line tool that captures all of it, indexes it for meaning rather than keywords, and lets you ask questions over everything you have seen, without leaving the terminal.

What it does

  1. Cross-platform screenshot capture with Tesseract OCR, detecting Wayland, X11, GNOME, or macOS and falling back across gnome-screenshot, scrot, and grim.
  2. Pluggable content extractors for web pages, Wikipedia, and YouTube transcripts, each behind an availability guard so a missing dependency disables just that extractor.
  3. Semantic search with sentence-transformers (all-MiniLM-L6-v2) and cosine similarity over captures stored in SQLite, with a keyword fallback when the model is not installed.
  4. Question-answering over captured content via GitHub Models, with question-type classification, source attribution, and confidence levels, plus a privacy mode that encrypts sensitive captures with Fernet.

Impact

  • A 27-module Python package (Click + Rich CLI) covering capture, extraction, semantic search, and AI Q&A, installable via pip with optional feature groups ([llm], [ocr], [youtube]).
  • Graceful degradation throughout: heavy dependencies (sentence-transformers, OCR, YouTube) are optional, and each subsystem disables itself cleanly when its dependency is absent.
  • Ships with a React documentation site (install, commands, API reference, demo) deployed on GitHub Pages, and is MIT-licensed.