PixelRAG - Visual Retrieval That Searches Documents as Screenshots

PixelRAG is an open-source retrieval system that renders documents as screenshots and searches over those images instead of parsed text. Its GitHub repository sums the idea up in one line: “Search any document by how it looks, not just the text it contains.”

The project comes out of the Berkeley Sky Computing Lab, BAIR, and the Berkeley NLP Group, and is released under the Apache-2.0 license. It is written in Python and pairs a rendering pipeline built on Playwright with a vision-language embedding model for retrieval.

Why this project matters

Conventional retrieval-augmented generation converts a page to text chunks before indexing it. That step is lossy in a way that is easy to overlook: tables collapse into runs of numbers, charts disappear entirely, and the layout cues that tell a reader which column a value belongs to are gone.

The repository states the problem plainly: “Text-based RAG parses the page to text chunks and loses the table — the reader can’t find the answer. PixelRAG renders the page to screenshot tiles, retrieves the right tile, and the reader reads the number straight off the image.”

Keeping the pixels means the visual structure survives into the retrieval step. For documents where the information lives in the layout — financial tables, spec sheets, dashboards, infographics, scanned reports — that is a meaningfully different approach than tuning yet another text chunker.

How it works

PixelRAG is built around two operations:

Render turns web pages, PDFs, and images into screenshot tiles using Playwright and the Chrome DevTools Protocol.
Search embeds and retrieves those tiles with Qwen3-VL-Embedding, which the project fine-tunes with LoRA on screenshot data.

Vectors go into FAISS by default, with Qdrant available as an alternative backend for larger deployments, and the search API is served with FastAPI.

Trying it without any setup

The fastest way to see the idea in action is the hosted API, which serves a pre-built index of 8.28M Wikipedia pages. The repository advertises it as requiring “no setup, no API key”:

curl -X POST https://api.pixelrag.ai/search \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}'

The endpoint also accepts an image as the query rather than text, which is the visual-search side of the same index.

Running it locally

Installation is a normal pip install, with extras that map to the stage of the pipeline you need:

pip install pixelrag

That gives you the pixelshot renderer:

pixelshot https://en.wikipedia.org/wiki/Python --output ./tiles

The additional extras are pixelrag[embed] for the embedding pipeline, pixelrag[index] for full indexing, pixelrag[serve] for the search API, pixelrag[pdf] for PDF rendering, and pixelrag[serve,qdrant] for the Qdrant backend.

Building and serving your own index over a local document folder follows the same shape — write a pixelrag.yaml describing the source, the embedding model, and the output directory, then run the pipeline:

pixelrag index build
pixelrag serve --index-dir ./my_index --port 30001

The Claude plugin

PixelRAG also ships a Claude Code plugin called pixelbrowse, which lets Claude screenshot a page and read it visually rather than scraping its text:

uv tool install pixelrag
claude plugin marketplace add StarTrail-org/PixelRAG
claude plugin install pixelbrowse@pixelrag-plugins
claude -p "screenshot https://news.ycombinator.com and summarize the top stories"

This is the lowest-effort way to get something out of the project if you are already a Claude Code user and do not want to build an index at all.

Practical considerations

A few details are worth checking before you commit to it:

Platform support is uneven. The documentation says index building works on Linux with CUDA and on macOS with Apple Silicon and MPS. Rendering is lighter: the project notes that no GPU is required and that it runs on macOS or any machine with Python 3.10+. The bundled turbo headless_shell auto-installs on linux-x64 only, and Windows users need to point at a Chrome binary via CHROME_PATH.
PDF rendering needs poppler in addition to the pdf extra.
Python 3.10 or newer is required.
Training is a separate project. Fine-tuning code lives in the train/ subdirectory as its own uv project, with pre-trained LoRA adapters and the training dataset published on Hugging Face.
The README does not publish benchmark tables. The evaluation lives in the accompanying paper, “PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation,” rather than in the repository itself — so if retrieval quality against your own corpus is the deciding factor, plan to measure it yourself.

Verdict

PixelRAG is a research-backed take on a real weakness in text-first RAG pipelines, packaged well enough to try in a few minutes. The hosted Wikipedia index and the pixelbrowse plugin make the concept easy to evaluate before any infrastructure decision, and the Apache-2.0 license and local indexing path are there if it proves useful on your own documents. If your retrieval problems are mostly about tables, charts, and layout rather than prose, it is a genuinely different angle worth testing.