Examples
Runnable code examples for every feature of Reader.
Every example below lives in the reader repository under /examples. Clone the repo to follow along:
git clone https://github.com/vakra-dev/reader.git
cd reader
npm install
Then run any example with tsx:
npx tsx examples/basic/basic-scrape.ts
Basics
Scrape a single URL and print the markdown result.
Request markdown and HTML in one call.
Scrape multiple URLs in parallel with progress tracking.
Scale up concurrency and tune the pool for hundreds of URLs.
Discover and scrape every page on a domain.
Tune pool size, recycling, and health checks.
Proxies & bypass
Pass a single proxy per request.
Rotate through a pool with round-robin or random strategy.
Scrape sites behind Cloudflare challenges.
AI & LLM integrations
Pipe scraped markdown to Claude for summarization.
Same pattern with GPT-4 or GPT-4o-mini.
Stream scrape + LLM output with the Vercel AI SDK.
Use Reader as a LangChain document loader.
Use Reader as a LlamaIndex data connector.
Scrape, chunk, embed, and upsert to Pinecone.
Same pattern with Qdrant as the vector store.
Production patterns
Embed Reader in an HTTP API with a persistent client.
Process scrape jobs from a Redis-backed queue with retries.
Patterns for scaling the pool under real load.
Deployment
Production Dockerfile + compose with correct shm_size and capabilities.
Deploy Reader as a serverless function handler.
Run Reader from a Vercel Edge/Node function.