Synapse RAG
Synapse is a multimodal document intelligence API built for grounded question-answering across PDFs, DOCX, TXT, and image files. It features an Agentic RAG mode with a ReAct-style agent capable of multi-step reasoning and dynamic tool use, alongside a standard RAG pipeline with hybrid retrieval, reranking, and citation-rich answers through a FastAPI backend.
The system is designed with production-grade features from the start: API key authentication, rate limiting, daily query quotas, usage analytics, metadata filters at query time, session exports, structured per-file ingestion outcomes, SSE streaming, and a lightweight evaluation harness with exact match, token F1, grounding score, and source recall metrics. It uses PostgreSQL with pgvector for retrieval, is containerized with Docker, tested with CI via GitHub Actions, and deployed on Hugging Face Spaces.
KEY CAPABILITIES
- Multimodal ingestion for PDF, DOCX, TXT, and image files with RapidOCR and optional table extraction
- Agentic RAG mode with ReAct-style agent supporting multi-step reasoning and dynamic tool use (retrieve, compare, summarize, refine)
- Hybrid retrieval with vector search, keyword search, Cohere/local reranking, dynamic top-k, and MMR diversification
- Grounded answers with citations, query rewriting, strict grounding guardrails, and SSE streaming
- Async ingestion pipeline with structured per-file outcomes and five-state status tracking (queued, processing, ready, ready_with_warnings, failed)
- Lightweight evaluation harness with exact match, token F1, grounding score, and source recall metrics
- API key auth, rate limiting, daily query quotas, usage analytics, feedback endpoint, and session export tools
PROBLEM
- Document Q&A tools often break down once files come from different formats, include scans, or require reliable citations backed by multi-step reasoning.
- Many RAG demos feel good in notebooks but are missing the operational pieces needed for real usage such as auth, quotas, ingestion feedback, evaluation, and monitoring.
CONSTRAINTS
- The system had to support mixed document types, OCR-heavy inputs, agentic reasoning, and retrieval quality without turning the stack into something overly hard to run locally.
- I wanted it to feel closer to a product than a demo, so auth, quotas, observability, evaluation, structured ingestion outcomes, and CI had to be part of the design from the beginning.
MY ROLE
- Owned the end-to-end system design, backend implementation, retrieval pipeline, agent orchestration, evaluation harness, CI pipeline, and deployment to Hugging Face Spaces.
- Designed the ingestion workflow, retrieval strategy, agentic RAG mode, schema choices, and the full API surface for querying, analytics, and export.
ENGINEERING CALLS
- I chose an async ingestion pipeline with structured per-file outcomes and five-state status tracking so uploads stay inspectable instead of failing as a black box.
- I combined vector retrieval, keyword retrieval, Cohere reranking, and MMR because retrieval quality mattered more here than keeping the implementation minimal.
- I added a ReAct-style agentic mode so the system could handle questions requiring multi-step reasoning, comparison, and summarization beyond single-pass retrieval.
- I treated auth, rate limiting, daily quotas, evaluation, and CI as first-class features so the project reflects how an AI backend behaves in production.
WHY THIS STACK
- FastAPI fit well because the project needed a clear API surface, async workflows, SSE streaming, and good control over ingestion and query endpoints.
- PostgreSQL plus pgvector gave me semantic retrieval without introducing another database system, which kept the infrastructure simpler and more portable.
- LangChain helped move faster on retrieval and agent orchestration, while the ReAct agent added multi-step reasoning without overcomplicating the architecture.
- GitHub Actions provided automated CI with linting, type checking, and tests on every push, ensuring code quality as the system grew.
Flow from document upload and ingestion to agentic question answering with hybrid retrieval
INPUT -> SYSTEM -> OUTPUT
- 1
Input
Users upload PDFs, DOCX, TXT, or images via the async pipeline, then ask questions with optional metadata filters, agent mode, and query controls.
- 2
System
Synapse runs OCR-aware ingestion, hybrid retrieval with reranking, and either standard grounding or a ReAct agent with multi-step tool use before generating the answer.
- 3
Output
The API returns citation-backed answers (streamed via SSE or non-streaming), session status, usage analytics, and export tools.
AGENTIC RAG OUTPUT
{
"question": "Summarize key risks and compare with Q1",
"answer": "The primary risk shifted from supply-chain delays in Q1 to regulatory compliance costs in Q2, with a 15% increase in projected mitigation spend.",
"citations": [
{ "chunk_id": "chunk_abc", "document_id": "doc456", "source": "risk-report.pdf", "page": 3 },
{ "chunk_id": "chunk_def", "document_id": "doc789", "source": "q1-summary.pdf", "page": 7 }
],
"meta": {
"retrieval": "hybrid + rerank",
"agent_mode": true,
"agent_steps": 3,
"grounded": true
}
}A realistic example of the agentic answer shape with multi-step reasoning, grounding metadata, and source citations.
WATCH ON YOUTUBE
This project proves I can build a production-grade Agentic RAG backend by combining multimodal ingestion, hybrid retrieval, multi-step agent reasoning, evaluation, CI, and operational controls in one deployed system.
5 formats
Input coverage
Supports PDF, DOCX, TXT, and image inputs (PNG/JPG/JPEG/WEBP) with RapidOCR.
Hybrid + rerank + agent
Retrieval strategy
Uses vector search, keyword search, reranking, MMR, and a ReAct agent for multi-step queries.
Eval + CI
Quality assurance
Includes an evaluation harness (exact match, token F1, grounding, source recall) and GitHub Actions CI with linting, type checks, and tests.
BEFORE
A typical RAG demo can answer questions from a single retrieval pass, but often has fragile ingestion, no agentic reasoning, weak grounding, and no operational visibility.
AFTER
Synapse behaves as a deployable Agentic RAG backend with multimodal ingestion, multi-step reasoning, grounded streaming responses, evaluation metrics, CI, and product-style controls.
IMPACT — The project shows not just prompt and retrieval usage, but architecture thinking across agent design, evaluation, API design, CI, and production ergonomics.
RESULTS
- Delivered a working multimodal Agentic RAG product with grounded answers, citations, streaming, session exports, evaluation metrics, and production-oriented controls.
- Turned a raw LLM prototype into a backend that is easier to evaluate, extend, and deploy, with a live API on Hugging Face Spaces and CI running on every commit.
NEXT IMPROVEMENTS
- I would add distributed background workers for ingestion instead of in-process tasks, and streamed uploads to reduce memory pressure on large files.
- I would expand the evaluation harness into an automated regression suite tied to CI so retrieval and agent quality changes can be caught before deployment.
HONEST TRADEOFFS
- Kept the infrastructure simpler with PostgreSQL plus pgvector instead of splitting retrieval across multiple storage systems, accepting some scale limits in exchange for portability.
- Accepted extra pipeline complexity by adding agentic reasoning and hybrid retrieval in exchange for significantly better answer quality on multi-step questions.
- Used in-process background tasks for async ingestion instead of a distributed worker, keeping the system easy to run locally at the cost of horizontal scalability.