Compile, Then Retrieve: An Open-Source Knowledge Base With No Embeddings ​

Most "knowledge-base Q&A" tools follow the same recipe: chunk the documents, compute embeddings, stuff them into a vector store, and at query time pull back a handful of similar fragments to feed the LLM. KaaS makes two choices that break from the mainstream:

1. Compile before you retrieve. No RAG directly over raw fragments. An LLM first compiles the scattered input into structured Markdown wiki articles, and retrieval runs on those.
2. No embeddings. There's no vector store and no similarity math in the retrieval stage. The article catalog goes straight to the LLM, which picks pages and reads them in full, the way a person scans a table of contents.

This piece covers the skeleton of the whole system and a few of the key trade-offs. The details of each mechanism are left to later articles.

The problem ​

KaaS started as an internal tool. Knowledge was scattered across documents, meetings, and email, and every time someone changed roles or left, the context they'd built up walked out with them. Whoever took over spent weeks piecing it back together.

What we wanted was to turn scattered input into a knowledge asset that's readable, editable, and able to evolve over time: Markdown a person can open and read, not a black-box vector store. That decided the shape of the architecture: compile quality comes first, and retrieval is just a thin layer of navigation on top of the compiled output.

What you get ​

For the people using it, the value lands in a few places:

• Scattered notes, documents, and meeting records get compiled into a cleanly classified Markdown wiki with source references, so it accumulates and gets reused over time.
• Ask in natural language, get an answer with citations. The same question doesn't have to be asked a second time.
• The output is plain Markdown: readable, versionable with git, editable by hand. AI writes the first pass, humans correct it, and quality climbs as you use it.
• Deployment is light. docker compose up or a single CLI command, SQLite by default, no heavy vector-store dependency.
• Through the MCP ask tool, Claude Code, Codex, and openclaw can query this wiki directly as a knowledge source.
• Point it at any OpenAI-compatible endpoint. It can run fully local, so data stays in place.

How it works ​

The architecture ​

Three layers:

Storage defaults to SQLite (zero dependencies, go run and it works) and holds the task queue and compile state. Retrieval is pure LLM iteration and depends on no embedding model.

Retrieval without embeddings: let the LLM read the table of contents ​

This is the key difference between KaaS and naive RAG. The compile stage maintains a master-index.md as the full article catalog (title plus summary). At retrieval time there's no vector recall. The catalog goes to the LLM along with the question, the model picks the most relevant article paths, and those articles are read in full as the answer context.

The core is three steps:

store = KBStore(kb_dir, read_only=True)
catalog = store.existing_articles()          # 1. read the master-index catalog
if not catalog:
    return []
meta_by_path = {a.path: a for a in catalog}
selected = _select_relevant(catalog, query, model, max_select=max_articles)  # 2. LLM picks pages
return _read_selected(store, meta_by_path, selected)                         # 3. read full text

Page selection is one structured prompt that asks the model to pick paths from the catalog and return {"paths": [...]}. Selected pages are read in whole, truncated only at MAX_ARTICLE_CHARS = 12_000 to keep the prompt budget in check. Nowhere in the flow is a vector store or embedding model imported.

This works because the compile stage already stripped out the noise: what enters retrieval is structured, deduplicated articles, and the titles and summaries in the catalog are enough for the LLM to navigate. The upside is a light deployment — no chromadb / sentence-transformers / pytorch, a small image, no models to pre-download. The cost: navigating by catalog summary alone, an article that's relevant only deep in its body — with no trace in the title or summary — can be missed. Whether closing this "body-depth recall" gap is worth introducing vector search is left for later evaluation on its merits.

The compile pipeline: Extract → Classify → Write → Index ​

Retrieval can be this light only because the compile is heavy. KaaS's compile is a four-stage pipeline:

• Extract: pull concepts, entities, decisions, and action items from the raw text
• Classify: map the extracted results onto existing articles, or mark them as new
• Write: create or merge Markdown articles
• Index: update the Markdown index (master-index / topic-index)

The server orchestrates Classify→Write→Index into a single pipeline with concurrency and SSE progress. The engineering hard parts are denoising (one stuck LLM call can't drag down the whole batch) and dedup (parallel groups can't each "invent" an article with the same name). Both are covered in the first article of the series, The Four-Stage Compile Pipeline: Compile Before You Retrieve, and How to Denoise and Dedup, so they aren't repeated here.

Plugging into any AI agent: the MCP design of a single ask tool ​

The compiled wiki isn't just for the built-in Web UI — any agent that speaks the Model Context Protocol (Claude Code / Codex / openclaw) can treat it as a queryable knowledge source. The KaaS MCP server exposes exactly one tool, ask:

@mcp.tool()
def ask(query: str, paths: list[str] | None = None, model: str | None = None) -> dict:

It doesn't reimplement the retrieval logic; it reuses the chat core (run_server_chat_http) directly: run one pass of LLM-iterative retrieval, then have the LLM produce an answer with inline [Title](path) citations. MCP's tools/call is request/response and the chat core is streaming, so ask uses a collector to gather the stream events into a complete answer before returning.

Two transports: stdio (the default — the agent spawns kb-ai mcp locally, self-contained) and streamable-http. For remote use the Go backend serves it natively at /mcp, keeping :8080 as the single external origin.

Wrapping up ​

KaaS's core move is to push the cost into the compile stage: do the denoising, dedup, and structuring at compile time, and retrieval gets to be light — no embeddings, just the LLM reading the catalog and pulling full pages. And because the output is Markdown, it can be managed with git, edited by hand, and plugged into any agent as a knowledge source.

Several of the implementation choices follow the same idea: don't reach for complexity when simple works. Retrieval iterates over the catalog with an LLM instead of stacking on a vector store.

Each following piece will dig into one mechanism: denoising and dedup in the compile pipeline (published), why we're confident skipping embeddings, worker concurrency and failure recovery, and the MCP design of the ask tool. The code is all public — if you're curious, browse the GitHub repo.

Acknowledgments ​

KaaS's core idea was inspired by Andrej Karpathy's LLM wiki gist: compile knowledge into a continuously evolving, interlinked wiki that settles and gets reused over time, instead of re-running RAG over the raw data on every question. Thanks to him for laying the pattern out clearly.

Repo ​

https://github.com/bybit-exchange/kaas

Give KaaS a try, and a star if it's useful to you!