Build a Codebase Q&A Tool with LlamaIndex & Supabase pgvector

What We're Building

A CLI tool that takes a local Git repository, splits every source file into semantic chunks, embeds them with a free LLM, stores the vectors in Supabase's pgvector (free tier), and then answers natural-language questions like "How is authentication middleware wired up?" or "Where is the database connection pool configured?" — with citations pointing to the exact files.

Feature List

• Local repo scanner: walks a directory, respecting .gitignore so you don't index node_modules or build artifacts
• Smart chunking: LlamaIndex's CodeSplitter respects language boundaries — functions, classes, and logical blocks stay intact
• Free embeddings: uses OpenRouter's free models (e.g., nomic-embed-text) so you pay $0
• Supabase pgvector storage: free-tier project with up to 500MB of vectors, plenty for a medium codebase
• Retrieval-augmented Q&A: top-k semantic search over code chunks, then a free LLM synthesizes an answer with file references
• Source citations: every answer includes the file path and line range, so you can jump straight to the code

Architecture Overview

The ingestion pipeline reads your repo, chunks it, embeds each chunk, and pushes everything to pgvector. The query pipeline embeds your question, fetches the most relevant chunks, and feeds them to an LLM with a prompt that insists on citing source files.

Prerequisites (All Free)

Sign-up steps:

1. Create a Supabase project — note your project URL and service_role key (or anon key if you set RLS policies, but for a local CLI, service_role is simpler).
2. Enable pgvector: in the Supabase SQL editor, run CREATE EXTENSION IF NOT EXISTS vector;.
3. Grab an OpenRouter API key — the free nomic-embed-text model for embeddings and google/gemini-2.0-flash-lite-preview-02-05:free or meta-llama/llama-3.2-3b-instruct:free for chat.

Project Setup & Dependencies

mkdir codebase-qa && cd codebase-qa
python -m venv .venv && source .venv/bin/activate  # Windows: .venv\Scripts\activate

Create requirements.txt:

llama-index==0.12.22
llama-index-vector-stores-supabase==0.3.3
llama-index-embeddings-openrouter==0.1.0
llama-index-llms-openrouter==0.2.0
supabase==2.7.1
python-dotenv==1.0.1
tree-sitter==0.23.0
tree-sitter-languages==1.10.2

Install:

pip install -r requirements.txt

Create .env:

OPENROUTER_API_KEY=sk-or-v1-your-key-here
SUPABASE_URL=https://your-project-id.supabase.co
SUPABASE_SERVICE_KEY=eyJhbGciOi...your-service-role-key

Ingesting the Codebase

Create ingest.py:

import os
from pathlib import Path
from dotenv import load_dotenv

load_dotenv()

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
    Settings,
)
from llama_index.core.node_parser import CodeSplitter
from llama_index.embeddings.openrouter import OpenRouterEmbedding
from llama_index.vector_stores.supabase import SupabaseVectorStore
from supabase import create_client

# --- Config ---
REPO_PATH = "/absolute/path/to/your/repo"  # Change this
TABLE_NAME = "code_chunks"
EMBED_MODEL = "nomic-ai/nomic-embed-text-v1.5:free"
FILE_EXTENSIONS = [".py", ".ts", ".tsx", ".js", ".jsx", ".go", ".rs", ".java", ".rb"]

# --- Init Supabase ---
supabase = create_client(
    os.environ["SUPABASE_URL"],
    os.environ["SUPABASE_SERVICE_KEY"],
)

# Create table if not exists (idempotent)
supabase.schema("public").query("""
    CREATE TABLE IF NOT EXISTS code_chunks (
        id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
        text text NOT NULL,
        metadata jsonb DEFAULT '{}'::jsonb,
        embedding vector(768)
    );
    CREATE INDEX IF NOT EXISTS code_chunks_embedding_idx
        ON code_chunks USING ivfflat (embedding vector_cosine_ops)
        WITH (lists = 100);
""").execute()

# --- Configure LlamaIndex ---
Settings.embed_model = OpenRouterEmbedding(
    model=EMBED_MODEL,
    api_key=os.environ["OPENROUTER_API_KEY"],
)

# --- Load and chunk ---
print(f"Scanning {REPO_PATH}...")
reader = SimpleDirectoryReader(
    input_dir=REPO_PATH,
    recursive=True,
    required_exts=FILE_EXTENSIONS,
    exclude_hidden=True,
)
documents = reader.load_data()
print(f"Loaded {len(documents)} files.")

splitter = CodeSplitter(
    language="python",  # LlamaIndex auto-detects per file; this is the fallback
    chunk_lines=40,
    chunk_lines_overlap=15,
    max_chars=1500,
)
nodes = splitter.get_nodes_from_documents(documents)
print(f"Created {len(nodes)} chunks.")

# --- Embed and store ---
vector_store = SupabaseVectorStore(
    supabase_client=supabase,
    table_name=TABLE_NAME,
    embedding_dimension=768,
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

print("Embedding and storing (this may take a few minutes on large repos)...")
index = VectorStoreIndex(
    nodes,
    storage_context=storage_context,
    show_progress=True,
)
print(f"Done. Indexed {len(nodes)} chunks into Supabase.")

What's happening: SimpleDirectoryReader walks the repo, CodeSplitter breaks files into semantic chunks using tree-sitter (AST-aware, not just line counts), OpenRouterEmbedding hits the free embedding endpoint, and SupabaseVectorStore pushes vectors into pgvector with an IVFFlat index for fast cosine-similarity search.

Querying the Codebase

Create query.py:

import os
from dotenv import load_dotenv

load_dotenv()

from llama_index.core import VectorStoreIndex, Settings
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.embeddings.openrouter import OpenRouterEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.supabase import SupabaseVectorStore
from supabase import create_client

# --- Config ---
TABLE_NAME = "code_chunks"
EMBED_MODEL = "nomic-ai/nomic-embed-text-v1.5:free"
CHAT_MODEL = "google/gemini-2.0-flash-lite-preview-02-05:free"
TOP_K = 8

# --- Init ---
supabase = create_client(
    os.environ["SUPABASE_URL"],
    os.environ["SUPABASE_SERVICE_KEY"],
)

Settings.embed_model = OpenRouterEmbedding(
    model=EMBED_MODEL,
    api_key=os.environ["OPENROUTER_API_KEY"],
)
Settings.llm = OpenRouter(
    model=CHAT_MODEL,
    api_key=os.environ["OPENROUTER_API_KEY"],
    temperature=0.1,
)

# --- Build retriever ---
vector_store = SupabaseVectorStore(
    supabase_client=supabase,
    table_name=TABLE_NAME,
    embedding_dimension=768,
)
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)
retriever = index.as_retriever(similarity_top_k=TOP_K)

# --- Custom prompt that forces citations ---
from llama_index.core.prompts import PromptTemplate

qa_prompt = PromptTemplate(
    "You are an expert codebase navigator. Answer the question using ONLY the code snippets below.\n"
    "Always cite the file path and line numbers from the metadata.\n"
    "If the answer isn't in the snippets, say 'I couldn't find that in the codebase.'\n\n"
    "Context:\n{context_str}\n\n"
    "Question: {query_str}\n\n"
    "Answer (with file references):"
)

query_engine = RetrieverQueryEngine(
    retriever=retriever,
    text_qa_template=qa_prompt,
)

# --- Interactive loop ---
print("Codebase Q&A ready. Type 'exit' to quit.\n")
while True:
    query = input(">>> ")
    if query.lower() in ("exit", "quit"):
        break
    response = query_engine.query(query)
    print(f"\n{response}\n")

    # Print source nodes
    print("--- Sources ---")
    for node in response.source_nodes[:5]:
        file_path = node.metadata.get("file_path", "unknown")
        start_line = node.metadata.get("start_line", "?")
        end_line = node.metadata.get("end_line", "?")
        score = node.score or 0
        print(f"  {file_path} (lines {start_line}-{end_line}) [score: {score:.3f}]")
    print()

Running the Tool End-to-End

# 1. Set your repo path in ingest.py
# 2. Run ingestion (one-time, or re-run when the codebase changes)
python ingest.py

# 3. Start querying
python query.py

Sample session:

>>> How is authentication handled?

Authentication is handled in `src/auth/middleware.py` (lines 12-45) using JWT tokens
validated by the `verify_token` function. The middleware extracts the token from the
Authorization header, decodes it with `python-jose`, and attaches the user object to
`request.state.user`...

--- Sources ---
  src/auth/middleware.py (lines 12-45) [score: 0.892]
  src/auth/tokens.py (lines 30-55) [score: 0.845]
  src/main.py (lines 88-102) [score: 0.791]

Sensible Extensions

• Git-aware incremental indexing: hash each file, store the hash in metadata, skip re-embedding unchanged files. Cuts re-index time by 90%+.
• Multi-language chunking: pass language per file extension — LlamaIndex's CodeSplitter supports python, typescript, go, rust, java, ruby, and more.
• Web UI: wrap the query engine in a tiny FastAPI server + vanilla HTML frontend. Same Supabase back-end, zero extra cost.
• Slack bot integration: similar pattern to our Discord FAQ bot guide — swap Qdrant for Supabase pgvector and you've got a team codebase assistant in Slack.
• Hybrid search: combine vector similarity with keyword BM25 via Supabase's built-in full-text search on the text column. Dramatically improves precision for exact symbol/function name queries.
• Structured output extraction: use a free LLM to extract function signatures, class hierarchies, and dependency graphs — similar to the receipt-to-JSON pattern.

Common Pitfalls & Fixes

FAQ

Q: How much does this cost to run? A: $0. OpenRouter's free models have rate limits but no credit-card requirement. Supabase's free tier gives you 500MB of database — enough for ~200k code chunks at 768 dimensions. For a typical medium repo (10k-50k chunks), you'll use 5-15% of the free tier.

Q: Can I use a local embedding model instead of OpenRouter? A: Yes. Swap OpenRouterEmbedding for HuggingFaceEmbedding with sentence-transformers/all-MiniLM-L6-v2 (runs locally, 384d). You'll need to change embedding_dimension to 384 and recreate the table. Total cost remains $0.

Q: How do I handle repos with multiple languages? A: LlamaIndex's CodeSplitter accepts a language parameter. Create a custom SimpleDirectoryReader subclass that maps file extensions to languages, or run separate ingestion passes per language into the same table (vectors are language-agnostic).

Q: What if my repo is private? A: Everything runs locally except the embedding API calls and Supabase storage. OpenRouter sees only the chunk text (not file paths) during embedding. For maximum privacy, use a local embedding model and a local Postgres + pgvector instance instead of Supabase.

Q: How does this compare to GitHub Copilot's codebase awareness? A: Copilot uses a proprietary index and is tightly integrated into your editor. This tool gives you an explicit, queryable index you control — useful for onboarding new team members, architecture exploration, and compliance audits. Different tools, different jobs. For a deeper dive on how AI-assisted coding changes the economics, see our piece on how Databricks cut AI coding costs by 70%.

Q: Can I deploy this as a team tool? A: Absolutely. The Supabase back-end is already cloud-hosted. Wrap the query engine in a FastAPI endpoint, add auth, and you've got a shared codebase Q&A service. The ingestion script can run in a GitHub Action on every push to main. If you're thinking about building internal tools like this as part of your career, check out what a Forward Deployed Engineer actually does in a week.