Build a Local Codebase Q&A Tool with Ollama, LlamaIndex, and Qdrant Free Tier

What We're Building

A command-line tool that points at any local git repository, ingests every source file, splits them into semantically meaningful chunks, generates embeddings using a local model served by Ollama, and pushes those vectors into Qdrant's free cloud tier. You then fire natural language questions at it—"how does authentication middleware chain work?" or "where is the database connection pool initialized?"—and get back precise file paths and code snippets with architectural context.

This isn't a toy. It's a practical engineering tool that costs zero dollars to run and keeps your proprietary code off third-party servers. The embedding model runs locally. The LLM that formulates answers runs locally. Only the vector index—which is a set of floating-point arrays, not your source code—lives in Qdrant's cloud.

Feature List

• Local-first embeddings and generation: Ollama runs on your machine. No API keys for inference.
• Semantic code chunking: LlamaIndex's CodeSplitter respects AST boundaries in Python, JavaScript, TypeScript, and more.
• Free cloud vector storage: Qdrant's free tier gives you a 1GB cluster. Plenty for a medium-sized monorepo.
• Hybrid retrieval: Combine dense vector search with metadata filters (file type, directory, function name).
• Source-anchored answers: Every response includes the originating file path and line range.
• Repo refresh without full re-index: Upsert chunks by file hash, so only changed files get re-embedded.

Architecture Overview

Before we write a line of code, here's how the pieces fit together.

The flow is linear but has a feedback loop at query time. Ingestion: repo → reader → code splitter → Ollama embeddings → Qdrant. Query: user question → embedding → Qdrant search → retrieved chunks → Ollama LLM → answer with citations.

Prerequisites and Setup

Everything here is free. No credit card required for the core tools, though Qdrant Cloud asks for one to prevent abuse—you won't be charged.

Qdrant Cloud setup steps:

1. Sign up at cloud.qdrant.io.
2. Create a new cluster, select the free tier (1GB RAM, 1 vCPU).
3. Once provisioned, grab your cluster URL and API key from the dashboard's "API Key" section.
4. Store them in environment variables—we'll use QDRANT_URL and QDRANT_API_KEY.

Ollama setup:

1. Install the Ollama desktop app or run the Linux server.
2. Pull two models: an embedding model and a chat model.

   ollama pull nomic-embed-text
   ollama pull llama3.2
   

   nomic-embed-text is a 137M-parameter embedding model that runs on a potato. llama3.2 is a capable 3B chat model that fits in 2-3GB of RAM. Swap in codellama or deepseek-coder if you prefer.
3. Verify they're available:

   ollama list
   

Step 1: Project Initialization

Create a directory and a virtual environment.

mkdir codebase-qa && cd codebase-qa
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

Install dependencies:

pip install llama-index llama-index-embeddings-ollama llama-index-vector-stores-qdrant qdrant-client

We're using LlamaIndex's modular packages rather than the monolithic llama-index to keep the footprint lean.

Create a .env file:

QDRANT_URL=https://your-cluster-id.us-east-1-0.aws.cloud.qdrant.io:6333
QDRANT_API_KEY=your-api-key-here

And a config.py to centralize settings:

import os
from dotenv import load_dotenv

load_dotenv()

QDRANT_URL = os.environ["QDRANT_URL"]
QDRANT_API_KEY = os.environ["QDRANT_API_KEY"]
OLLAMA_BASE_URL = "http://localhost:11434"
EMBED_MODEL = "nomic-embed-text"
CHAT_MODEL = "llama3.2"
COLLECTION_NAME = "codebase_chunks"
VECTOR_SIZE = 768  # nomic-embed-text outputs 768-dimensional vectors

Step 2: Ingesting and Chunking the Codebase

This is where most code Q&A tools fall over. Splitting on newlines or token counts destroys function boundaries. LlamaIndex's CodeSplitter uses tree-sitter to parse the AST and split at function, class, and method boundaries.

# ingest.py
from pathlib import Path
from llama_index.core import SimpleDirectoryReader
from llama_index.core.node_parser import CodeSplitter
from llama_index.core.schema import Document

def ingest_repo(repo_path: str, extensions: list[str] | None = None) -> list[Document]:
    if extensions is None:
        # Sensible defaults: cover the big ones without pulling in binaries
        extensions = [".py", ".js", ".ts", ".tsx", ".jsx", ".go", ".rs", ".java", ".rb"]

    reader = SimpleDirectoryReader(
        input_dir=repo_path,
        recursive=True,
        required_exts=extensions,
        exclude=["node_modules", ".git", "__pycache__", "dist", "build", ".venv", "target"],
    )
    docs = reader.load_data()
    print(f"Loaded {len(docs)} source files")

    splitter = CodeSplitter(
        language="python",  # Default; it auto-detects per file extension
        chunk_lines=40,
        chunk_lines_overlap=15,
        max_chars=1500,
    )
    nodes = splitter.get_nodes_from_documents(docs)
    print(f"Split into {len(nodes)} chunks")
    return nodes

CodeSplitter handles Python, JS/TS, Go, Rust, Java, and more. The 40-line chunk with 15-line overlap means a 60-line function becomes two chunks with shared context at the boundary. Adjust chunk_lines based on your codebase's typical function length.

Step 3: Embedding with Ollama

LlamaIndex has a first-party Ollama embedding integration. It calls Ollama's HTTP API, which is already running locally on port 11434.

# embed.py
from llama_index.embeddings.ollama import OllamaEmbedding
from config import OLLAMA_BASE_URL, EMBED_MODEL

def get_embed_model() -> OllamaEmbedding:
    return OllamaEmbedding(
        model_name=EMBED_MODEL,
        base_url=OLLAMA_BASE_URL,
        # nomic-embed-text expects a specific prompt format
        ollama_additional_kwargs={"mirostat": 0},
    )

The embedding model runs entirely on your machine. Embedding 10,000 chunks takes a few minutes on a modern laptop, most of it I/O-bound waiting for Ollama.

Step 4: Indexing into Qdrant Cloud

Now we wire LlamaIndex's ingestion pipeline to Qdrant. The key decision: use LlamaIndex's IngestionPipeline with a QdrantVectorStore. This handles batching, retries, and deduplication via document hashes.

# index.py
import hashlib
from llama_index.core.ingestion import IngestionPipeline
from llama_index.vector_stores.qdrant import QdrantVectorStore
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams
from config import QDRANT_URL, QDRANT_API_KEY, COLLECTION_NAME, VECTOR_SIZE
from embed import get_embed_model
from ingest import ingest_repo

def build_index(repo_path: str):
    client = QdrantClient(url=QDRANT_URL, api_key=QDRANT_API_KEY)

    # Create collection if it doesn't exist
    existing = client.get_collections().collections
    if not any(c.name == COLLECTION_NAME for c in existing):
        client.create_collection(
            collection_name=COLLECTION_NAME,
            vectors_config=VectorParams(size=VECTOR_SIZE, distance=Distance.COSINE),
        )
        print(f"Created collection '{COLLECTION_NAME}'")

    vector_store = QdrantVectorStore(
        client=client,
        collection_name=COLLECTION_NAME,
    )

    embed_model = get_embed_model()
    nodes = ingest_repo(repo_path)

    # Attach a content hash to each node for deduplication
    for node in nodes:
        node.metadata["doc_hash"] = hashlib.md5(node.text.encode()).hexdigest()

    pipeline = IngestionPipeline(
        transformations=[embed_model],
        vector_store=vector_store,
    )

    pipeline.run(nodes=nodes, show_progress=True)
    print(f"Indexed {len(nodes)} chunks into Qdrant")

if __name__ == "__main__":
    import sys
    build_index(sys.argv[1])

Run it:

python index.py /path/to/your/repo

Step 5: Building the Q&A CLI

The query side: take a question, embed it, retrieve the top-k relevant chunks from Qdrant, stuff them into a prompt, and ask Ollama's chat model to synthesize an answer.

# query.py
from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.llms.ollama import Ollama
from llama_index.vector_stores.qdrant import QdrantVectorStore
from qdrant_client import QdrantClient
from config import QDRANT_URL, QDRANT_API_KEY, COLLECTION_NAME, EMBED_MODEL, CHAT_MODEL, OLLAMA_BASE_URL

def setup_query_engine():
    client = QdrantClient(url=QDRANT_URL, api_key=QDRANT_API_KEY)
    vector_store = QdrantVectorStore(client=client, collection_name=COLLECTION_NAME)

    Settings.embed_model = OllamaEmbedding(
        model_name=EMBED_MODEL,
        base_url=OLLAMA_BASE_URL,
    )
    Settings.llm = Ollama(
        model=CHAT_MODEL,
        base_url=OLLAMA_BASE_URL,
        temperature=0.1,
        request_timeout=120.0,
    )

    index = VectorStoreIndex.from_vector_store(vector_store=vector_store)
    return index.as_query_engine(
        similarity_top_k=8,
        response_mode="compact",
    )

def ask(question: str):
    engine = setup_query_engine()
    response = engine.query(question)
    print(f"\n{response}\n")
    # Print source nodes
    print("--- Sources ---")
    for node in response.source_nodes:
        file_path = node.metadata.get("file_path", "unknown")
        print(f"{file_path} (score: {node.score:.3f})")

if __name__ == "__main__":
    import sys
    if len(sys.argv) < 2:
        print("Usage: python query.py 'your question here'")
        sys.exit(1)
    ask(sys.argv[1])

The response_mode="compact" tells LlamaIndex to pack as many chunks as possible into the context window before summarizing. For a 3B model with an 8K context, this works well with 8 chunks of ~1500 characters each.

Step 6: Running the Tool

Full workflow from zero to answer:

# Terminal 1: Ensure Ollama is running
ollama serve

# Terminal 2: Index a repo
python index.py ~/projects/my-fastapi-app

# Ask a question
python query.py "How does the authentication middleware verify JWT tokens?"

Expected output:

The JWT authentication middleware is defined in `src/auth/middleware.py` (lines 12-58).
It extracts the token from the `Authorization` header using `Bearer` scheme, decodes it
with `jose.jwt.decode()` using the secret from `config.AUTH_SECRET`, and attaches the
payload to `request.state.user`. If the token is expired or invalid, it raises an
`HTTPException` with status 401.

--- Sources ---
src/auth/middleware.py (score: 0.892)
src/config.py (score: 0.845)
src/auth/dependencies.py (score: 0.801)

Sensible Extensions

Once the core loop works, you'll want to harden it.

Incremental indexing. Before re-embedding, check if a file's hash matches the stored doc_hash. Skip if unchanged. This turns a 10-minute full re-index into a 30-second delta.

def get_file_hash(filepath: str) -> str:
    with open(filepath, "rb") as f:
        return hashlib.md5(f.read()).hexdigest()

Metadata filtering. Add a --filter flag to restrict searches to specific directories or file types. Qdrant supports payload filters natively.

from qdrant_client.models import Filter, FieldCondition, MatchValue

filters = Filter(
    must=[FieldCondition(key="file_type", match=MatchValue(value=".py"))]
)

Multiple repos in one collection. Prefix each chunk's metadata with a repo_name field, then filter queries to a specific repo. One Qdrant free-tier cluster can index several small-to-medium repos.

Web UI with Streamlit. Wrap the query engine in a Streamlit app (also free) and share it with your team over localhost. This is the bridge between a messy customer problem and a shipped prototype in a single week.

Common Pitfalls and How to Avoid Them

Ollama connection refused. If query.py fails with a connection error, Ollama isn't running. On macOS, the menu bar app must show the llama icon as active. On Linux, run ollama serve in a dedicated terminal.

Qdrant collection already exists with wrong vector size. If you switch embedding models, you'll get a dimension mismatch. Delete the collection via the Qdrant dashboard or client.delete_collection(COLLECTION_NAME) and re-index.

Out-of-memory during indexing. SimpleDirectoryReader loads all files into memory. For repos with >5,000 source files, switch to a streaming approach or increase the chunk size to reduce node count.

Slow embedding on CPU. nomic-embed-text is fast, but embedding 20,000 chunks will still take time. Run it overnight or on a machine with a GPU. Ollama automatically uses Metal on macOS and CUDA on Linux if available.

Irrelevant retrieval results. If answers miss the mark, your chunking strategy is likely the culprit. Functions split mid-body lose semantic coherence. Try reducing chunk_lines to 25 or switching to a SentenceSplitter for languages without tree-sitter support. This is the same class of problem discussed in debugging concurrent LLM agents—small context window mismatches cascade into bad outputs.

The prototype-product gap. This tool works great on your machine. Shipping it to a team means handling environment drift, model version pinning, and index staleness. Read why LLM-generated code still needs systems thinking to ship before promising it to your eng org.

FAQ

Does Qdrant's free tier have usage limits? Yes: 1GB RAM, 1 vCPU, and 1GB disk. That's enough for roughly 500,000 768-dimensional vectors. For a typical codebase of 5,000 chunks, you're using about 2% of capacity.

Can I use a different embedding model? Absolutely. Any model Ollama can serve works. mxbai-embed-large outputs 1024-dimensional vectors—just update VECTOR_SIZE in config.py and recreate the collection.

Why not use ChromaDB or FAISS locally? You can, and it removes the cloud dependency entirely. Qdrant's free tier gives you a persistent, shareable index without running a local server process. The architecture is the same either way.

How do I handle private/proprietary code? Your source code never leaves your machine. Only the vector embeddings—which are irreversible floating-point arrays—are sent to Qdrant Cloud. The embedding and LLM models run locally via Ollama.

What if I want to chat with the codebase instead of single-shot Q&A? Swap response_mode="compact" for response_mode="chat" and maintain a ChatMemory buffer. LlamaIndex handles the conversation loop. This is a natural extension toward the kind of personal meeting notetaker pattern, applied to code.

Does this work with monorepos containing multiple languages? Yes. CodeSplitter auto-detects the language from the file extension and applies the appropriate tree-sitter parser. Polyglot repos are a first-class use case.