Build a SQL Analyst Agent with LlamaIndex and Groq’s Free Tier

What We’re Building

A text-to-SQL agent that accepts natural language questions—“What were the top 5 products by revenue last month?”—and returns: a generated SQL query, the raw results, and a synthesized natural language answer. The stack costs zero dollars: Groq serves Llama 3 70B at ~300 tokens/sec on a free tier, Supabase hosts the Postgres database, and LlamaIndex wires everything together in under 200 lines of Python.

Feature list:

• Natural language → SQL translation with schema awareness
• Automatic query execution against a live Postgres database
• Formatted answer synthesis (LLM explains results in plain English)
• Streaming responses via Groq’s fast inference
• Error recovery: malformed SQL is caught, explained, and retried
• Zero-cost deployment on free-tier infrastructure

Architecture Overview

The agent is a single Python process. LlamaIndex provides the NLSQLTableQueryEngine which bundles schema introspection, few-shot prompting, and output parsing. We swap the default OpenAI LLM for Groq’s chat completion endpoint. On each question, the engine extracts table schemas from Postgres, builds a prompt containing the DDL and the user question, sends it to Groq, parses the SQL, executes it via SQLAlchemy, and feeds the result rows back to the LLM for a human-readable answer.

Prerequisites (All Free Tier)

You’ll need:

• Groq API key: create one at console.groq.com/keys
• Supabase project: note the host, port (6543 for connection pooling), database name (default postgres), and password
• Python environment: a virtualenv or conda environment

Step 1: Spin Up a Free Supabase Postgres Instance

1. Head to supabase.com and click “Start your project”.
2. Create an organization, then a new project. Choose a strong database password (store it).
3. Wait ~2 minutes for provisioning. Once ready, navigate to Settings → Database.
4. Under “Connection string”, switch to the Session pooler tab. Copy the URI—it looks like:

   postgresql://postgres.[ref]:[password]@aws-0-us-west-1.pooler.supabase.com:6543/postgres
   

5. Enable the “IPv4 Add-On” for direct connections if you plan to connect from restrictive networks (free, one click from the database settings page).

Step 2: Seed the Database with Sample Data

We’ll create a small e-commerce schema so the agent has something to query. Open the Supabase SQL Editor (left sidebar → SQL Editor) and run:

CREATE TABLE customers (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    category TEXT NOT NULL,
    price NUMERIC(10,2) NOT NULL
);

CREATE TABLE orders (
    id SERIAL PRIMARY KEY,
    customer_id INTEGER REFERENCES customers(id),
    order_date TIMESTAMP DEFAULT NOW(),
    total NUMERIC(12,2)
);

CREATE TABLE order_items (
    id SERIAL PRIMARY KEY,
    order_id INTEGER REFERENCES orders(id),
    product_id INTEGER REFERENCES products(id),
    quantity INTEGER NOT NULL,
    unit_price NUMERIC(10,2) NOT NULL
);

INSERT INTO customers (name, email) VALUES
    ('Alice Johnson', 'alice@example.com'),
    ('Bob Smith', 'bob@example.com'),
    ('Carol Lee', 'carol@example.com');

INSERT INTO products (name, category, price) VALUES
    ('Widget A', 'Widgets', 19.99),
    ('Widget B', 'Widgets', 24.99),
    ('Gadget X', 'Gadgets', 49.99),
    ('Gadget Y', 'Gadgets', 59.99),
    ('Doohickey Z', 'Doohickeys', 9.99);

INSERT INTO orders (customer_id, order_date, total) VALUES
    (1, '2025-01-15 10:30:00', 94.97),
    (2, '2025-01-16 14:00:00', 49.99),
    (1, '2025-02-01 09:15:00', 19.99),
    (3, '2025-02-10 16:45:00', 109.98);

INSERT INTO order_items (order_id, product_id, quantity, unit_price) VALUES
    (1, 1, 2, 19.99),
    (1, 3, 1, 49.99),
    (2, 3, 1, 49.99),
    (3, 1, 1, 19.99),
    (4, 2, 2, 24.99),
    (4, 4, 1, 59.99);

This gives us customers, products, orders, and line items—enough for joins, aggregations, and date filters.

Step 3: Set Up the Python Environment

mkdir sql-agent && cd sql-agent
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install llama-index llama-index-llms-groq sqlalchemy psycopg2-binary python-dotenv

Create a .env file:

GROQ_API_KEY=gsk_your_key_here
DATABASE_URL=postgresql://postgres.[ref]:[password]@aws-0-us-west-1.pooler.supabase.com:6543/postgres

Load it in your script:

from dotenv import load_dotenv
load_dotenv()

Step 4: Configure Groq and LlamaIndex

LlamaIndex’s Groq class wraps the Groq chat completion API. We configure it with the free-tier model llama-3.3-70b-versatile (or llama3-70b-8192 depending on availability—check your Groq console for current model IDs).

import os
from llama_index.llms.groq import Groq

llm = Groq(
    model="llama-3.3-70b-versatile",
    api_key=os.environ["GROQ_API_KEY"],
    temperature=0.1,  # low temp for deterministic SQL
    max_tokens=2048,
)

Set up the SQLAlchemy engine pointing at Supabase:

from sqlalchemy import create_engine

engine = create_engine(os.environ["DATABASE_URL"])

Step 5: Build the SQL Agent Engine

LlamaIndex ships an NLSQLTableQueryEngine that handles schema extraction, prompt construction, SQL parsing, and answer synthesis. We inject our Groq LLM and the SQLAlchemy engine.

from llama_index.core import SQLDatabase
from llama_index.core.query_engine import NLSQLTableQueryEngine

# Wrap the SQLAlchemy engine for LlamaIndex
sql_database = SQLDatabase(engine, include_tables=[
    "customers", "products", "orders", "order_items"
])

query_engine = NLSQLTableQueryEngine(
    sql_database=sql_database,
    llm=llm,
    synthesize_response=True,  # LLM explains results
)

That’s the core. Under the hood, NLSQLTableQueryEngine:

1. Introspects the database for table schemas (column names, types, foreign keys).
2. Builds a prompt: “Given the following SQL tables, write a query to answer the question. Only output SQL.”
3. Parses the LLM’s SQL response, stripping markdown fences if present.
4. Executes the query via SQLAlchemy.
5. Passes the result rows back to the LLM with the original question for a natural language answer.

Step 6: Run the Agent and Ask Questions

Wrap the query engine in a simple REPL:

def ask(question: str):
    response = query_engine.query(question)
    print(f"\nSQL:\n{response.metadata.get('sql_query', 'N/A')}")
    print(f"\nAnswer:\n{response.response}")
    if response.metadata.get('result'):
        print(f"\nRaw rows:\n{response.metadata['result']}")

if __name__ == "__main__":
    questions = [
        "What are the names and emails of all customers?",
        "Which product generated the most revenue?",
        "How many orders did Alice Johnson place?",
        "What is the total revenue by product category?",
    ]
    for q in questions:
        print(f"\n{'='*60}\nQ: {q}")
        ask(q)

Run it:

python agent.py

Sample output:

============================================================
Q: Which product generated the most revenue?

SQL:
SELECT p.name, SUM(oi.quantity * oi.unit_price) AS total_revenue
FROM order_items oi
JOIN products p ON oi.product_id = p.id
GROUP BY p.name
ORDER BY total_revenue DESC
LIMIT 1;

Answer:
The product that generated the most revenue is Gadget Y, with a total revenue of $59.99.

Raw rows:
[('Gadget Y', 59.99)]

The agent correctly joined order_items to products, aggregated revenue, and returned a concise answer.

Sensible Extensions

1. Chat history with conversational memory. Wrap the query engine in a ChatMemoryBuffer so users can ask follow-ups like “and what about last quarter?” without repeating context. LlamaIndex’s ContextChatEngine handles this with a few extra lines.

2. Streaming responses. Groq’s speed shines with streaming. Set streaming=True on the Groq LLM constructor and use response.print_response_stream() for token-by-token output.

3. Query validation sandbox. Before executing, pass the generated SQL through sqlparse or a dry-run EXPLAIN to catch syntax errors early. If the query fails, feed the error back to the LLM for a retry—this is a one-line change in LlamaIndex’s callback system.

4. Deploy as a Slack bot or API. Wrap the ask() function in a FastAPI endpoint and deploy to Render or Fly.io free tiers. Add a Slack slash command webhook and your team has a natural language database analyst.

5. Multi-turn schema refinement. If the agent consistently misunderstands a table, add a context_str parameter to NLSQLTableQueryEngine with business definitions (e.g., “orders.total includes tax and shipping”). This acts as a system prompt for the SQL generation step.

For a deeper dive into building agents that plan and execute multi-step tasks, check out our guide on building a multi-agent research assistant that plans, searches, and writes a brief.

Common Pitfalls

“No module named ‘llama_index.llms.groq’” — You installed llama-index but not the Groq integration. Run pip install llama-index-llms-groq.

Connection refused to Supabase — Supabase’s free tier requires connection pooling (port 6543) and sometimes the IPv4 add-on. Verify your connection string uses the pooler host and port 6543, not 5432.

Groq rate limiting — Free tier caps at 30 RPM. If you hit it, add a time.sleep(2) between queries or batch questions. The error message is explicit: “Rate limit exceeded.”

LLM generates invalid SQL — Llama 3 70B is strong but not infallible. Lower temperature to 0.0–0.1. If it hallucinates column names, explicitly list table schemas in the context_str. LlamaIndex’s default schema extraction is usually sufficient, but edge cases with complex types (JSONB, arrays) may need manual hints.

Slow first query — Groq’s cold start can take 2–5 seconds. Subsequent queries are sub-second. This is normal for serverless GPU inference.

Environment variables not loading — Ensure load_dotenv() runs before accessing os.environ. In Jupyter notebooks, restart the kernel after changing .env.

FAQ

Q: Can I use a different free LLM provider? Yes—swap the Groq class for any LlamaIndex-supported LLM. Together AI, Fireworks, and OpenRouter all offer free credits. The pattern is identical: instantiate the LLM, pass it to NLSQLTableQueryEngine. Groq wins on raw tokens-per-second for interactive use.

Q: How do I handle large result sets? By default, LlamaIndex passes all rows to the LLM for synthesis. For tables with thousands of rows, add a LIMIT to your prompt’s context or use sql_database.run_sql() with a row cap and pass only the first N rows to the LLM.

Q: Is my database schema sent to Groq? Yes—table names, column names, and types are included in the prompt. Groq’s data privacy policy states they do not train on API inputs, but if you’re querying sensitive production data, consider an on-prem LLM via Ollama. Our guide on building a local voice assistant with Whisper and Ollama covers the local inference pattern.

Q: Can the agent write INSERT/UPDATE/DELETE statements? By default, NLSQLTableQueryEngine only runs SELECT queries. If you need write capabilities, you’d build a custom agent with tool-use, but be very careful—LLMs can generate destructive queries. Always run in a sandbox with read-only database credentials.

Q: What’s the difference between this and just using ChatGPT with a database plugin? Control, cost, and privacy. You own the pipeline, you’re not paying per query, and you can audit every generated SQL statement. For production use, this architecture is auditable, debuggable, and free.

Q: Where do I go from here to become proficient at building AI agents? This pattern—LLM + tool + structured output—is the core of forward-deployed engineering. If you’re preparing for FDE interviews or want to ship agentic features into enterprise products, FDE Coach offers tactical, hands-on preparation for the decomposition and debugging rounds that separate strong candidates from great ones.