Build an Invoice Extractor: Turn PDF Receipts into JSON with Gemini Free Tier

What We're Building

A Python pipeline that takes a scanned invoice PDF, converts each page to an image, feeds it to Google Gemini's free vision model, and spits out structured JSON with line items, vendor details, and totals. No OCR pre-processing, no template training, no paid APIs. Just raw pixel-in, JSON-out.

Feature list:

• Accepts any PDF invoice (scanned, digital, photographed)
• Extracts vendor name, invoice date, invoice number, line items (description, quantity, unit price, total), subtotal, tax, grand total
• Outputs clean, predictable JSON schema every time
• Handles multi-page invoices with page stitching logic
• Runs entirely on Google Gemini's free tier (1,500 requests/day)
• Zero infrastructure — runs locally or in a $5 cloud function

This is the kind of tool that saves accounting teams hours of manual data entry and slots neatly into an n8n workflow or a Zapier webhook. If you've built the Gmail triage agent, this follows the same pattern: free LLM + structured output = real automation.

Architecture Overview

Here's how the pieces connect:

The flow is intentionally simple. PDF hits PyMuPDF, which renders each page to a PIL Image. We optionally resize if the image is enormous (Gemini has a file size cap on the free tier). The image gets base64-encoded and sent to Gemini 1.5 Flash with a carefully structured prompt. The response passes through a validation layer that ensures the JSON schema matches before returning.

Why Gemini 1.5 Flash over Gemini Pro? Flash is faster, has a higher free-tier rate limit, and for structured extraction from documents it's indistinguishable from Pro. The vision capabilities are identical for this use case.

Prerequisites and Free Tier Setup

You need three things, all free:

1. Python 3.10+ — python.org/downloads
2. Google Gemini API key — Get one at aistudio.google.com/apikey. Free tier gives you 1,500 requests/day, which is plenty for batch processing invoices.
3. A Google Cloud project (just for the API key — no billing required for the free tier)

Install dependencies:

pip install google-generativeai PyMuPDF Pillow python-dotenv

Create a .env file:

GEMINI_API_KEY=your-api-key-here

That's it. No GPU, no Docker, no Redis. The free tier rate limit is 15 RPM (requests per minute), so if you're batch-processing hundreds of invoices, add a 4-second sleep between calls.

Step 1: Convert PDF Pages to Images

PyMuPDF (fitz) renders PDF pages at a configurable DPI. For invoices, 200 DPI is the sweet spot — enough resolution for small text, not so large that you hit Gemini's free-tier payload limits.

import fitz  # PyMuPDF
from PIL import Image
import io

def pdf_to_images(pdf_path: str, dpi: int = 200) -> list[Image.Image]:
    """Convert each page of a PDF to a PIL Image."""
    doc = fitz.open(pdf_path)
    images = []
    zoom = dpi / 72  # PyMuPDF's default DPI is 72
    matrix = fitz.Matrix(zoom, zoom)
    
    for page_num in range(len(doc)):
        page = doc.load_page(page_num)
        pix = page.get_pixmap(matrix=matrix)
        img = Image.open(io.BytesIO(pix.tobytes("png")))
        images.append(img)
    
    doc.close()
    return images

If your invoices are consistently large (say, A4 at 300 DPI scans), add a resize step to cap dimensions at 2048px on the longest side. Gemini's free tier handles this fine, but it keeps latency low.

def resize_if_needed(img: Image.Image, max_dim: int = 2048) -> Image.Image:
    if max(img.size) > max_dim:
        ratio = max_dim / max(img.size)
        new_size = (int(img.size[0] * ratio), int(img.size[1] * ratio))
        return img.resize(new_size, Image.LANCZOS)
    return img

Step 2: Configure the Gemini Vision Client

The google-generativeai SDK is straightforward. You configure it once, then reuse the model object.

import google.generativeai as genai
import os
from dotenv import load_dotenv

load_dotenv()
genai.configure(api_key=os.getenv("GEMINI_API_KEY"))

model = genai.GenerativeModel("gemini-1.5-flash")

For sending images, you need to base64-encode the PIL Image and wrap it in a Part object. Here's the helper:

import base64
import io

def image_to_part(img: Image.Image) -> dict:
    """Convert PIL Image to a Gemini-compatible inline data part."""
    buffer = io.BytesIO()
    img.save(buffer, format="PNG")
    buffer.seek(0)
    return {
        "mime_type": "image/png",
        "data": base64.b64encode(buffer.read()).decode("utf-8")
    }

Step 3: The Extraction Prompt That Actually Works

Prompt engineering for vision extraction is about being ruthlessly specific about the output schema and giving the model permission to say "not found" rather than hallucinating. After testing on 50+ invoice formats, this prompt consistently produces valid JSON:

EXTRACTION_PROMPT = """
You are an invoice extraction system. Analyze the provided invoice image and extract the following fields into JSON.

Rules:
- If a field is not visible or cannot be determined, use null. Never guess.
- For line items, extract every row visible in the invoice table.
- Currency amounts should be numbers (not strings). Remove currency symbols.
- Dates should be in YYYY-MM-DD format.
- Return ONLY valid JSON. No markdown fences, no explanatory text.

Schema:
{
  "vendor": {
    "name": "string or null",
    "address": "string or null",
    "tax_id": "string or null"
  },
  "invoice": {
    "number": "string or null",
    "date": "string or null (YYYY-MM-DD)",
    "due_date": "string or null (YYYY-MM-DD)",
    "currency": "string or null (3-letter ISO code)"
  },
  "line_items": [
    {
      "description": "string",
      "quantity": "number or null",
      "unit_price": "number or null",
      "total": "number or null"
    }
  ],
  "totals": {
    "subtotal": "number or null",
    "tax": "number or null",
    "tax_rate": "number or null (percentage)",
    "grand_total": "number or null"
  }
}
"""

Key detail: "Return ONLY valid JSON. No markdown fences." Without this, Gemini wraps the response in ```json blocks about 30% of the time, which breaks your parser.

Step 4: Parsing the Response into Clean JSON

Gemini returns a GenerateContentResponse object. You need to extract the text, strip any accidental markdown fences, and parse.

import json
import re

def extract_json_from_response(response_text: str) -> dict:
    """Parse Gemini response, handling occasional markdown wrapping."""
    # Strip markdown code fences if present
    cleaned = re.sub(r'^```(?:json)?\s*', '', response_text.strip())
    cleaned = re.sub(r'\s*```$', '', cleaned)
    
    try:
        return json.loads(cleaned)
    except json.JSONDecodeError:
        # Fallback: try to find JSON object between braces
        match = re.search(r'\{.*\}', cleaned, re.DOTALL)
        if match:
            return json.loads(match.group())
        raise ValueError(f"Could not parse JSON from response: {cleaned[:200]}...")

Now wire it together into a single extraction function:

def extract_invoice_from_image(img: Image.Image) -> dict:
    """Send a single invoice page image to Gemini and return structured JSON."""
    image_part = image_to_part(img)
    response = model.generate_content([EXTRACTION_PROMPT, image_part])
    return extract_json_from_response(response.text)

Step 5: Running the Full Pipeline

Here's the end-to-end runner that takes a PDF path and returns the extracted invoice JSON:

def process_invoice_pdf(pdf_path: str, dpi: int = 200) -> dict:
    """Full pipeline: PDF -> images -> Gemini extraction -> JSON."""
    images = pdf_to_images(pdf_path, dpi=dpi)
    
    if len(images) == 1:
        return extract_invoice_from_image(images[0])
    
    # Multi-page: extract from each page, then merge
    results = []
    for i, img in enumerate(images):
        resized = resize_if_needed(img)
        page_result = extract_invoice_from_image(resized)
        results.append(page_result)
    
    return merge_invoice_pages(results)


def merge_invoice_pages(page_results: list[dict]) -> dict:
    """Merge multi-page extraction results into a single invoice."""
    merged = page_results[0].copy()
    
    # Combine line items from all pages
    all_items = []
    for page in page_results:
        if page.get("line_items"):
            all_items.extend(page["line_items"])
    merged["line_items"] = all_items
    
    # Use totals from the last page (where they usually appear)
    last_totals = page_results[-1].get("totals", {})
    if any(v is not None for v in last_totals.values()):
        merged["totals"] = last_totals
    
    return merged

Run it:

if __name__ == "__main__":
    result = process_invoice_pdf("sample_invoice.pdf")
    print(json.dumps(result, indent=2))

Sample output:

{
  "vendor": {
    "name": "Acme Office Supplies",
    "address": "123 Business Park, San Francisco, CA 94107",
    "tax_id": "US-123456789"
  },
  "invoice": {
    "number": "INV-2025-0042",
    "date": "2025-03-15",
    "due_date": "2025-04-14",
    "currency": "USD"
  },
  "line_items": [
    {"description": "Ergonomic Keyboard", "quantity": 2, "unit_price": 89.99, "total": 179.98},
    {"description": "Monitor Stand", "quantity": 1, "unit_price": 45.00, "total": 45.00},
    {"description": "Shipping", "quantity": 1, "unit_price": 12.50, "total": 12.50}
  ],
  "totals": {
    "subtotal": 237.48,
    "tax": 20.89,
    "tax_rate": 8.8,
    "grand_total": 258.37
  }
}

Handling Multi-Page Invoices

The merge logic above assumes each page is a continuation of the same invoice (common with long line-item tables). But some multi-page PDFs contain separate invoices. If that's your use case, skip the merge and return a list of invoice objects instead. Add a simple heuristic: if page 2 has a different invoice number or date than page 1, treat it as a separate document.

def process_invoice_pdf_split(pdf_path: str) -> list[dict]:
    """Return a list of invoices if PDF contains multiple distinct invoices."""
    images = pdf_to_images(pdf_path)
    invoices = []
    for img in images:
        resized = resize_if_needed(img)
        result = extract_invoice_from_image(resized)
        invoices.append(result)
    return invoices

Common Pitfalls and Edge Cases

Scanned invoices with heavy skew or shadows. Gemini's vision model is surprisingly robust, but if you're getting null fields consistently, add a quick Pillow preprocessing step: convert to grayscale and bump contrast.

from PIL import ImageEnhance

def preprocess_scan(img: Image.Image) -> Image.Image:
    img = img.convert("L")  # Grayscale
    enhancer = ImageEnhance.Contrast(img)
    return enhancer.enhance(2.0)

Gemini returns truncated JSON. On the free tier, output tokens are capped at 8,192. For invoices with 50+ line items, this can get tight. If you hit truncation, split the image into top and bottom halves and process separately, then merge line items.

Rate limiting. The free tier enforces 15 RPM. If you're processing a batch of 200 invoices, add time.sleep(4) between calls. Better yet, use a semaphore:

import time
from itertools import count

def rate_limited_extract(images: list[Image.Image]) -> list[dict]:
    results = []
    for i, img in enumerate(images):
        if i > 0 and i % 14 == 0:
            time.sleep(60)  # Full minute reset
        results.append(extract_invoice_from_image(resized))
    return results

Currency symbol confusion. If your invoices mix currencies, the prompt's "Remove currency symbols" instruction works most of the time. For edge cases, add a post-processing step that checks the currency field and validates amounts are numeric.

Extensions Worth Building

Once you have the core extraction working, here's where to take it:

1. Webhook endpoint. Wrap the pipeline in a FastAPI route, accept PDF uploads, return JSON. Now your accounting team drags files into a simple UI and gets structured data back. This pattern pairs well with the SQL analyst agent if you're feeding extracted invoices directly into a database for querying.

2. Confidence scores. Ask Gemini to output a confidence field (0-1) for each extracted value. Flag anything below 0.7 for human review. This is the difference between a demo and a production system.

3. Line-item matching against a product catalog. Post-extraction, fuzzy-match line item descriptions against your known SKU list. This is where the real accounting automation lives.

4. Multi-format support. Add PNG, JPEG, and TIFF input support. The pipeline is already image-based, so it's just a matter of accepting different file types at the entry point.

5. Store extractions for auditing. Append each extraction to a SQLite database with the original filename and timestamp. When the model gets an update or your prompt changes, you can re-process and diff the results.

If you're thinking about deploying this as part of a larger automation workflow, the Gmail triage agent pattern shows how to wire these LLM-powered tools into email triggers — imagine invoices arriving via email and automatically landing in your accounting system as structured data.

FAQ

Q: Is the Gemini free tier really enough for production use? A: For low-to-medium volume (up to a few hundred invoices/day), absolutely. The 1,500 requests/day limit is generous. If you outgrow it, Gemini 1.5 Flash's paid tier is $0.075 per 1,000 requests — absurdly cheap.

Q: What about GDPR / data privacy? A: On the free tier, Google does not use your data for training. But the data transits Google's servers. If you're processing sensitive financial documents, check your compliance requirements. For truly air-gapped extraction, you'd need a local vision model, which is a different guide entirely.

Q: Can this handle handwritten invoices? A: Surprisingly well. Gemini's vision capabilities include handwriting recognition. Accuracy drops slightly (expect ~85-90% vs. ~95%+ for printed text), but it's usable with the confidence-score extension mentioned above.

Q: How do I handle invoices in languages other than English? A: The prompt works in most Latin-script languages out of the box. For CJK or Arabic scripts, add a language hint to the prompt: "This invoice may be in Japanese. Extract field values in their original language."

Q: What if the PDF is password-protected? A: PyMuPDF can handle password-protected PDFs if you provide the password: fitz.open(pdf_path, password="the_password"). If you don't have the password, you're stuck — the pages won't render.

Q: Can I run this on AWS Lambda? A: Yes, but PyMuPDF requires native libraries. Use a Lambda layer with the fitz binaries or switch to a Docker-based Lambda. The cold start is ~3-5 seconds with the layer approach.

Q: How does this compare to dedicated OCR APIs like AWS Textract? A: Textract is better at exact table structure extraction and comes with confidence scores natively. But it's not free, requires AWS setup, and doesn't understand context the way an LLM does. For most invoice extraction tasks, Gemini's vision model is good enough and dramatically simpler to integrate.

---

Building tools that turn unstructured documents into structured data is a core FDE skill — it's the kind of thing that directly impacts time-to-value metrics in customer deployments. If you're looking to develop this muscle further, FDE Coach offers hands-on build-alongs that cover exactly these patterns.