Build a Receipt-to-JSON Extractor with Google Gemini 1.5 Flash Free Tier

What We’re Building

A hands-free receipt digitization pipeline that runs on your laptop. Drop a PDF, JPEG, or PNG receipt into a folder, and the script automatically extracts structured data using Google’s Gemini 1.5 Flash model—all within the generous free tier limits. The output is clean JSON you can pipe into accounting software, expense databases, or a personal finance dashboard.

Feature list:

• Folder watching with watchdog—no polling loops, instant reaction to new files
• Multi-format support: PDFs, PNGs, JPEGs, even HEIC with a quick conversion
• Gemini 1.5 Flash free tier (15 RPM, 1,500 requests/day) for vision-based extraction
• Structured JSON output: vendor name, date, line items with quantities and prices, subtotal, tax, total
• Duplicate detection via file hash to avoid re-processing
• Error handling with retries and dead-letter folder for failed receipts

Architecture Overview

Three lightweight components communicate through the filesystem. No message queues, no databases—just Python, folders, and an API call.

The watcher fires an event when a file lands. The processor reads the file, encodes it as base64, and sends it to Gemini with a carefully engineered prompt that requests structured output. The response is parsed, validated, and saved alongside the original receipt. Failures route to a dead-letter folder for manual review.

Prerequisites (All Free Tier)

Before writing a single line, grab these:

1. Google Gemini API key — Go to Google AI Studio, sign in with a Google account, click “Create API Key.” The free tier gives you 1,500 requests/day and 15 RPM on Gemini 1.5 Flash. No credit card needed.

2. Python 3.10+ — We use match statements and modern type hints. Grab it from python.org if you’re not already on it.

3. Poppler (for PDF-to-image conversion) — macOS: brew install poppler. Linux: sudo apt install poppler-utils. Windows: download from poppler releases and add bin/ to PATH.

That’s it. No cloud accounts beyond Google, no vector databases, no paid OCR services.

Step 1: Project Setup and Dependencies

Create a project directory and a virtual environment:

mkdir receipt-extractor && cd receipt-extractor
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

Install the four dependencies:

pip install watchdog google-generativeai pillow pdf2image python-dotenv

Create a .env file for your API key:

echo 'GEMINI_API_KEY=your-key-here' > .env

Create the folder structure the watcher expects:

mkdir -p receipts output dead_letter

Your tree should look like:

receipt-extractor/
├── .env
├── .venv/
├── receipts/          # drop files here
├── output/            # JSON results appear here
├── dead_letter/       # failed receipts land here
└── main.py            # we'll write this next

Step 2: Writing the Gemini Client

Create gemini_client.py. This module handles authentication, prompt construction, and response parsing. The prompt is the secret sauce—we constrain Gemini to return only valid JSON with a specific schema.

# gemini_client.py
import os
import json
import google.generativeai as genai
from dotenv import load_dotenv

load_dotenv()
genai.configure(api_key=os.getenv("GEMINI_API_KEY"))

RECEIPT_PROMPT = """
You are a precise receipt parser. Analyze the provided receipt image and return ONLY a valid JSON object with this exact structure:
{
  "vendor": string,
  "date": string (YYYY-MM-DD if found, else null),
  "line_items": [
    {"description": string, "quantity": number, "unit_price": number, "total_price": number}
  ],
  "subtotal": number,
  "tax": number,
  "total": number,
  "currency": string (3-letter code, default "USD")
}
Rules:
- If a field cannot be determined, use null for strings/numbers and empty array for line_items.
- quantity defaults to 1 if only a total price is visible.
- Return ONLY the JSON object, no markdown fences, no explanatory text.
"""

def extract_receipt(image_path: str) -> dict:
    """Send a receipt image to Gemini 1.5 Flash and return parsed JSON."""
    model = genai.GenerativeModel("gemini-1.5-flash")
    
    # Upload the image file
    image_file = genai.upload_file(image_path)
    
    response = model.generate_content([RECEIPT_PROMPT, image_file])
    
    # Gemini sometimes wraps JSON in ```json fences despite instructions
    raw_text = response.text.strip()
    if raw_text.startswith("```"):
        raw_text = raw_text.split("```")[1]
        if raw_text.startswith("json"):
            raw_text = raw_text[4:]
        raw_text = raw_text.strip()
    
    return json.loads(raw_text)

Why Gemini 1.5 Flash over Pro? Flash is faster (sub-second latency on receipts), cheaper (free tier is generous), and receipts don’t need the reasoning depth of Pro. The 1M token context window is overkill here, but the vision capabilities are identical for OCR tasks.

Step 3: Building the File Watcher

Create watcher.py. We subclass FileSystemEventHandler and override on_created. The handler adds a 2-second debounce to ensure the file is fully written before we touch it.

# watcher.py
import time
import hashlib
from pathlib import Path
from watchdog.events import FileSystemEventHandler

class ReceiptHandler(FileSystemEventHandler):
    def __init__(self, processor, output_dir: Path, dead_letter_dir: Path):
        self.processor = processor
        self.output_dir = output_dir
        self.dead_letter_dir = dead_letter_dir
        self.seen_hashes = set()
    
    def on_created(self, event):
        if event.is_directory:
            return
        
        file_path = Path(event.src_path)
        if file_path.suffix.lower() not in ('.pdf', '.png', '.jpg', '.jpeg'):
            return
        
        # Debounce: wait for file write to complete
        time.sleep(2)
        
        # Deduplicate by SHA-256 hash
        file_hash = self._hash_file(file_path)
        if file_hash in self.seen_hashes:
            print(f"Skipping duplicate: {file_path.name}")
            return
        self.seen_hashes.add(file_hash)
        
        print(f"Processing: {file_path.name}")
        try:
            result = self.processor(file_path)
            output_path = self.output_dir / f"{file_path.stem}.json"
            output_path.write_text(result)
            print(f"  -> Saved: {output_path.name}")
        except Exception as e:
            print(f"  -> Failed: {e}")
            self.dead_letter_dir.mkdir(exist_ok=True)
            file_path.rename(self.dead_letter_dir / file_path.name)
    
    @staticmethod
    def _hash_file(path: Path) -> str:
        return hashlib.sha256(path.read_bytes()).hexdigest()

Why SHA-256 for dedup? File names change, timestamps differ. Hashing the bytes catches identical receipts even if you accidentally drop the same file twice. The set lives in memory, so restarting the script resets it—persist to SQLite if you need cross-session dedup.

Step 4: The Core Extraction Pipeline

Now main.py—the orchestrator. It ties the watcher to the Gemini client, handles PDF-to-image conversion with pdf2image, and writes the final JSON.

# main.py
import json
import time
from pathlib import Path
from watchdog.observers import Observer
from pdf2image import convert_from_path
from gemini_client import extract_receipt
from watcher import ReceiptHandler

WATCH_DIR = Path("receipts")
OUTPUT_DIR = Path("output")
DEAD_LETTER_DIR = Path("dead_letter")

def process_receipt(file_path: Path) -> str:
    """
    Convert PDFs to image if needed, call Gemini, return pretty-printed JSON string.
    """
    if file_path.suffix.lower() == '.pdf':
        # Convert first page to PNG (most receipts are single-page)
        images = convert_from_path(file_path, first_page=1, last_page=1)
        if not images:
            raise ValueError("PDF has no pages")
        temp_path = file_path.with_suffix('.png')
        images[0].save(temp_path, 'PNG')
        result = extract_receipt(str(temp_path))
        temp_path.unlink()  # clean up temp image
    else:
        result = extract_receipt(str(file_path))
    
    # Validate required fields exist
    required = ['vendor', 'date', 'line_items', 'subtotal', 'tax', 'total', 'currency']
    for field in required:
        if field not in result:
            result[field] = None if field != 'line_items' else []
    
    return json.dumps(result, indent=2, ensure_ascii=False)

def main():
    WATCH_DIR.mkdir(exist_ok=True)
    OUTPUT_DIR.mkdir(exist_ok=True)
    DEAD_LETTER_DIR.mkdir(exist_ok=True)
    
    handler = ReceiptHandler(process_receipt, OUTPUT_DIR, DEAD_LETTER_DIR)
    observer = Observer()
    observer.schedule(handler, str(WATCH_DIR), recursive=False)
    observer.start()
    
    print(f"Watching {WATCH_DIR.absolute()} for new receipts...")
    print("Press Ctrl+C to stop.")
    
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
        print("\nShutting down.")
    observer.join()

if __name__ == "__main__":
    main()

The PDF conversion step is critical—Gemini’s vision API accepts images, not PDFs directly. pdf2image uses Poppler under the hood to render page 1 at 300 DPI (default). For multi-page receipts, you’d stitch pages vertically, but 95% of receipts are single-page.

Why validate the response? Even with a strict prompt, LLMs are stochastic. Sometimes Gemini returns null for a field we expect, or omits line_items entirely. The validation block ensures downstream consumers always get a consistent schema, even if values are null.

Step 5: Running the Extractor End-to-End

1. Start the watcher:

   python main.py
   

   You’ll see: Watching /path/to/receipts for new receipts...

2. Drop a receipt in the receipts/ folder. On macOS, you can drag a PDF from Finder. On Linux/WSL, cp ~/Downloads/walmart-receipt.pdf ./receipts/.

3. Watch the output. Within 2-5 seconds, output/walmart-receipt.json appears:

   {
     "vendor": "Walmart",
     "date": "2025-01-15",
     "line_items": [
       {"description": "Milk 1gal", "quantity": 1, "unit_price": 3.49, "total_price": 3.49},
       {"description": "Bread", "quantity": 2, "unit_price": 2.99, "total_price": 5.98}
     ],
     "subtotal": 9.47,
     "tax": 0.76,
     "total": 10.23,
     "currency": "USD"
   }
   

4. Test with a bad file (e.g., a photo of your cat). The script will hit Gemini’s safety filters or return unparseable JSON, catch the exception, and move the file to dead_letter/.

Performance note: The free tier caps at 15 requests per minute. If you’re bulk-processing a backlog of 100 receipts, add a time.sleep(4) between calls or implement a token bucket. For folder-watching use, 15 RPM is plenty—you’re not dropping 15 receipts per minute manually.

Sensible Extensions

This pipeline is a foundation. Here’s where to take it next:

1. CSV export for accounting software. Add a --csv flag that appends extracted data to a running ledger. Tools like QuickBooks and Xero accept CSV imports with vendor, date, and total columns.

2. Multi-page receipt stitching. For receipts that span pages, convert all PDF pages, stitch them vertically with Pillow, and send the combined image. Gemini’s long-context vision handles tall images gracefully.

3. Expense categorization with a second LLM call. After extraction, send the line items to Gemini with a prompt like “Categorize each item as Food, Office Supplies, Travel, or Other.” Append a category field to each line item.

4. Web dashboard with Streamlit. Build a simple UI that shows recent receipts, lets you correct mis-extracted fields, and exports corrected data. Streamlit’s free tier handles this easily.

5. Slack notifications. Use a Slack webhook to post a summary every time a receipt is processed: “Processed Walmart receipt: $10.23 on 2025-01-15.”

If you’re looking to level up your extraction pipelines with more advanced patterns—like integrating structured outputs into RAG systems—check out our guide on building a codebase Q&A tool with LlamaIndex and Supabase for patterns that transfer directly to document processing workflows.

Common Pitfalls and Fixes

FAQ

Q: How accurate is Gemini 1.5 Flash on crumpled, low-light receipt photos? A: Surprisingly good. I’ve tested it on photos taken at restaurant tables with poor lighting, and it correctly extracts ~90% of line items. The biggest failure mode is handwritten totals—it struggles with cursive. For production expense tracking, plan for a manual review step on flagged receipts.

Q: Can I use this for non-English receipts? A: Yes. Gemini 1.5 Flash supports over 100 languages. The prompt is in English, but the model will extract vendor names and line items in their original language. Add "language": string to the output schema if you need language detection.

Q: What happens if I exceed the free tier? A: Google returns a 429 status code. The free tier is 1,500 requests/day—enough for 50 receipts/day with some headroom for retries. If you hit the limit, requests queue until the next UTC day. No charges unless you explicitly enable billing.

Q: Can I deploy this on a headless server? A: Absolutely. The watcher runs fine on any Linux VPS. Just install Poppler, set up the folder structure, and run main.py in a tmux session or as a systemd service. No GUI required.

Q: How does this compare to dedicated OCR APIs like AWS Textract? A: For structured receipt extraction, Gemini 1.5 Flash is 80-90% as accurate as Textract’s expense analyzer—and completely free. Textract charges $0.01-0.05 per page. The gap narrows further with good prompt engineering. For an FDE shipping internal tools, free and “good enough” beats paid and perfect, especially when you can iterate on prompts. This aligns with the core FDE skill of high-leverage prompting and rapid data modeling that turns raw LLM capabilities into production-grade pipelines.

Q: How do I handle receipts that are photos of screens? A: Gemini handles screen photos fine, but moiré patterns can confuse OCR. If you get bad results, try increasing image contrast with Pillow before sending: ImageEnhance.Contrast(image).enhance(2.0). For a deeper dive into vision model limitations, see our piece on why spatial reasoning remains a hard ceiling for vision models.

Q: What’s the next step after extracting JSON? A: Integration. Pipe the JSON into Notion, Airtable, or Google Sheets via their free APIs. Or build a multi-agent research assistant pattern where one agent extracts, another categorizes, and a third generates monthly spending reports. The JSON output is your integration surface—treat it like an API contract.