Build a Flashcard Generator from Lecture Audio Using Whisper + Gemini

What We're Building

A command-line tool that ingests an .mp3 or .wav lecture recording, transcribes it using OpenAI's Whisper model, feeds the transcript to Google Gemini, and produces a CSV file you can drag directly into Anki. No GPU required. No paid APIs. Just free credits, Python, and about 100 lines of glue code.

Feature list:

• Accepts common audio formats (MP3, WAV, M4A)
• Splits long audio into chunks Whisper can handle reliably
• Transcribes via Whisper's base model (free, runs locally)
• Extracts concept–definition Q&A pairs using Gemini 1.5 Flash (free tier)
• Outputs a clean anki_import.csv with columns: Front, Back, Tags
• Skips fluff—only returns actual testable content

This is the kind of tool a Forward Deployed Engineer builds in an afternoon to unblock a student, a researcher, or anyone drowning in recorded lectures. It's not a product. It's a sharp, single-purpose instrument.

Architecture Overview

The pipeline is linear: chunk audio → transcribe → prompt LLM → format CSV. No vector databases, no RAG, no over-engineering. Whisper handles the heavy lifting of speech-to-text locally. Gemini does the cognitive work of identifying what's worth memorizing. The CSV formatter is pure string manipulation.

Prerequisites & Free Tier Setup

You need three things: Python, an OpenAI Whisper install, and a Google Gemini API key.

1. Python 3.10+ Grab it from python.org. Verify:

python --version

2. Whisper (OpenAI, local, free)

pip install openai-whisper

Whisper runs entirely on your machine. The base model is ~142MB and works fine on CPU for short lectures. For multi-hour recordings, you'll want at least 8GB RAM. No API key needed.

3. Google Gemini API (free tier)

• Go to aistudio.google.com
• Sign in with a Google account
• Click "Get API key" → "Create API key"
• Copy the key. The free tier gives you 15 requests per minute and 1,500 requests per day on Gemini 1.5 Flash. More than enough for personal use.

4. pydub (audio chunking)

pip install pydub

pydub requires ffmpeg on your system:

• macOS: brew install ffmpeg
• Ubuntu: sudo apt install ffmpeg
• Windows: download from ffmpeg.org and add to PATH

5. python-dotenv (optional but clean)

pip install python-dotenv

Store your Gemini key in a .env file:

GEMINI_API_KEY=your_key_here

Step 1: Project Scaffolding

Create a single directory and a single Python file. No frameworks.

mkdir flashcard-generator
cd flashcard-generator
touch main.py .env

main.py starts with imports and config:

import os
import json
import csv
import whisper
import google.generativeai as genai
from pydub import AudioSegment
from dotenv import load_dotenv

load_dotenv()

GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
genai.configure(api_key=GEMINI_API_KEY)

WHISPER_MODEL = "base"
CHUNK_LENGTH_MS = 30_000  # 30 seconds per chunk

Why 30-second chunks? Whisper's base model has a 30-second context window. Longer audio gets truncated silently. Chunking ensures every word gets transcribed.

Step 2: Transcribing Audio with Whisper

Load the model once, then process chunks sequentially.

def transcribe_audio(file_path: str) -> str:
    model = whisper.load_model(WHISPER_MODEL)
    audio = AudioSegment.from_file(file_path)
    
    chunks = [audio[i:i + CHUNK_LENGTH_MS] 
              for i in range(0, len(audio), CHUNK_LENGTH_MS)]
    
    full_transcript = []
    for idx, chunk in enumerate(chunks):
        chunk_path = f"chunk_{idx}.wav"
        chunk.export(chunk_path, format="wav")
        result = model.transcribe(chunk_path)
        full_transcript.append(result["text"])
        os.remove(chunk_path)  # clean up temp file
    
    return " ".join(full_transcript)

This writes each chunk to a temporary .wav file because Whisper's transcribe() expects a file path. We clean up immediately after transcribing. If you're processing a 90-minute lecture, expect ~180 chunks and roughly 10-15 minutes of processing on a modern CPU.

Pro tip: If you hit memory issues, add fp16=False to whisper.load_model() to force 32-bit float mode on CPU.

Step 3: Extracting Flashcards with Gemini

This is where the magic happens. We send the full transcript to Gemini with a structured prompt that forces JSON output—critical for reliable parsing.

def extract_flashcards(transcript: str) -> list[dict]:
    model = genai.GenerativeModel("gemini-1.5-flash")
    
    prompt = f"""
You are an expert study coach. Given a lecture transcript, identify 10-20 key concepts 
and create question-answer flashcard pairs. Focus on definitions, cause-effect relationships, 
and testable facts. Skip filler, anecdotes, and meta-commentary.

Return ONLY valid JSON. No markdown fences, no explanation. Format:
{{ "flashcards": [
  {{ "front": "What is X?", "back": "X is a Y that does Z.", "tags": "topic" }}
] }}

Transcript:
{transcript[:30_000]}
"""
    
    response = model.generate_content(prompt)
    raw = response.text.strip()
    
    # Gemini sometimes wraps JSON in ```json fences despite instructions
    if raw.startswith("```"):
        raw = raw.split("```")[1]
        if raw.startswith("json"):
            raw = raw[4:]
    
    data = json.loads(raw)
    return data["flashcards"]

Why truncate to 30,000 characters? Gemini 1.5 Flash has a 1M token context window, but the free tier throttles large inputs. Truncating keeps latency low and stays within the free tier's comfort zone. If your lecture transcript exceeds 30k chars, consider summarizing it first (see Extensions).

JSON enforcement matters. Without the explicit instruction, Gemini happily returns markdown, commentary, and unusable garbage. The cleaning block handles the most common failure mode: the model wrapping JSON in code fences despite being told not to.

Step 4: Generating the Anki CSV

Anki expects a specific CSV format: column 1 is the front of the card, column 2 is the back, and optional column 3 is tags (semicolon-separated).

def write_anki_csv(flashcards: list[dict], output_path: str = "anki_import.csv"):
    with open(output_path, "w", newline="", encoding="utf-8") as f:
        writer = csv.writer(f)
        # Anki header row is technically optional but good practice
        writer.writerow(["Front", "Back", "Tags"])
        for card in flashcards:
            front = card["front"].replace("\n", "<br>")  # Anki uses HTML line breaks
            back = card["back"].replace("\n", "<br>")
            tags = card.get("tags", "lecture")
            writer.writerow([front, back, tags])
    print(f"Wrote {len(flashcards)} cards to {output_path}")

The <br> substitution is important—Anki renders card content as HTML. Raw newlines get collapsed. If you want bold or italic, you can extend this to handle markdown→HTML conversion.

Step 5: Running the Pipeline End-to-End

Wire everything together with a main() function:

def main(audio_path: str):
    print(f"Transcribing {audio_path}...")
    transcript = transcribe_audio(audio_path)
    print(f"Transcript length: {len(transcript)} chars")
    
    print("Extracting flashcards with Gemini...")
    flashcards = extract_flashcards(transcript)
    
    write_anki_csv(flashcards)
    print("Done. Import anki_import.csv into Anki.")

if __name__ == "__main__":
    import sys
    if len(sys.argv) < 2:
        print("Usage: python main.py lecture.mp3")
        sys.exit(1)
    main(sys.argv[1])

Run it:

python main.py my_lecture.mp3

Importing into Anki:

1. Open Anki → File → Import
2. Select anki_import.csv
3. Choose "Basic" note type (or a custom one with Front/Back/Tags fields)
4. Map columns: Field 1 → Front, Field 2 → Back, Field 3 → Tags
5. Import. You're done.

Sensible Extensions

This is a minimum viable pipeline. Here's where you take it next:

1. Speaker diarization. If your lecture has multiple speakers (Q&A, panel), use pyannote.audio (free for research) to label who said what before transcription. Makes flashcards contextually richer.

2. Summarize before extraction. For 2-hour lectures, add a Gemini summarization pass before flashcard extraction. Prompt: "Summarize this lecture into 500 words of key points." Feed the summary to the flashcard prompt. Reduces noise dramatically.

3. Automatic deck organization. Have Gemini output a subdeck field. Post-process the CSV to create Anki subdecks by topic. This requires the AnkiConnect plugin and a few extra lines of Python.

4. Batch processing. Drop a folder of .mp3 files and process them overnight. Add a --watch flag that monitors a directory for new recordings and auto-generates decks.

5. Confidence scoring. Ask Gemini to rate each flashcard's quality (1-5). Filter out low-confidence cards before writing the CSV. Fewer cards, higher signal.

This pattern—transcribe, extract, format—is the same one I've used in enterprise deployments where we needed to turn customer call recordings into structured knowledge base articles. The tools change (Whisper → enterprise ASR, Gemini → GPT-4), but the pipeline shape is identical. If you're curious about how FDEs take prototypes like this and deploy them at enterprise scale, check out What a Forward Deployed Engineer Actually Does in a Week: Code, Customers, Chaos.

Common Pitfalls

Whisper hallucinates on silence. If your lecture has long pauses, Whisper may invent text. Trim silence with pydub's strip_silence() before chunking.

Gemini returns malformed JSON. The prompt engineering here is robust, but edge cases happen. Add a try/except around json.loads() with a retry loop that asks Gemini to fix its own JSON. Two retries covers 99% of failures.

Free tier rate limits. Gemini's free tier is 15 RPM. If you're processing multiple lectures back-to-back, add time.sleep(4) between API calls.

Large audio files eat RAM. Loading a 200MB WAV into memory with AudioSegment.from_file() can spike RAM usage. For production use, stream chunks from disk instead of loading the entire file. But for a 90-minute lecture at 16-bit mono, you're looking at ~500MB—manageable on any modern laptop.

Anki import fails silently. If Anki imports 0 cards, check that your CSV uses UTF-8 encoding and that the header row matches your note type fields exactly. Anki is picky.

FAQ

Q: Can I use this for non-English lectures? Yes. Whisper's base model supports 99 languages. Gemini 1.5 Flash works well in most major languages. No code changes needed—just feed it non-English audio.

Q: Why not use Gemini's native audio input? Gemini 1.5 Flash does accept audio directly, but the free tier's audio processing is slower and less reliable than Whisper for long-form transcription. Whisper is purpose-built for this task.

Q: How accurate are the flashcards? With a clean lecture recording (minimal background noise, clear speaker), expect 85-90% of flashcards to be factually correct and useful. Always spot-check before relying on them for exam prep.

Q: Can I deploy this as a web app? Absolutely. Wrap the pipeline in a Flask or FastAPI endpoint, add a file upload form, and you've got a deployable tool. The architecture doesn't change—just the interface. For a deep dive on taking prototypes to production, read Case Study: Deploying an LLM Feature at an Enterprise Customer as an FDE.

Q: What if my lecture is 3 hours long? The 30k character truncation in the Gemini prompt will cut off most of it. Add a summarization step (see Extensions) or use a sliding window approach: extract flashcards from each 30k-char segment, then deduplicate.

Q: Where do I go from here as a builder? This project sits at the intersection of audio processing, prompt engineering, and pipeline design—core FDE skills. If you want to level up your ability to ship useful AI tools fast, the patterns here repeat across domains. For a look at how these skills play out in customer-facing engineering roles, How Palantir-Style FDEs Embed with Customers: Rituals, Artifacts, and Trust is a solid next read.