Build an On-Call Incident Summarizer with Gemini's Free Tier

What We're Building

We are building a local CLI tool that takes the drudgery out of incident response. You feed it a time-boxed slice of structured logs (JSON lines exported from CloudWatch or Datadog) and a raw Slack thread dump. The tool uses Google Gemini's free tier to identify the blast radius, construct a timeline, and output a polished postmortem draft in markdown.

Feature List:

• Time-Boxed Filtering: Only analyzes logs within the incident window.
• Dual Input: Accepts both machine logs (JSON) and human chat (plain text).
• Root Cause Analysis: Gemini infers the "why" between correlated events.
• Timeline Generation: Auto-extracts a chronological sequence of failure events.
• Markdown Output: Produces a clean, structured report ready for Confluence or Notion.

Architecture Overview

We aren't building a persistent service. This is a single Python script that acts as an orchestrator. It reads local files, constructs a massive prompt, calls the Gemini API, and writes the result to disk.

The secret sauce is the Context Window Assembler. Free-tier Gemini models have generous but finite context limits. We must intelligently truncate noisy logs to fit the critical errors alongside the human discussion without hitting the token ceiling.

Prerequisites (All Free Tier)

Before writing code, grab these:

1. Python 3.9+: Most systems have it. If not, download from python.org.
2. Google AI Studio API Key: This is the engine.
   • Visit aistudio.google.com.
   • Sign in with a Google account.
   • Click "Get API Key" and create a key in a new project.
   • The free tier offers a generous rate limit perfect for drafting postmortems.
3. Sample Data: You need a logs.json file (array of structured log objects) and a slack.txt file.

Step 1: Setting Up the Python Environment

We only need the google-generativeai SDK. We'll load the API key from an environment variable to avoid hardcoding secrets.

mkdir incident-summarizer && cd incident-summarizer
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install python-dotenv google-generativeai

Create a .env file:

GEMINI_API_KEY=AIzaSy...your_key_here

Step 2: Parsing Structured Logs (JSON)

We need a resilient parser. Log files might be an array of objects or newline-delimited JSON. We'll read the file, strip whitespace, and handle both formats. We also filter by incident start/end times.

Create log_parser.py:

import json
from datetime import datetime, timezone

def parse_logs(filepath, start_time_str, end_time_str):
    """
    Reads JSON logs, filters by time window.
    Expects ISO 8601 strings for start/end.
    Assumes logs have a 'timestamp' field.
    """
    start_dt = datetime.fromisoformat(start_time_str)
    end_dt = datetime.fromisoformat(end_time_str)
    
    with open(filepath, 'r') as f:
        raw = f.read().strip()
    
    # Handle both a JSON array and NDJSON
    if raw.startswith('['):
        try:
            all_logs = json.loads(raw)
        except json.JSONDecodeError:
            raise ValueError("Corrupted JSON array")
    else:
        all_logs = []
        for line in raw.splitlines():
            if line.strip():
                all_logs.append(json.loads(line))
    
    filtered = []
    for log in all_logs:
        ts = log.get('timestamp')
        if not ts:
            continue
        log_dt = datetime.fromisoformat(ts.replace('Z', '+00:00'))
        if start_dt <= log_dt <= end_dt:
            filtered.append(log)
    
    # Sort just in case
    filtered.sort(key=lambda x: x['timestamp'])
    return filtered

Step 3: Parsing the Slack Thread Dump

Slack exports are messy. We want the raw text, but we need to strip timestamps and usernames to save tokens for the LLM. We'll keep the chronological order.

Create slack_parser.py:

import re

def parse_slack_dump(filepath):
    """
    Extracts a clean conversation flow from a Slack dump.
    Removes noisy metadata like user IDs and precise timestamps.
    """
    with open(filepath, 'r') as f:
        lines = f.readlines()
    
    conversation = []
    # Regex to match typical Slack export lines: "[10:15 AM] John Doe: message"
    pattern = re.compile(r'^\[.*?\]\s+(.*?):\s+(.*)')
    
    for line in lines:
        match = pattern.match(line)
        if match:
            user = match.group(1)
            message = match.group(2)
            conversation.append(f"{user}: {message}")
        else:
            # Continuation of previous message
            if conversation and line.strip():
                conversation[-1] += " " + line.strip()
    
    return "\n".join(conversation)

Step 4: The Prompting Strategy for Gemini

This is where the engineering judgment happens. We need to force structured output from Gemini so we can parse it, but we also want a polished narrative. We'll use a strict system prompt and request a specific markdown format.

Create prompter.py:

import google.generativeai as genai
import os
from dotenv import load_dotenv

load_dotenv()

genai.configure(api_key=os.getenv("GEMINI_API_KEY"))

# Using the free tier model. It's fast and sufficient for summarization.
model = genai.GenerativeModel('gemini-1.5-flash')

def generate_postmortem(logs, slack_text, incident_id):
    # Truncate logs to avoid token limits. Keep error/fatal severity first.
    # This is a naive priority filter; production code might use embeddings.
    error_logs = [l for l in logs if l.get('severity') in ['ERROR', 'FATAL', 'CRITICAL']]
    info_logs = [l for l in logs if l.get('severity') not in ['ERROR', 'FATAL', 'CRITICAL']]
    
    # Combine, errors first, then a sample of info logs
    combined_logs = error_logs + info_logs[:50]  # Limit to 50 info logs
    log_str = json.dumps(combined_logs, indent=2)
    
    # If still too long, slice the string. A token is roughly 4 chars.
    if len(log_str) > 30000:
        log_str = log_str[:30000] + "\n... [TRUNCATED]"
    
    system_prompt = """
You are an SRE expert. Analyze the provided structured logs and Slack conversation.
Generate a postmortem in the following markdown format. Do not deviate.

## Incident Postmortem: [Incident ID]

### Executive Summary
[2-3 sentences summarizing impact and root cause]

### Timeline (UTC)
- `[HH:MM]` [Event description]
- `[HH:MM]` [Event description]

### Root Cause Analysis
[Detailed explanation of the technical failure]

### Impact
- **Duration:** [X minutes]
- **Users Affected:** [Description]
- **Data Loss:** [Yes/No]

### Action Items
- [ ] [Task 1]
- [ ] [Task 2]
"""

    user_prompt = f"""
Incident ID: {incident_id}

--- SLACK CONVERSATION ---
{slack_text}

--- STRUCTURED LOGS (Errors first) ---
{log_str}
"""
    
    full_prompt = f"{system_prompt}\n\n{user_prompt}"
    
    response = model.generate_content(full_prompt)
    return response.text

Step 5: Assembling the Postmortem Markdown

We could just print the response, but let's add a wrapper to save it cleanly and inject the actual incident time boundaries.

Create writer.py:

from datetime import datetime
import os

def save_report(content, incident_id, start_time, end_time):
    output_dir = "postmortems"
    os.makedirs(output_dir, exist_ok=True)
    
    # Create a clean filename
    date_str = datetime.now().strftime("%Y%m%d_%H%M")
    filename = f"{output_dir}/{date_str}_{incident_id}.md"
    
    # Add metadata header to the file
    header = f"---\nIncident: {incident_id}\nWindow: {start_time} to {end_time}\nGenerated: {datetime.now().isoformat()}\n---\n\n"
    
    with open(filename, 'w') as f:
        f.write(header + content)
    
    print(f"Postmortem saved to {filename}")
    return filename

Step 6: The Main Execution Script

Bring it all together in main.py.

import argparse
from log_parser import parse_logs
from slack_parser import parse_slack_dump
from prompter import generate_postmortem
from writer import save_report

def main():
    parser = argparse.ArgumentParser(description='Generate incident postmortems using Gemini.')
    parser.add_argument('--logs', required=True, help='Path to logs.json')
    parser.add_argument('--slack', required=True, help='Path to slack.txt')
    parser.add_argument('--start', required=True, help='Incident start time (ISO 8601)')
    parser.add_argument('--end', required=True, help='Incident end time (ISO 8601)')
    parser.add_argument('--id', required=True, help='Incident ID (e.g. INC-142)')
    
    args = parser.parse_args()
    
    print(f"Parsing logs from {args.start} to {args.end}...")
    logs = parse_logs(args.logs, args.start, args.end)
    print(f"Found {len(logs)} relevant log entries.")
    
    print("Parsing Slack thread...")
    slack_text = parse_slack_dump(args.slack)
    
    print("Drafting postmortem with Gemini...")
    postmortem_md = generate_postmortem(logs, slack_text, args.id)
    
    print("Saving report...")
    save_report(postmortem_md, args.id, args.start, args.end)
    
    print("Done!")

if __name__ == "__main__":
    main()

Running the Summarizer

Export your API key and run the script. Make sure your time strings are clean.

export GEMINI_API_KEY="your-key-here"

python main.py \
  --logs ./sample_data/logs.json \
  --slack ./sample_data/slack.txt \
  --start "2025-05-20T14:30:00Z" \
  --end "2025-05-20T15:45:00Z" \
  --id "INC-421"

Sample logs.json:

[
  {"timestamp": "2025-05-20T14:31:05Z", "severity": "INFO", "message": "Request processed"},
  {"timestamp": "2025-05-20T14:35:22Z", "severity": "ERROR", "message": "DB connection timeout"},
  {"timestamp": "2025-05-20T14:35:23Z", "severity": "FATAL", "message": "Service restart initiated"}
]

Sample slack.txt:

[2:35 PM] Alice: Hey, is the checkout page down?
[2:36 PM] Bob: Seeing 500s here. Checking logs.
[2:40 PM] Alice: Looks like the primary DB is refusing connections.

Extensions and Production Hardening

This script works for a single incident. To make it production-grade:

• Token Management: Implement a tiktoken counter to precisely truncate logs instead of the naive char limit.
• Streaming: Use stream=True in the Gemini SDK to see the report generate in real-time.
• Security: Never send PII in logs. Implement a regex redaction layer before the API call.
• CI/CD Integration: Wrap this in a Docker container and trigger it via a webhook when an incident channel is archived.

For a deeper look at automating workflows with LLMs, check out our guide on Building a Smart Clipboard That Summarizes and Translates Anything You Copy with Ollama.

Common Pitfalls

1. Hallucinated Timelines: The LLM might invent precise millisecond timestamps. The prompt forces it to use the data provided, but always verify the timeline against the raw logs.
2. Token Overflow: Free tier Gemini models have a 1M token context window, but the output limit is smaller. If logs are huge, the script silently truncates them. Check the [TRUNCATED] marker in your prompt.
3. JSON Parsing Errors: Logs from Datadog and CloudWatch have different schemas. Ensure your log_parser.py maps the correct timestamp and severity fields.
4. API Key Exposure: Beginners commit the .env file. Add .env to your .gitignore immediately.

FAQ

Q: Why not use a local model like Ollama? A: For quick incident summaries, you want zero infrastructure overhead. Gemini's free tier is always available and fast. If you want to run entirely air-gapped, our guide on Building a Smart Clipboard with Ollama shows the local approach.

Q: Can I use this for compliance audits? A: No. This is a draft assistant. The LLM can hallucinate details. Always have a human review the root cause analysis before publishing it externally.

Q: How do I handle multi-channel incidents? A: Concatenate the Slack dumps before feeding them to the parser, but keep them in chronological order. The slack_parser.py is stateless and will just merge the flows.

Q: The output markdown formatting is broken. How do I fix it? A: Gemini Flash sometimes adds extra backticks or explanatory text. The system prompt is strict, but if it breaks, you can increase the temperature to 0.0 for more deterministic output, or add a post-processing step that strips anything before the first ## heading.

Q: How do I move from incident response to proactive engineering? A: Automating postmortems is the first step. The real FDE skill is knowing when to hand off these custom toolchains to core engineering. We cover that transition in detail in Scaling Yourself: When and How an FDE Hands Off to Core Engineering.

Q: Can I use this to tailor my resume for SRE roles? A: Building tools like this is a great portfolio project. If you are looking to automate job applications too, see our guide on Building a Resume Tailoring Agent Using Gemini's Free Tier.