Build a GitHub PR Review Bot with Groq + Local Ollama Fallback

What We're Building

A GitHub App that fires on every pull request, grabs the diff, runs it through a fast LLM for style and logic checks, and posts inline review comments directly on the code. The primary engine is Groq's free tier (blazing fast LLaMA 3 inference). For repos that can't leave the local network, we add a local Ollama fallback that kicks in when the PR carries a specific label.

Feature list:

• Inline PR comments tied to specific diff lines
• Groq free tier as default (no credit card, 30 req/min)
• Automatic Ollama fallback for sensitive-repo labeled PRs
• Runs entirely on Cloudflare Workers free tier (100k req/day)
• Configurable review prompt via repo .github/pr-review-bot.md
• Skips draft PRs and bot-authored PRs to avoid noise

Architecture & Flow

The worker listens for pull_request.opened and pull_request.synchronize events. It checks the PR labels. If sensitive-repo is present, it routes to your local Ollama instance (exposed via a tunnel like ngrok or Cloudflare Tunnel). Otherwise, it hits Groq. The LLM response is parsed into a structured JSON array of {path, line, body} objects, then posted via the GitHub API as review comments.

Prerequisites (All Free Tier)

• GitHub Account – to create the App. Sign up
• Groq API Key – free tier, no credit card. Get one here
• Ollama – installed locally. curl -fsSL https://ollama.com/install.sh | sh
• Cloudflare Account – Workers free tier. Sign up
• Node.js 18+ – for Wrangler CLI. npm install -g wrangler
• A tunnel tool – ngrok free tier or cloudflared tunnel to expose your local Ollama

Pull a model locally: ollama pull llama3:8b (or mistral:7b for lower RAM).

Step 1: Scaffold the GitHub App

1. Go to Settings > Developer settings > GitHub Apps > New GitHub App.
2. GitHub App name: pr-review-bot-<your-handle>
3. Homepage URL: your repo or placeholder
4. Webhook URL: leave blank for now (we'll fill after Worker deploy)
5. Webhook secret: generate a strong random string, save it as WEBHOOK_SECRET
6. Permissions:
   • Pull requests: Read & Write (for posting comments)
   • Metadata: Read (mandatory)
7. Subscribe to events: Pull request
8. Create the App. Generate a private key (.pem file) and note the App ID.
9. Install the App on a test repo.

Step 2: Build the Cloudflare Worker

Initialize the project:

mkdir pr-review-bot && cd pr-review-bot
npm create cloudflare@latest -- --template worker-typescript
npm install @octokit/auth-app octokit

Set secrets:

wrangler secret put GROQ_API_KEY
wrangler secret put GITHUB_APP_ID
wrangler secret put GITHUB_PRIVATE_KEY  # paste the full .pem content
wrangler secret put WEBHOOK_SECRET
wrangler secret put OLLAMA_ENDPOINT  # e.g., https://your-tunnel.ngrok-free.app

Core worker structure (src/index.ts):

import { App } from '@octokit/app';

interface Env {
  GROQ_API_KEY: string;
  GITHUB_APP_ID: string;
  GITHUB_PRIVATE_KEY: string;
  WEBHOOK_SECRET: string;
  OLLAMA_ENDPOINT: string;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    if (request.method !== 'POST') return new Response('OK');
    
    // Verify webhook signature
    const sig = request.headers.get('x-hub-signature-256');
    if (!sig || !(await verifySignature(request, env.WEBHOOK_SECRET, sig))) {
      return new Response('Unauthorized', { status: 401 });
    }

    const event = request.headers.get('x-github-event');
    const payload: any = await request.json();

    if (event === 'pull_request' && 
        ['opened', 'synchronize'].includes(payload.action)) {
      // Don't await — GitHub expects 200 quickly
      ctx.waitUntil(handlePullRequest(payload, env));
    }

    return new Response('Accepted', { status: 202 });
  }
};

We return 202 immediately so GitHub doesn't timeout. The ctx.waitUntil keeps the worker alive for background processing.

Step 3: Implement the Groq Review Logic

Groq's free tier gives us ~30 requests per minute to models like llama3-70b-8192 or the faster llama3-8b-8192. The 8B model is more than enough for diff review.

async function reviewWithGroq(diff: string, instructions: string, apiKey: string) {
  const systemPrompt = `You are a senior code reviewer. Review the following git diff.
Follow these instructions if provided: ${instructions || 'Check for logic errors, style issues, potential bugs, and suggest improvements.'}

Return ONLY a JSON array of review comments. Each object must have:
- "path": the file path from the diff
- "line": the line number in the NEW file (the @@ diff header shows the new file start line)
- "body": the comment text (concise, actionable)

If no issues, return an empty array [].`;

  const response = await fetch('https://api.groq.com/openai/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      model: 'llama3-8b-8192',
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: diff }
      ],
      temperature: 0.2,
      max_tokens: 4096
    })
  });

  const data: any = await response.json();
  const content = data.choices?.[0]?.message?.content || '[]';
  
  // Extract JSON from possible markdown fences
  const jsonMatch = content.match(/```(?:json)?\s*([\s\S]*?)\s*```/) || [null, content];
  return JSON.parse(jsonMatch[1].trim());
}

Key trick: We ask the LLM to return structured JSON. The @@ diff header tells us the new file's starting line — we include that in the diff context so the model can calculate absolute line numbers.

Step 4: Add the Local Ollama Fallback

When the PR has a sensitive-repo label, we route to the local Ollama instance. This keeps the code on-prem.

async function reviewWithOllama(diff: string, instructions: string, endpoint: string) {
  const prompt = `Review this git diff. ${instructions || 'Check for bugs, style, logic.'}
Return ONLY a JSON array: [{"path":"...", "line": number, "body":"..."}].

Diff:
${diff}`;

  const response = await fetch(`${endpoint}/api/generate`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'llama3:8b',
      prompt: prompt,
      stream: false,
      options: { temperature: 0.2 }
    })
  });

  const data: any = await response.json();
  const jsonMatch = data.response.match(/```(?:json)?\s*([\s\S]*?)\s*```/) || 
                    [null, data.response];
  return JSON.parse(jsonMatch[1].trim());
}

For the fallback to work, your local Ollama must be reachable. Use cloudflared tunnel (free) or ngrok:

# Option A: cloudflared
cloudflared tunnel --url http://localhost:11434

# Option B: ngrok
ngrok http 11434

Set the resulting HTTPS URL as OLLAMA_ENDPOINT secret.

Step 5: Wire Up GitHub Inline Comments

This is where we authenticate as the GitHub App and post review comments.

async function postInlineComments(
  payload: any, 
  comments: Array<{path: string; line: number; body: string}>, 
  env: Env
) {
  const app = new App({
    appId: env.GITHUB_APP_ID,
    privateKey: env.GITHUB_PRIVATE_KEY
  });

  const octokit = await app.getInstallationOctokit(
    payload.installation.id
  );

  const { owner, repo } = payload.repository;
  const pullNumber = payload.pull_request.number;
  const commitId = payload.pull_request.head.sha;

  // Create a review with inline comments
  await octokit.request(
    'POST /repos/{owner}/{repo}/pulls/{pull_number}/reviews',
    {
      owner: owner.login,
      repo: repo.name,
      pull_number: pullNumber,
      commit_id: commitId,
      event: 'COMMENT',
      comments: comments.map(c => ({
        path: c.path,
        line: c.line,
        body: c.body
      }))
    }
  );
}

The Octokit App instance handles JWT generation and installation token exchange automatically. Each review is tied to the head commit SHA so comments appear on the correct diff.

Full handlePullRequest function:

async function handlePullRequest(payload: any, env: Env) {
  const pr = payload.pull_request;
  
  // Skip drafts and bot PRs
  if (pr.draft || pr.user.type === 'Bot') return;

  // Check for sensitive label
  const labels = pr.labels?.map((l: any) => l.name) || [];
  const isSensitive = labels.includes('sensitive-repo');

  // Fetch the diff
  const diffResponse = await fetch(pr.diff_url, {
    headers: { 'Authorization': `token ${env.GITHUB_TOKEN}` }
  });
  const diff = await diffResponse.text();

  // Check for custom instructions in the repo
  let instructions = '';
  try {
    const configUrl = `https://raw.githubusercontent.com/${payload.repository.full_name}/main/.github/pr-review-bot.md`;
    const configResp = await fetch(configUrl);
    if (configResp.ok) instructions = await configResp.text();
  } catch {}

  // Route to appropriate LLM
  const comments = isSensitive 
    ? await reviewWithOllama(diff, instructions, env.OLLAMA_ENDPOINT)
    : await reviewWithGroq(diff, instructions, env.GROQ_API_KEY);

  if (comments.length > 0) {
    await postInlineComments(payload, comments, env);
  }
}

Step 6: Deploy and Test

wrangler deploy

Copy the deployed Worker URL (e.g., https://pr-review-bot.<your-subdomain>.workers.dev) and paste it into your GitHub App's Webhook URL setting. Make sure the content type is application/json.

Test it:

1. Open a PR on your test repo
2. The bot should post inline comments within 10-15 seconds (Groq) or 30-60 seconds (Ollama)
3. Add the sensitive-repo label and push a new commit — comments should now come from Ollama

Sensible Extensions

• Diff chunking: Large diffs exceed context windows. Split the diff by file and review each independently, then merge the JSON arrays.
• Review summary comment: Post a top-level PR comment with an overall summary alongside inline nits. The Groq model handles this well with a two-pass approach.
• Ignore patterns: Add a .github/pr-review-bot-ignore file with glob patterns to skip generated files, lockfiles, or vendored code.
• Confidence scoring: Ask the model to include a confidence (0-1) field. Only post comments above a threshold to reduce noise.
• Slack notifications: When the review is posted, fire a Slack webhook to the team channel. For a full build on this pattern, see Automate Daily Slack Channel Summaries with n8n and Groq's Free Tier.

Common Pitfalls

1. Webhook timeout: GitHub expects a response within 10 seconds. Always return 202 immediately and use ctx.waitUntil for the actual work.
2. Line number mapping: The @@ -a,b +c,d @@ header means the new file starts at line c. Include this context in the prompt so the model calculates correctly. If the model still gets it wrong, post-process by parsing the diff and validating line ranges.
3. Ollama tunnel instability: Free ngrok URLs change on restart. Use cloudflared tunnel with a named tunnel for a stable subdomain, or reserve a free ngrok domain.
4. Private key formatting: The .pem file contains newlines. When storing as a Cloudflare secret, keep the \n characters intact — don't strip them.
5. Rate limits: Groq free tier is 30 req/min. For busy repos, add a simple in-memory queue or debounce so multiple synchronize events within a minute don't all trigger reviews. GitHub's own API rate limit for Apps is 5,000/hr, which is plenty.
6. Model context window: llama3-8b has an 8K context window. Diffs larger than ~6K tokens will be truncated. Implement chunking early if your repo has large PRs. For a deeper dive on context management patterns, check out Claude Code Sessions: Cost Engineering and Context Reuse Patterns That Ship Faster.

FAQ

Why not just use GitHub Copilot code review? Copilot's review feature is solid but doesn't let you customize the review prompt per repo, route to local models, or control exactly when reviews fire. This bot gives you full control over the review criteria.

Can I use a different Groq model? Absolutely. Swap llama3-8b-8192 for mixtral-8x7b-32768 if you need a larger context window, or gemma2-9b-it for a different style. All are on the free tier.

What if I want to review more than just code style? The .github/pr-review-bot.md file is your control surface. Write a custom prompt that checks for security vulnerabilities, i18n issues, accessibility regressions, or team-specific conventions. The model follows instructions well. For an example of prompt engineering at this level of specificity, Prompting as Delegation: Why AI-Assisted Coding Mirrors Engineering Management breaks down the mental model.

How do I handle monorepos with different review rules per directory? Extend the config file to a JSON format with path-specific rules, or check for multiple config files at different directory levels. The worker fetches the diff per-file anyway, so you can match each file path against a rules map.

Is the local Ollama fallback actually secure? The tunnel is encrypted in transit, and the Worker-to-Ollama connection is HTTPS. For production sensitive workflows, add a shared secret header that your Worker sends and Ollama (behind a lightweight proxy) validates. Never expose raw Ollama without authentication on the public internet.