Build a GitHub Issue Triager Using Groq and Cloudflare Workers

What We're Building

We're shipping a serverless GitHub App that acts as a first-responder for every new issue. The moment an issue lands, our worker fires, asks Llama 3 (via Groq) to analyze the title and body, then programmatically applies relevant labels, suggests an assignee, and posts a structured triage comment.

Feature list:

• Listens for issues.opened webhook events from GitHub
• Sends issue content to Groq's Llama 3 70B/8B (free tier) for classification
• Maps the LLM's output to your repo's existing labels (bug, enhancement, docs, etc.)
• Attempts to match context against a pre-defined list of contributors for routing
• Posts a comment summarizing the triage decision
• Runs entirely on Cloudflare Workers' free tier (100k requests/day)
• Zero cold starts, zero infrastructure to manage

Architecture Overview

The flow is dead simple: GitHub hits our worker, the worker calls Groq, Groq returns structured JSON, and the worker mutates the issue on GitHub. No databases, no queues—just three services talking HTTP.

Prerequisites (All Free Tier)

Before touching code, grab these accounts and keys. Everything here stays within free limits unless you're processing thousands of issues an hour.

Important: The GitHub App requires a publicly accessible webhook URL. Cloudflare Workers gives you a *.workers.dev domain for free, which is perfect for development. For production, consider tunneling with ngrok during local testing or just use the workers.dev domain directly.

Step 1: Scaffold the Cloudflare Worker

Open a terminal and bootstrap a new worker project. We'll use the create-cloudflare CLI for a TypeScript template.

npm create cloudflare@latest issue-triager -- --type=webworker
cd issue-triager
npm install

Replace the generated src/index.ts with a minimal skeleton to confirm deployment works:

export default {
  async fetch(request: Request): Promise<Response> {
    // Health check or future webhook handler
    if (request.method === 'POST') {
      const body = await request.text();
      console.log('Received webhook:', body.slice(0, 200));
      return new Response('OK', { status: 200 });
    }
    return new Response('Send a POST to this endpoint', { status: 200 });
  },
};

Deploy it to verify:

npx wrangler deploy

Note the *.workers.dev URL from the output. You'll need it in the next step.

Step 2: Register a GitHub App

Head to Settings > Developer settings > GitHub Apps in your account and click "New GitHub App". Fill in:

• GitHub App name: issue-triager-{your-username} (must be unique)
• Homepage URL: Your repo URL or the worker URL
• Webhook URL: https://your-worker.workers.dev
• Webhook secret: Generate a random string (openssl rand -hex 32) and save it—this verifies payloads are genuinely from GitHub
• Permissions:
  • Repository > Issues: Read & write
  • Repository > Metadata: Read-only (auto-selected)
• Subscribe to events: Issues (check the box)

After creation, generate a private key (.pem file) and download it. Note the App ID from the top of the settings page. Install the app on your target repository from the "Install App" sidebar.

Store these secrets for your worker:

npx wrangler secret put GITHUB_APP_ID
# Paste your App ID (numeric)

npx wrangler secret put GITHUB_PRIVATE_KEY
# Paste the full contents of the .pem file, including the BEGIN/END lines

npx wrangler secret put GITHUB_WEBHOOK_SECRET
# Paste your webhook secret

npx wrangler secret put GROQ_API_KEY
# Paste your Groq API key from console.groq.com

Step 3: Implement the Webhook Handler

Now we build the real handler. First, install the dependencies we need for JWT generation and request signing:

npm install @octokit/auth-app octokit

Update src/index.ts with webhook verification and the main dispatch logic. We split concerns: verify the signature, parse the event, and hand off to the triage function.

import { App } from '@octokit/app';

export interface Env {
  GITHUB_APP_ID: string;
  GITHUB_PRIVATE_KEY: string;
  GITHUB_WEBHOOK_SECRET: string;
  GROQ_API_KEY: string;
}

async function verifySignature(payload: string, signature: string, secret: string): Promise<boolean> {
  const encoder = new TextEncoder();
  const key = await crypto.subtle.importKey(
    'raw',
    encoder.encode(secret),
    { name: 'HMAC', hash: 'SHA-256' },
    false,
    ['sign']
  );
  const sigBytes = new Uint8Array(payload.length);
  for (let i = 0; i < payload.length; i++) sigBytes[i] = payload.charCodeAt(i);
  const hmac = await crypto.subtle.sign('HMAC', key, sigBytes);
  const expected = 'sha256=' + Array.from(new Uint8Array(hmac))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
  return expected === signature;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    if (request.method !== 'POST') {
      return new Response('Method not allowed', { status: 405 });
    }

    const signature = request.headers.get('x-hub-signature-256') || '';
    const event = request.headers.get('x-github-event') || '';
    const deliveryId = request.headers.get('x-github-delivery') || '';
    const payload = await request.text();

    // Verify webhook signature
    const isValid = await verifySignature(payload, signature, env.GITHUB_WEBHOOK_SECRET);
    if (!isValid) {
      return new Response('Invalid signature', { status: 401 });
    }

    // Only process new issues
    if (event !== 'issues' || !payload.includes('"action":"opened"')) {
      return new Response('Ignored event', { status: 200 });
    }

    const parsed = JSON.parse(payload);
    const issue = parsed.issue;
    const repo = parsed.repository;

    // Fire-and-forget triage (respond 200 immediately to avoid webhook timeout)
    const ctx = { issue, repo, deliveryId };
    // Don't await—GitHub expects a fast 200
    ctx.waitUntil(triageIssue(ctx, env));

    return new Response('Accepted', { status: 200 });
  },
};

Key design choice: We return 200 immediately and process the triage asynchronously using ctx.waitUntil(). GitHub's webhook delivery expects a response within 10 seconds; Groq's API can take 2-5 seconds, and applying labels is another round trip. This pattern keeps webhook deliveries green.

Step 4: Integrate Groq's Llama 3 for Triage

The core intelligence: we send the issue title and body to Llama 3 with a strict system prompt that forces structured JSON output. Groq's free tier gives us access to both llama3-8b-8192 (faster, ~1-2s) and llama3-70b-8192 (more accurate, ~3-5s). We'll default to 8B for speed, but you can swap.

Add the triage function below your handler in src/index.ts:

interface TriageResult {
  labels: string[];
  assignee: string | null;
  summary: string;
}

async function triageIssue(ctx: { issue: any; repo: any }, env: Env): Promise<void> {
  const { issue, repo } = ctx;
  const issueBody = (issue.body || '').slice(0, 3000); // Truncate to avoid token bloat
  const title = issue.title;

  // Build the prompt
  const systemPrompt = `You are a GitHub issue triage assistant. Analyze the issue title and body.
Return ONLY a valid JSON object with these keys:
- labels: array of strings from this allowed set: ["bug", "enhancement", "documentation", "question", "help wanted", "good first issue", "duplicate", "invalid"]
- assignee: string or null. Choose from these contributors based on context: alice-frontend, bob-backend, carol-docs, dave-devops. If no clear match, use null.
- summary: a concise one-sentence summary of the issue.

Do not include markdown fences, explanations, or any text outside the JSON object.`;

  const userMessage = `Title: ${title}\n\nBody: ${issueBody}`;

  // Call Groq
  const groqResponse = await fetch('https://api.groq.com/openai/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${env.GROQ_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'llama3-8b-8192',
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: userMessage },
      ],
      temperature: 0.1,
      max_tokens: 300,
    }),
  });

  if (!groqResponse.ok) {
    console.error(`Groq API error: ${groqResponse.status} ${await groqResponse.text()}`);
    return;
  }

  const data: any = await groqResponse.json();
  const rawOutput = data.choices?.[0]?.message?.content || '';

  // Parse the JSON (with fallback for occasional markdown fences)
  let triage: TriageResult;
  try {
    triage = JSON.parse(rawOutput);
  } catch {
    // Llama sometimes wraps JSON in ```json
    const cleaned = rawOutput.replace(/```json\n?|```/g, '').trim();
    try {
      triage = JSON.parse(cleaned);
    } catch {
      console.error('Failed to parse Groq output:', rawOutput);
      return;
    }
  }

  await applyTriage(ctx, triage, env);
}

Why temperature 0.1: We want deterministic classification, not creative writing. Low temperature keeps label choices consistent across similar issues.

Step 5: Apply Labels and Comment via GitHub API

We need to authenticate as the GitHub App installation to mutate the issue. The @octokit/app package handles JWT generation and installation token exchange.

async function applyTriage(
  ctx: { issue: any; repo: any },
  triage: TriageResult,
  env: Env
): Promise<void> {
  const app = new App({
    appId: env.GITHUB_APP_ID,
    privateKey: env.GITHUB_PRIVATE_KEY.replace(/\\n/g, '\n'),
  });

  const octokit = await app.getInstallationOctokit(ctx.repo.owner.id);
  const { owner, name: repo } = ctx.repo;
  const issueNumber = ctx.issue.number;

  // 1. Apply labels
  if (triage.labels && triage.labels.length > 0) {
    try {
      await octokit.request('POST /repos/{owner}/{repo}/issues/{issue_number}/labels', {
        owner: owner.login,
        repo,
        issue_number: issueNumber,
        labels: triage.labels,
      });
    } catch (err: any) {
      console.error('Label application failed:', err.message);
    }
  }

  // 2. Assign if we have a match
  if (triage.assignee) {
    try {
      await octokit.request('POST /repos/{owner}/{repo}/issues/{issue_number}/assignees', {
        owner: owner.login,
        repo,
        issue_number: issueNumber,
        assignees: [triage.assignee],
      });
    } catch (err: any) {
      console.error('Assignee update failed:', err.message);
    }
  }

  // 3. Post triage comment
  const commentBody = `## 🤖 Automated Triage Summary\n\n${triage.summary}\n\n**Labels applied:** ${triage.labels?.join(', ') || 'none'}\n**Suggested assignee:** ${triage.assignee || 'None — needs human routing'}\n\n> This triage was performed by Llama 3 via Groq. Human review recommended.`;

  try {
    await octokit.request('POST /repos/{owner}/{repo}/issues/{issue_number}/comments', {
      owner: owner.login,
      repo,
      issue_number: issueNumber,
      body: commentBody,
    });
  } catch (err: any) {
    console.error('Comment creation failed:', err.message);
  }
}

Caveat: The private key from wrangler secret will have literal \n strings. The .replace(/\\n/g, '\n') normalizes them back to actual newlines that the JWT library expects.

Step 6: Deploy and Test

You're ready to ship. Run:

npx wrangler deploy

Once deployed, go to your GitHub App's Advanced tab and confirm the webhook is delivering. Create a test issue in your repo with a descriptive title like "Login button unresponsive on mobile Safari" and a body that explains the bug.

Within 5-10 seconds, you should see:

1. Labels appear on the issue
2. An assignee (if the context matched one of your contributor names)
3. A comment with the triage summary

Debugging: Use npx wrangler tail to stream live logs from your worker. Any Groq parsing failures or GitHub API errors will appear there.

Extensions and Common Pitfalls

Extensions worth building:

• Duplicate detection: Embed the issue body using a free embedding model, compare against a vector store (like Pinecone's free tier—see our Discord FAQ Bot guide for the pattern), and auto-close with a "possible duplicate" comment.
• Priority scoring: Add a priority: high/medium/low label by including urgency analysis in the Groq prompt.
• Slack notification: If the issue is high priority, fire a Slack webhook. This turns triage into an alerting system.
• Custom contributor mapping: Instead of hardcoded names, fetch the repo's collaborators from GitHub's API and feed the list into the prompt dynamically.

Common pitfalls:

• Webhook timeout: If you await the Groq call before returning 200, GitHub will mark deliveries as failed after 10 seconds. Always use the fire-and-forget pattern shown above.
• Label must exist: GitHub's API will 422 if you try to apply a label that doesn't exist in the repo. Pre-create your label set or add logic to catch 422s and skip missing labels.
• Groq rate limits: The free tier allows ~30 requests per minute. If you're testing aggressively, you'll hit 429 responses. Implement exponential backoff or add a simple in-worker rate limiter.
• Private key formatting: The most common deploy blocker is the private key not having real newlines. Double-check the .replace() call if JWT generation fails.

For a deeper dive into building LLM-powered automation pipelines—including how to handle document extraction when sources block copy-paste—check out our OCR-to-LLM pipeline guide. If you're thinking about deploying similar features in a regulated enterprise environment, our enterprise LLM deployment case study walks through the architectural considerations.

FAQ

Q: Can I use a different model on Groq? A: Absolutely. Swap llama3-8b-8192 for mixtral-8x7b-32768 or gemma2-9b-it. The prompt format is identical since Groq follows the OpenAI chat completions spec. Just ensure the model supports structured JSON output in practice.

Q: What if Llama 3 returns labels not in my allowed set? A: Add a post-processing filter that intersects the returned labels with your repo's actual label names. Fetch the label list from GitHub's API at startup and cache it in a module-level variable.

Q: Does this work for private repositories? A: Yes. The GitHub App installation has access to whatever repos you grant it. Cloudflare Workers and Groq's free tier don't care about repo visibility—just ensure your webhook secret stays secret.

Q: How much does this cost at scale? A: Zero for moderate use. Cloudflare Workers free tier: 100k requests/day. Groq free tier: ~14,400 requests/day at 10 req/min sustained. If you exceed that, Groq's paid tier is still extremely cheap compared to GPT-4.

Q: Can I make the triage comment less robotic? A: Tweak the system prompt to match your team's voice. Add something like "Write the summary in a casual, friendly tone as if you're a helpful teammate named TriageBot." Llama 3 is highly steerable.