Skip to content

webhooks.md

Latest commit

 

History

History
333 lines (229 loc) · 8.99 KB

webhooks.md

File metadata and controls

333 lines (229 loc) · 8.99 KB

Webhooks Guide

Codebot can automatically respond to PR review comments by making code changes or answering questions.

Overview

Codebot can receive PR review comments via two methods:

  • Webhooks (default): GitHub sends events directly to the server
  • Polling: Server periodically checks PRs for new comments (for GitHub Enterprise environments where webhooks cannot reach the server)

Both methods:

  • Receive PR review comments
  • Classify them as queries or change requests
  • Respond with answers or code changes
  • Update PR descriptions after changes

Note: Webhooks and polling are mutually exclusive - use one or the other, not both.

Setup

Option 1: Webhook Mode (Default)

1. Start Webhook Server

export GITHUB_APP_ID="123456"
export GITHUB_APP_PRIVATE_KEY_PATH="./codebot-private-key.pem"
export GITHUB_APP_INSTALLATION_ID="789012"
export GITHUB_WEBHOOK_SECRET="your_webhook_secret"

codebot serve --port 5000

2. Expose Server (Development)

For local testing, use a tool like ngrok:

ngrok http 5000

This gives you a public URL like https://abc123.ngrok.io.

3. Configure GitHub Webhook

  1. Go to your repository Settings → Webhooks
  2. Click Add webhook
  3. Configure:
    • Payload URL: https://your-server.com/webhook
    • Content type: application/json
    • Secret: Same as GITHUB_WEBHOOK_SECRET
    • Events: Select individual events:
      • ✅ Issue comments
      • ✅ Pull request reviews
      • ✅ Pull request review comments
  4. Click Add webhook

4. Verify Setup

Check the health endpoint:

curl http://localhost:5000/health

Response:

{
  "status": "healthy",
  "review_queue_size": 0
}

Option 2: Polling Mode (For GitHub Enterprise)

When webhooks cannot reach your server (e.g., GitHub Enterprise behind firewall), use polling mode:

1. Start Server with Polling Enabled

export GITHUB_APP_ID="123456"
export GITHUB_APP_PRIVATE_KEY_PATH="./codebot-private-key.pem"
export GITHUB_APP_INSTALLATION_ID="789012"
export CODEBOT_POLL_INTERVAL="300"  # Optional: poll interval in seconds (default: 300)

codebot serve --port 5000 --enable-polling

Note: Do NOT set GITHUB_WEBHOOK_SECRET when using polling mode.

2. Configure Poll Interval (Optional)

Set poll interval via environment variable or CLI option:

# Via environment variable
export CODEBOT_POLL_INTERVAL="600"  # Poll every 10 minutes

# Via CLI option
codebot serve --enable-polling --poll-interval 600

Default: 300 seconds (5 minutes)

How Polling Works

  • Polls PRs for tasks with pending_review status
  • Only fetches comments created since last poll (uses GitHub API since parameter)
  • Tracks processed comments to avoid duplicates
  • Updates task status when PR is merged (completed) or closed (rejected)

Important: Polling only checks PRs for tasks that are in pending_review status. Tasks transition to this status automatically after PR creation.

Task Status Lifecycle

Tasks progress through these statuses:

  • pending → Task queued, waiting to start
  • running → Task is being executed
  • pending_review → PR created, waiting for review/approval
  • completed → PR merged successfully
  • rejected → PR closed without merge
  • failed → Task execution failed

How It Works

Comment Types

Codebot handles three types of PR comments:

  1. Inline code comments (pull_request_review_comment)

    • Comments on specific lines of code
    • Includes file, line, and diff context

  2. Review summaries (pull_request_review)

    • Overall review with approval/changes requested
    • May include general feedback

  3. General PR comments (issue_comment)

    • Comments on the PR conversation
    • Not tied to specific code

Processing Flow

When a reviewer leaves a comment:

  1. Webhook Receives Event - GitHub sends webhook payload
  2. Signature Verification - Validates request authenticity
  3. Queue Comment - Adds to FIFO queue for processing
  4. Classify Comment - Uses Claude AI to determine intent:
    • Query: Asking a question → Answer without code changes
    • Change Request: Requesting changes → Make changes and commit
    • Ambiguous: Unclear intent → Ask for clarification
  5. Process with Claude - Provides full context:
    • PR title and description
    • Files changed
    • Comment location (file, line, diff)
    • Previous comment thread
  6. Respond - Posts reply to GitHub
  7. Update PR (if changes made) - Refreshes PR description

Comment Classification

Codebot uses Claude AI to intelligently classify comments:

Query/Question Examples:

  • "Why did you choose this approach?"
  • "What does this parameter do?"
  • "How does this function work?"

Change Request Examples:

  • "Please add error handling here"
  • "This should use async/await"
  • "Remove this console.log"

Ambiguous Examples:

  • "This could be better"
  • "Not sure about this"
  • "Hmm..."

For ambiguous comments, codebot asks for clarification:

🤔 Could you clarify what you'd like me to do?

Please clarify if you'd like me to:

  • Answer a question about the code
  • Make specific changes to the code

Context Provided to Claude

When processing comments, Claude receives:

  • PR Information

    • Title and description
    • List of files changed

  • Comment Location (for inline comments)

    • File path
    • Line number
    • Diff hunk (surrounding code)

  • Comment Thread

    • Previous comments in the thread
    • Full conversation history

  • Repository Context

    • CLAUDE.md file (if present)
    • Project conventions

This rich context allows Claude to provide accurate, contextual responses.

Workspace Reuse

Codebot reuses workspaces for efficiency:

  1. First comment on a PR → Clone repository
  2. Subsequent comments → Reuse existing workspace
  3. Pull latest changes before processing
  4. Workspace identified by UUID in branch name

PR Description Updates

After making changes, codebot automatically updates the PR description:

  • Uses Claude to analyze the full diff
  • Generates a unified, cohesive description
  • Focuses on what was built, not individual commits
  • Simplifies file list (only shows names, omits if >5 files)

Example updated PR:

## 📋 Task Description

Add dark mode support to the application

## 🔨 Changes Made

Implemented a comprehensive dark mode feature with automatic theme detection...

## 📁 Files Changed

- `src/theme.ts`
- `src/components/ThemeToggle.tsx`
- `src/styles/dark.css`

---
*This PR was automatically generated by codebot 🤖*

Recursion Prevention

Codebot ignores its own comments to avoid infinite loops:

  • Checks for signature: Reply generated by codebot 🤖
  • Filters out before processing
  • Works with both PATs and GitHub Apps

FIFO Queue Processing

Multiple review comments are processed sequentially:

  • First In, First Out order
  • Prevents conflicts between concurrent changes
  • Ensures predictable processing
  • Maintains stable workspace state

Required GitHub Permissions

The GitHub token needs these permissions:

Classic Token:

  • ✅ repo (full repository access)

Fine-Grained Token:

  • ✅ Pull requests: Read and Write
  • ✅ Contents: Read and Write
  • ✅ Metadata: Read (automatic)

See Configuration Guide for details.

Development Mode

Enable auto-reload for development:

codebot serve --port 5000 --debug

Features:

  • Auto-restart on code changes
  • Detailed error pages
  • Verbose logging

Note: Never use --debug in production.

Troubleshooting

For webhook issues, signature verification problems, and comment processing errors, see the Troubleshooting Guide.

Best Practices

  1. Use webhook secrets - Always set GITHUB_WEBHOOK_SECRET
  2. Monitor queue size - Check /health endpoint regularly
  3. Test locally first - Use ngrok for development
  4. Review logs - Monitor server output for issues
  5. Set up alerts - Monitor for failed webhook deliveries

Real-World Examples

See Codebot in action with these example pull requests:

  • PR #3: Confetti Feature - Demonstrates how Codebot handles review comments, makes code changes, and updates PR descriptions based on feedback
  • PR #4: Calculator Enhancement - Shows Codebot responding to questions about code implementation and providing detailed explanations

These PRs showcase:

  • Initial task execution and PR creation
  • Responding to review comments with code changes
  • Answering questions about implementation details
  • Automatic PR description updates after changes
  • Comment thread management

Next Steps

← Back to Documentation