Skip to content

Customization Guide

Jump to bottom
BatraXPankaj edited this page Nov 13, 2025 · 1 revision

Customization Guide

Tailor Smart Issue Analyzer to your specific workflow needs.

Basic Customization

1. Change Trigger Events

Current: Only runs on new issues

Add issue editing:

on:
  issues:
    types: [opened, edited]

Add manual trigger:

on:
  issues:
    types: [opened]
  workflow_dispatch:
    inputs:
      issue_number:
        description: 'Issue number to analyze'
        required: true

Run on comments:

on:
  issue_comment:
    types: [created]

2. Customize Labels

Add custom topic labels:

// In classification system prompt:
Available labels:
- bug, enhancement, documentation
- YOUR_CUSTOM_LABEL_1
- YOUR_CUSTOM_LABEL_2

Modify priority levels:

// Change from P0-P3 to Critical/High/Medium/Low:
"priority": "critical"  // instead of "P0"

// Update label application:
labels.push(`priority-${classifyResult.priority.toLowerCase()}`);

Change size scale:

// T-shirt sizes to story points:
"size": "3"  // instead of "M"

// Or use hours:
"estimatedHours": 8

3. Modify LLM Parameters

Increase response length:

// For more detailed analysis:
callLLM(systemPrompt, userPrompt, 500)  // was 300

Adjust creativity:

// More consistent (less creative):
temperature: 0.1

// More creative (less consistent):
temperature: 0.7

Change model:

model: 'gpt-4o'           // More powerful
model: 'gpt-3.5-turbo'    // Faster
model: 'gpt-4o-mini'      // Current (balanced)

Advanced Customization

4. Add New Analysis Types

Example: Code Quality Analysis

// Add 5th parallel call:
const [dup, classify, contextAnalysis, translation, codeQuality] = await Promise.all([
  // ... existing calls ...
  callLLM(
    'You are a code quality analyzer.',
    `Analyze this issue for code quality concerns:
    Title: ${issueTitle}
    Body: ${issueBody}
    
    Return JSON:
    {
      "hasCodeSnippet": true/false,
      "qualityIssues": ["naming", "performance", "security"],
      "complexity": "low/medium/high"
    }`,
    200
  )
]);

// Parse and use results:
const codeQualityResult = JSON.parse(codeQuality.match(/\{[\s\S]*\}/)?.[0] || '{}');
if (codeQualityResult.hasCodeSnippet) {
  labels.push('has-code');
}

5. Multi-Language Translation

Add multiple languages:

const translations = await Promise.all([
  // Spanish
  callLLM('Translator', `Translate to Spanish:\n${issueBody}`, 400),
  
  // French
  callLLM('Translator', `Translate to French:\n${issueBody}`, 400),
  
  // German
  callLLM('Translator', `Translate to German:\n${issueBody}`, 400)
]);

const [spanish, french, german] = translations.map(t => 
  JSON.parse(t.match(/\{[\s\S]*\}/)?.[0] || '{}')
);

// Add to comment:
commentBody += `
### 🌐 Translations
**Spanish:** ${spanish.title}
**French:** ${french.title}
**German:** ${german.title}
`;

6. Custom Team Assignment

Map labels to teams:

const teamMapping = {
  'bug': ['backend-team', 'qa-team'],
  'ui/ux': ['design-team', 'frontend-team'],
  'security': ['security-team'],
  'performance': ['devops-team', 'backend-team'],
  'documentation': ['tech-writers']
};

// Get teams for this issue:
const suggestedTeams = classifyResult.labels
  .flatMap(label => teamMapping[label] || [])
  .filter((team, index, self) => self.indexOf(team) === index); // Unique

// Auto-assign:
if (suggestedTeams.length > 0) {
  await github.rest.issues.addAssignees({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number,
    assignees: suggestedTeams
  });
}

7. Add External Integrations

Slack Notifications

// After high-priority issue detection:
if (classifyResult.priority === 'P0' || labels.includes('security')) {
  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `🚨 Critical Issue Created`,
      blocks: [{
        type: 'section',
        text: {
          type: 'mrkdwn',
          text: `*${issueTitle}*\nPriority: ${classifyResult.priority}\n<${issueUrl}|View Issue>`
        }
      }]
    })
  });
}

Jira Integration

// Create Jira ticket for high-priority issues:
if (classifyResult.priority === 'P0') {
  await fetch('https://your-domain.atlassian.net/rest/api/3/issue', {
    method: 'POST',
    headers: {
      'Authorization': `Basic ${Buffer.from(
        `${process.env.JIRA_EMAIL}:${process.env.JIRA_API_TOKEN}`
      ).toString('base64')}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      fields: {
        project: { key: 'PROJ' },
        summary: issueTitle,
        description: issueBody,
        issuetype: { name: 'Bug' },
        priority: { name: 'Highest' }
      }
    })
  });
}

Email Notifications

// Using SendGrid:
if (contextAnalysis.sentiment === 'negative') {
  await fetch('https://api.sendgrid.com/v3/mail/send', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.SENDGRID_API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      personalizations: [{
        to: [{ email: 'support@example.com' }],
        subject: `Frustrated user: ${issueTitle}`
      }],
      from: { email: 'noreply@example.com' },
      content: [{
        type: 'text/plain',
        value: `A user submitted an issue with negative sentiment:\n\n${issueBody}`
      }]
    })
  });
}

8. Custom Duplicate Handling

Instead of auto-closing:

// Just flag as potential duplicate:
if (dupResult.isDuplicate) {
  await github.rest.issues.addLabels({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number,
    labels: ['potential-duplicate']
  });
  
  await github.rest.issues.createComment({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number,
    body: `⚠️ This may be a duplicate of #${dupResult.duplicateOf}. Please review before closing.`
  });
  
  // Don't auto-close
}

Merge duplicates:

if (dupResult.isDuplicate) {
  // Copy comments from new issue to original
  const comments = await github.rest.issues.listComments({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number
  });
  
  for (const comment of comments.data) {
    await github.rest.issues.createComment({
      owner: context.repo.owner,
      repo: context.repo.repo,
      issue_number: dupResult.duplicateOf,
      body: `> From duplicate #${context.issue.number}:\n\n${comment.body}`
    });
  }
  
  // Then close
  await github.rest.issues.update({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number,
    state: 'closed'
  });
}

9. Enhanced Missing Info Detection

Auto-create checklist:

if (contextAnalysis.missingInfo?.length > 0) {
  const checklist = contextAnalysis.missingInfo
    .map(item => `- [ ] ${item}`)
    .join('\n');
    
  await github.rest.issues.createComment({
    owner: context.repo.owner,
    repo: context.repo.repo,
    issue_number: context.issue.number,
    body: `### ⚠️ Please provide the following information:\n\n${checklist}\n\n*Check items as you add them to the issue description.*`
  });
  
  // Apply "needs-more-info" label
  labels.push('needs-more-info');
}

10. Project Board Automation

Add to project board column:

// Get project board:
const projects = await github.rest.projects.listForRepo({
  owner: context.repo.owner,
  repo: context.repo.repo
});

const projectId = projects.data[0]?.id; // First project

if (projectId) {
  // Get columns:
  const columns = await github.rest.projects.listColumns({ project_id: projectId });
  
  // Determine column based on priority:
  const columnMap = {
    'P0': 'Urgent',
    'P1': 'High Priority',
    'P2': 'Backlog',
    'P3': 'Nice to Have'
  };
  
  const targetColumn = columns.data.find(c => 
    c.name === columnMap[classifyResult.priority]
  );
  
  if (targetColumn) {
    // Create card:
    await github.rest.projects.createCard({
      column_id: targetColumn.id,
      content_id: context.issue.id,
      content_type: 'Issue'
    });
  }
}

11. Time-Based Rules

Escalate old issues:

// Check issue age:
const createdAt = new Date(context.payload.issue.created_at);
const now = new Date();
const ageHours = (now - createdAt) / (1000 * 60 * 60);

if (ageHours > 24 && classifyResult.priority === 'P1') {
  // Escalate to P0 if not addressed in 24h:
  await github.rest.issues.addLabels({
    labels: ['priority-p0', 'escalated']
  });
  
  await github.rest.issues.createComment({
    body: '⚠️ Escalated to P0 due to age (24+ hours without response)'
  });
}

12. Custom Metrics & Analytics

Track analysis metrics:

// At end of workflow:
const metrics = {
  timestamp: new Date().toISOString(),
  issue_number: context.issue.number,
  analysis_time: duration,
  labels_applied: labels.length,
  priority: classifyResult.priority,
  size: classifyResult.size,
  is_duplicate: dupResult.isDuplicate,
  sentiment: contextAnalysis.sentiment
};

// Send to analytics service:
await fetch('https://your-analytics.com/api/metrics', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(metrics)
});

// Or store in GitHub as artifact:
const fs = require('fs');
fs.writeFileSync('metrics.json', JSON.stringify(metrics));

13. A/B Testing Prompts

Test different prompts:

// Randomly select prompt version:
const promptVersion = Math.random() < 0.5 ? 'v1' : 'v2';

const systemPromptV1 = 'You are a classifier. Return JSON.';
const systemPromptV2 = 'You are an expert GitHub issue triager with 10 years experience. Analyze carefully and return structured JSON.';

const systemPrompt = promptVersion === 'v1' ? systemPromptV1 : systemPromptV2;

// Log which version was used:
console.log(`Using prompt version: ${promptVersion}`);

// Track results to see which performs better

14. Conditional Analysis

Skip analysis for certain users:

// Don't analyze issues from bots:
if (context.payload.issue.user.type === 'Bot') {
  console.log('Skipping bot-created issue');
  return;
}

// Skip issues with specific labels:
const skipLabels = ['wontfix', 'invalid', 'spam'];
if (context.payload.issue.labels.some(l => skipLabels.includes(l.name))) {
  console.log('Skipping issue with exclusion label');
  return;
}

Configuration File Approach

Instead of hardcoding, use config file:

.github/issue-analyzer-config.json:

{
  "models": {
    "classification": "gpt-4o-mini",
    "translation": "gpt-4o-mini",
    "temperature": 0.3
  },
  "labels": {
    "topics": ["bug", "enhancement", "documentation"],
    "priorities": ["P0", "P1", "P2", "P3"],
    "sizes": ["XS", "S", "M", "L", "XL"]
  },
  "teams": {
    "bug": ["backend-team"],
    "ui/ux": ["frontend-team", "design-team"]
  },
  "autoClose": {
    "duplicates": true,
    "spam": true
  },
  "notifications": {
    "slack": {
      "enabled": true,
      "priorityThreshold": "P1"
    }
  }
}

Load in workflow:

const fs = require('fs');
const config = JSON.parse(
  fs.readFileSync('.github/issue-analyzer-config.json', 'utf8')
);

// Use config values:
const model = config.models.classification;
const temperature = config.models.temperature;
const validLabels = config.labels.topics;

Testing Custom Changes

1. Create Test Workflow

name: Test Custom Analysis
on:
  workflow_dispatch:
    inputs:
      test_issue:
        description: 'Test issue number'
        required: true

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/github-script@v6
        with:
          script: |
            // Your custom code here
            console.log('Testing with issue:', '${{ github.event.inputs.test_issue }}');

2. Use Separate Test Repo

  • Fork repository
  • Test changes there first
  • Verify results before deploying to production

3. Add Dry-Run Mode

const DRY_RUN = process.env.DRY_RUN === 'true';

if (DRY_RUN) {
  console.log('DRY RUN: Would apply labels:', labels);
  console.log('DRY RUN: Would post comment:', commentBody);
} else {
  await github.rest.issues.addLabels({...});
  await github.rest.issues.createComment({...});
}

Next Steps

Clone this wiki locally