-
Notifications
You must be signed in to change notification settings - Fork 0
FAQ
Quick answers to common questions about Smart Issue Analyzer.
A GitHub Actions workflow that uses AI (LLM) to automatically analyze, classify, and triage new issues in your repository.
Yes! Uses GitHub Models API which is free for GitHub users (subject to rate limits).
- Analysis works with any language (English recommended)
- Spanish translation is built-in
- Additional languages can be added by modifying the workflow
- Duplicate detection: ~94% accuracy
- Label classification: ~89% precision
- Priority assignment: ~91% accuracy
- Size estimation: ±1 category variance
No, works with free GitHub accounts. Requires access to GitHub Models (currently in beta).
- Go to Settings → Developer settings → Personal access tokens → Fine-grained tokens
- Generate new token
- Permissions: Account permissions → Models (Read-only)
- Copy token and add as repository secret named
MODELS_PAT
Yes, works in both public and private repos. Ensure GitHub Actions is enabled.
Yes! Edit the workflow file and modify the label arrays in the classification prompts.
Comment out the corresponding LLM call in the workflow. For example, to disable translation:
// const translation = await callLLM(...); // Disabled
const translation = '{"title": "", "description": ""}'; // Fallbackgpt-4o-mini via GitHub Models API. Fast, cost-effective, and good for classification.
Yes, modify the model parameter:
model: 'gpt-4o' // More powerful but slower
model: 'gpt-3.5-turbo' // Faster but less accurate4 parallel calls:
- Duplicate detection (250 tokens)
- Classification (200 tokens)
- Context analysis (300 tokens)
- Spanish translation (400 tokens)
Total: ~1150 tokens per issue
Depends on GitHub Models tier. Free tier: ~60 requests/minute. Each issue uses 4 requests.
Typically 3-5 seconds from issue creation to analysis completion.
No, requires internet access to call GitHub Models API.
Triggered automatically when a new issue is opened (not edited or commented).
Not by default. To enable manual triggers, add:
on:
issues:
types: [opened]
workflow_dispatch: # Adds manual triggerNo, only runs on new issues. To analyze existing issues, create a separate workflow with workflow_dispatch trigger.
The issue is left unchanged. Check Actions logs for error details. Workflow has extensive error handling to prevent partial updates.
Yes, change trigger:
on:
issues:
types: [opened, edited]But be cautious - will re-analyze every edit.
LLM compares new issue title and description against all existing issues to find semantic matches. Returns issue number if duplicate found.
- Reopen the issue
- Add comment explaining why it's not a duplicate
- Apply
not-duplicatelabel to prevent future false positives
No, only searches issues within the same repository.
- P0 - Critical: Production down, security breach, data loss
- P1 - High: Major feature broken
- P2 - Medium: Minor bugs, enhancements
- P3 - Low: Nice-to-haves
LLM analyzes issue complexity and scope:
- XS: 1-2 hours
- S: 2-4 hours
- M: 1-2 days
- L: 3-5 days
- XL: 1+ week
Negative sentiment detection: urgency, frustration, anger in issue description.
It suggests assignees based on issue type. You can enhance it to auto-assign:
if (assignmentSuggestion) {
await github.rest.issues.addAssignees({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
assignees: [assignmentSuggestion.replace('@', '')]
});
}Yes, if isDuplicate: true is returned, it posts a comment and closes the issue.
- Check Actions are enabled (Settings → Actions)
- Verify workflow file is on default branch
- Check YAML syntax is valid
- Ensure file path is exactly
.github/workflows/smart-issue-analyzer.yml
- Verify
MODELS_PATsecret exists - Check token hasn't expired
- Ensure token has Models permission
- Try regenerating token
- Check workflow logs for API errors
- Verify LLM response format in logs
- Ensure adequate token limits
- Check for rate limiting
- Ensure labels exist in repository (create them first)
- Check workflow has
issues: writepermission - Verify label names match exactly (case-sensitive)
See Troubleshooting for detailed solutions.
Yes, add another LLM call:
const frenchTranslation = await callLLM(
'You are a translator.',
`Translate to French:\nTitle: ${title}\nBody: ${body}`,
400
);Yes, edit the commentBody variable to customize markdown output.
Yes, add webhook calls after analysis:
await fetch('https://hooks.slack.com/...', {
method: 'POST',
body: JSON.stringify({ text: `New P0 issue: ${title}` })
});Yes, add API calls to store results:
await fetch('https://your-api.com/issues', {
method: 'POST',
body: JSON.stringify({ issue: context.issue.number, analysis: classifyResult })
});Limited by GitHub Actions concurrency (typically 20 parallel jobs) and API rate limits. Approximately 300-600 issues/hour.
Yes, fetching all issues for context can be slow (1000+ issues). Solution: Limit context to recent 100 issues.
Each gets its own workflow run. All run in parallel (up to concurrency limit).
Yes, use workflow conditions:
jobs:
analyze:
runs-on: ubuntu-latest
if: contains(github.event.issue.title, 'URGENT')Yes, issue content is sent to GitHub Models API (Microsoft Azure). Review GitHub's privacy policy.
Yes, but requires significant modification to replace GitHub Models API with local endpoint.
No, GitHub Actions automatically masks secrets in logs.
No, LLM only analyzes. Workflow code controls what actions are taken. Review workflow file to understand exact behavior.
- Copilot: Code completion assistant
- This: Automated issue triage workflow
- Can be used together!
Recommended. This automates initial classification, but human review ensures accuracy.
No, complementary. Templates collect structured data, this workflow analyzes it.
Potentially! Similar analysis could work for PR descriptions.
Not currently, but could be extended to generate code snippets for simple bugs.
Not planned, but you could build one using GitHub GraphQL API to query issue labels.
Yes! Fork the repository, make changes, and submit a pull request.
Create an issue in this repository with:
- Workflow run link
- Error message
- Expected vs actual behavior
Yes! MIT licensed (or check repository license). Copy workflow file and customize.
- Getting Started - Setup guide
- Best Practices - Optimization tips
- Examples - Real-world use cases
- Troubleshooting - Fix common issues