Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 54 additions & 32 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,54 +85,73 @@ jobs:
body: |
## 🚀 swagger-coverage-cli v${{ env.NEW_VERSION }}

### ✨ Features
### ✨ New Features
- **🌐 Multi-Protocol Support**: Native support for REST (OpenAPI/Swagger), gRPC (Protocol Buffers), and GraphQL schemas
- **🔄 Mixed API Analysis**: Process multiple API specifications with different protocols in a single run
- **🎯 Protocol-Aware Matching**: Intelligent request matching tailored to each API protocol's characteristics
- **📊 Unified Reporting**: Generate consolidated HTML reports with protocol-specific insights and color coding
- **⚡ Universal CLI**: Single interface works across all supported protocols with consistent syntax

### 🎨 Enhanced Features
- **Smart Endpoint Mapping**: Intelligent endpoint matching with status code prioritization enabled by default
- **Enhanced Path Matching**: Improved handling of path parameters with different naming conventions
- **Confidence Scoring**: Match quality assessment with 0.0-1.0 confidence scores
- **Status Code Intelligence**: Prioritizes successful (2xx) codes over error codes for better coverage
- **Multi-API Support**: Process multiple Swagger/OpenAPI specifications in a single run
- **Unified Reporting**: Generate combined coverage reports with individual API metrics
- **Format Support**: YAML, JSON, and CSV file formats supported
- **Enhanced HTML Reports**: Clean, accessible reports with confidence indicators
- **Multi-API Support**: Process multiple API specifications in a single run
- **Enhanced HTML Reports**: Interactive reports with protocol identification and color coding

### 🎯 Protocol Support

#### 📋 REST/OpenAPI
- OpenAPI v2/v3 specifications (YAML/JSON)
- Smart path matching with parameter variations
- Request body and query parameter validation
- Multiple status codes per operation

#### ⚡ gRPC
- Protocol Buffer (.proto) file parsing
- Service and method extraction
- HTTP/2 path mapping (/package.service/method)
- Content-type validation (application/grpc)

### 🚀 Smart Mapping Benefits
- **Improved Coverage Accuracy**: Smart mapping significantly improves coverage detection
- **Status Code Prioritization**: Prioritizes 2xx → 4xx → 5xx for better matching
- **Path Intelligence**: Handles parameter variations like `/users/{id}` vs `/users/{userId}`
- **Confidence Assessment**: Shows match quality to help identify reliable matches
- **Default Behavior**: No flags required - smart mapping works automatically
#### 🔀 GraphQL
- GraphQL schema (.graphql/.gql) parsing
- Query, mutation, and subscription extraction
- Type system with arguments and unions
- Unified /graphql endpoint handling

### 🚀 Usage Examples
```bash
# Single protocol APIs
swagger-coverage-cli api.yaml collection.json # OpenAPI/REST
swagger-coverage-cli service.proto collection.json # gRPC
swagger-coverage-cli schema.graphql collection.json # GraphQL

# Mixed protocol APIs (Enterprise-ready)
swagger-coverage-cli "api.yaml,service.proto,schema.graphql" collection.json

# All existing options work across protocols
swagger-coverage-cli "api.yaml,service.proto" collection.json --verbose --strict-body
```

### 🔧 Compatibility
- ✅ Maintains backwards compatibility with existing workflows
- ✅ Node.js 14+ required
- ✅ Node.js 14+ required
- ✅ NPM package available globally
- ✅ Smart mapping enabled by default
- ✅ All existing CLI options work with new protocols

### 📦 Installation
```bash
npm install -g swagger-coverage-cli@${{ env.NEW_VERSION }}
```

### 🎯 Usage Examples
```bash
# Smart mapping enabled by default
swagger-coverage-cli api-spec.yaml collection.json

# With verbose output to see smart mapping statistics
swagger-coverage-cli api-spec.yaml collection.json --verbose

# Multiple APIs with smart mapping
swagger-coverage-cli api1.yaml,api2.yaml,api3.json collection.json

# Works with Newman reports too
swagger-coverage-cli api-spec.yaml newman-report.json --newman
```

### 🧪 Quality Assurance
- **130 Tests**: Comprehensive test suite covering all smart mapping scenarios
- **38 Smart Mapping Tests**: Dedicated tests for status code priority, path matching, confidence scoring
- **147 Tests**: Comprehensive test suite covering all protocols and scenarios
- **19 Test Suites**: Dedicated test coverage for each protocol and integration scenarios
- **Edge Case Coverage**: Robust handling of malformed URLs, missing data, and complex scenarios
- **Performance Tested**: Validated with large datasets (1000+ operations)
- **Performance Tested**: Validated with large datasets and mixed protocol specifications
- **Protocol Isolation**: Each protocol's parsing and matching logic is independently tested

---

Expand Down Expand Up @@ -165,12 +184,15 @@ jobs:
echo "- **GitHub Release:** [v${{ env.NEW_VERSION }}](https://github.com/${{ github.repository }}/releases/tag/v${{ env.NEW_VERSION }})" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### 🎯 Key Features" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Multi-protocol support (REST, gRPC, GraphQL)" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Protocol-aware matching logic" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Mixed API analysis in single run" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Smart endpoint mapping (enabled by default)" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Status code prioritization" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Enhanced path matching" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Confidence scoring" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Multi-API support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Newman report support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Enhanced HTML reports" >> $GITHUB_STEP_SUMMARY
echo "- ✅ YAML, JSON, CSV support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Enhanced HTML reports with protocol identification" >> $GITHUB_STEP_SUMMARY
echo "- ✅ YAML, JSON, CSV, .proto, .graphql support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Backwards compatibility" >> $GITHUB_STEP_SUMMARY
42 changes: 35 additions & 7 deletions auto-detect-newman.html
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ <h1>Swagger Coverage Report</h1>
&#128262; <!-- flashlight icon -->
</button>
<div class="meta-info">
<p><strong>Timestamp:</strong> 9/18/2025, 12:15:43 PM</p>
<p><strong>Timestamp:</strong> 9/18/2025, 2:32:58 PM</p>
<p><strong>API Spec:</strong> Test API</p>

<p><strong>Postman Collection:</strong> Test Newman Collection</p>
Expand Down Expand Up @@ -436,6 +436,7 @@ <h1>Swagger Coverage Report</h1>
<thead>
<tr>

<th onclick="sortTableBy('protocol')">Protocol</th>
<th onclick="sortTableBy('method')">Method</th>
<th onclick="sortTableBy('path')">Path</th>
<th onclick="sortTableBy('name')">Name</th>
Expand All @@ -461,7 +462,7 @@ <h1>Swagger Coverage Report</h1>
hljs.highlightAll();

// coverageData from server
let coverageData = [{"method":"GET","path":"/users","name":"getUsers","statusCode":"200","tags":[],"expectedStatusCodes":["200"],"apiName":"Test API","sourceFile":"test-api.yaml","unmatched":false,"matchedRequests":[{"name":"Get Users","rawUrl":"https://api.example.com/users","method":"GET","testedStatusCodes":["200"],"testScripts":"// Status code is 200","confidence":0.8999999999999999}],"isPrimaryMatch":true,"matchConfidence":0.8999999999999999},{"method":"POST","path":"/users","name":"createUser","statusCode":"201","tags":[],"expectedStatusCodes":["201","400"],"apiName":"Test API","sourceFile":"test-api.yaml","unmatched":false,"matchedRequests":[{"name":"Create User","rawUrl":"https://api.example.com/users","method":"POST","testedStatusCodes":["201"],"testScripts":"// Status code is 201","confidence":0.8999999999999999}],"isPrimaryMatch":true,"matchConfidence":0.8999999999999999},{"method":"POST","path":"/users","name":"createUser","statusCode":"400","tags":[],"expectedStatusCodes":["201","400"],"apiName":"Test API","sourceFile":"test-api.yaml","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0},{"method":"GET","path":"/users/{id}","name":"getUserById","statusCode":"200","tags":[],"expectedStatusCodes":["200","404"],"apiName":"Test API","sourceFile":"test-api.yaml","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0},{"method":"GET","path":"/users/{id}","name":"getUserById","statusCode":"404","tags":[],"expectedStatusCodes":["200","404"],"apiName":"Test API","sourceFile":"test-api.yaml","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0}];
let coverageData = [{"method":"GET","path":"/users","name":"getUsers","statusCode":"200","tags":[],"expectedStatusCodes":["200"],"apiName":"Test API","sourceFile":"test-api.yaml","protocol":"rest","unmatched":false,"matchedRequests":[{"name":"Get Users","rawUrl":"https://api.example.com/users","method":"GET","testedStatusCodes":["200"],"testScripts":"// Status code is 200","confidence":0.8999999999999999}],"isPrimaryMatch":true,"matchConfidence":0.8999999999999999},{"method":"POST","path":"/users","name":"createUser","statusCode":"201","tags":[],"expectedStatusCodes":["201","400"],"apiName":"Test API","sourceFile":"test-api.yaml","protocol":"rest","unmatched":false,"matchedRequests":[{"name":"Create User","rawUrl":"https://api.example.com/users","method":"POST","testedStatusCodes":["201"],"testScripts":"// Status code is 201","confidence":0.8999999999999999}],"isPrimaryMatch":true,"matchConfidence":0.8999999999999999},{"method":"POST","path":"/users","name":"createUser","statusCode":"400","tags":[],"expectedStatusCodes":["201","400"],"apiName":"Test API","sourceFile":"test-api.yaml","protocol":"rest","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0},{"method":"GET","path":"/users/{id}","name":"getUserById","statusCode":"200","tags":[],"expectedStatusCodes":["200","404"],"apiName":"Test API","sourceFile":"test-api.yaml","protocol":"rest","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0},{"method":"GET","path":"/users/{id}","name":"getUserById","statusCode":"404","tags":[],"expectedStatusCodes":["200","404"],"apiName":"Test API","sourceFile":"test-api.yaml","protocol":"rest","unmatched":true,"matchedRequests":[],"isPrimaryMatch":false,"matchConfidence":0}];
let apiCount = 1;

// Merge duplicates for display only
Expand Down Expand Up @@ -741,6 +742,23 @@ <h1>Swagger Coverage Report</h1>
tr.appendChild(tdApi);
}

// Add protocol column
const tdProtocol = document.createElement('td');
tdProtocol.className = "spec-cell protocol-cell";
const protocol = item.protocol || 'rest';
tdProtocol.textContent = protocol.toUpperCase();
// Add styling based on protocol
if (protocol === 'grpc') {
tdProtocol.style.color = '#4caf50';
tdProtocol.style.fontWeight = 'bold';
} else if (protocol === 'graphql') {
tdProtocol.style.color = '#e91e63';
tdProtocol.style.fontWeight = 'bold';
} else {
tdProtocol.style.color = '#2196f3';
}
tr.appendChild(tdProtocol);

const tdMethod = document.createElement('td');
tdMethod.className = "spec-cell";
tdMethod.textContent = (item.method || "").toUpperCase();
Expand Down Expand Up @@ -784,7 +802,7 @@ <h1>Swagger Coverage Report</h1>
subTr.className = "matched-requests-row";

const subTd = document.createElement('td');
subTd.colSpan = apiCount > 1 ? 5 : 4;
subTd.colSpan = apiCount > 1 ? 6 : 5;

const pmTable = document.createElement('table');
pmTable.className = "postman-table";
Expand Down Expand Up @@ -976,10 +994,20 @@ <h1>Swagger Coverage Report</h1>
const rows = document.querySelectorAll('#specTable tbody tr.spec-row');

rows.forEach(row => {
const method = row.querySelector('td:nth-child(1)').textContent;
const path = row.querySelector('td:nth-child(2)').textContent;
const name = row.querySelector('td:nth-child(3)').textContent;
const text = method + ' ' + path + ' ' + name.toLowerCase();
// Column indices adjust based on whether we have API column and protocol column
let colIndex = 1;
if (false) colIndex++; // API column
colIndex++; // Protocol column

const method = row.querySelector(`td:nth-child(${colIndex})`).textContent;
const path = row.querySelector(`td:nth-child(${colIndex + 1})`).textContent;
const name = row.querySelector(`td:nth-child(${colIndex + 2})`).textContent;

// Include protocol in search
const protocolIndex = false ? 2 : 1;
const protocol = row.querySelector(`td:nth-child(${protocolIndex})`).textContent;

const text = method + ' ' + path + ' ' + name.toLowerCase() + ' ' + protocol.toLowerCase();

const matchesSearch = searchText === '' || text.includes(searchText);
const matchesFilter = (filterMode === 'all') ||
Expand Down
68 changes: 49 additions & 19 deletions cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,64 +11,94 @@ const { loadNewmanReport, extractRequestsFromNewman } = require("./lib/newman");
const { matchOperationsDetailed } = require("./lib/match");
const { generateHtmlReport } = require("./lib/report");
const { loadExcelSpec } = require("./lib/excel");
const { loadAndParseProto, extractOperationsFromProto, isProtoFile } = require("./lib/grpc");
const { loadAndParseGraphQL, extractOperationsFromGraphQL, isGraphQLFile } = require("./lib/graphql");

const program = new Command();

program
.name("swagger-coverage-cli")
.description(
"CLI tool for comparing OpenAPI/Swagger specifications with a Postman collection or Newman run report, producing an enhanced HTML report"
"CLI tool for comparing API specifications (OpenAPI/Swagger, gRPC Protocol Buffers, GraphQL) with Postman collections or Newman run reports, producing an enhanced HTML report"
)
.version("4.0.0")
.argument("<swaggerFiles>", "Path(s) to the Swagger/OpenAPI file(s) (JSON or YAML). Use comma-separated values for multiple files.")
.version("7.0.0")
.argument("<apiFiles>", "Path(s) to API specification file(s): OpenAPI/Swagger (JSON/YAML), gRPC (.proto), GraphQL (.graphql/.gql), or CSV. Use comma-separated values for multiple files.")
.argument("<postmanCollectionOrNewmanReport>", "Path to the Postman collection (JSON) or Newman run report (JSON).")
.option("-v, --verbose", "Show verbose debug info")
.option("--strict-query", "Enable strict validation of query parameters")
.option("--strict-body", "Enable strict validation of requestBody (JSON)")
.option("--output <file>", "HTML report output file", "coverage-report.html")
.option("--newman", "Treat input file as Newman run report instead of Postman collection")
.action(async (swaggerFiles, postmanFile, options) => {
.action(async (apiFiles, postmanFile, options) => {
try {
const { verbose, strictQuery, strictBody, output, newman } = options;

// Parse comma-separated swagger files
const files = swaggerFiles.includes(',') ?
swaggerFiles.split(',').map(f => f.trim()) :
[swaggerFiles];
// Parse comma-separated API files
const files = apiFiles.includes(',') ?
apiFiles.split(',').map(f => f.trim()) :
[apiFiles];

let allSpecOperations = [];
let allSpecNames = [];
const excelExtensions = [".xlsx", ".xls", ".csv"];

// Process each swagger file
for (const swaggerFile of files) {
const ext = path.extname(swaggerFile).toLowerCase();
// Process each API specification file
for (const apiFile of files) {
const ext = path.extname(apiFile).toLowerCase();
let specOperations;
let specName;
let protocol;

if (excelExtensions.includes(ext)) {
// Parse Excel
specOperations = loadExcelSpec(swaggerFile);
specName = path.basename(swaggerFile);
// Parse Excel/CSV
specOperations = loadExcelSpec(apiFile);
specName = path.basename(apiFile);
protocol = 'rest';
} else if (isProtoFile(apiFile)) {
// Parse gRPC Protocol Buffer
const protoRoot = await loadAndParseProto(apiFile);
specName = path.basename(apiFile, '.proto');
specOperations = extractOperationsFromProto(protoRoot, verbose);
protocol = 'grpc';
if (verbose) {
console.log(
"gRPC specification loaded successfully:",
specName
);
}
} else if (isGraphQLFile(apiFile)) {
// Parse GraphQL schema
const graphqlData = loadAndParseGraphQL(apiFile);
specName = path.basename(apiFile);
specOperations = extractOperationsFromGraphQL(graphqlData, verbose);
protocol = 'graphql';
if (verbose) {
console.log(
"GraphQL specification loaded successfully:",
specName
);
}
} else {
// Original Swagger flow
const spec = await loadAndParseSpec(swaggerFile);
// Original OpenAPI/Swagger flow
const spec = await loadAndParseSpec(apiFile);
specName = spec.info.title;
protocol = 'rest';
if (verbose) {
console.log(
"Specification loaded successfully:",
"OpenAPI specification loaded successfully:",
specName,
spec.info.version
);
}
specOperations = extractOperationsFromSpec(spec, verbose);
}

// Add API name to each operation for identification
// Add API name and protocol to each operation for identification
const operationsWithSource = specOperations.map(op => ({
...op,
apiName: specName,
sourceFile: path.basename(swaggerFile)
sourceFile: path.basename(apiFile),
protocol: protocol
}));

allSpecOperations = allSpecOperations.concat(operationsWithSource);
Expand Down
Loading