feat: Include folder-level tests in Postman collection coverage calculation

[Copilot]

Overview

This PR implements support for folder-level tests in Postman collections. Previously, tests defined at the folder level were ignored during coverage calculation. Now, folder-level tests are automatically extracted and applied to all requests within that folder and its subfolders, ensuring more accurate coverage metrics.

Problem

When organizing Postman collections into folders with common test assertions at the folder level, these tests were not being included in the coverage calculation. This led to:

• Inaccurate coverage metrics: Requests that were actually tested via folder-level assertions showed as having no tests
• Forced duplication: Users had to duplicate common tests on every request instead of using folder-level tests
• Poor support for organized collections: Enterprise teams using well-organized folder structures were penalized

Example

Consider this Postman collection structure:

{
  "name": "Users API",
  "item": [
    { "name": "Get Users", "request": { "method": "GET", "url": "/users" } },
    { "name": "Create User", "request": { "method": "POST", "url": "/users" } }
  ],
  "event": [{
    "listen": "test",
    "script": {
      "exec": ["pm.expect(pm.response.code).to.be.oneOf([200, 201]);"]
    }
  }]
}

Before: Both requests showed testedStatusCodes = [] ❌
After: Both requests show testedStatusCodes = [200, 201] ✅

Solution

Implementation

1. Created extractTestsFromEvents() helper function

   • Extracts status codes and test scripts from event arrays
   • Reusable for both folder-level and request-level events
   • Supports all common Postman test patterns

2. Modified traverseItems() to support test inheritance

   • Added folderTests parameter to track inherited tests through the folder hierarchy
   • When encountering a folder, extracts its tests and combines with parent folder tests
   • Passes combined tests down to all child items recursively

3. Enhanced regex patterns for better status code extraction

   • Fixed oneOf pattern to properly match: pm.expect(pm.response.code).to.be.oneOf([200, 201])
   • Added alternative pattern: to.be.oneOf([200, 201])
   • Better handling of multiple status codes in a single assertion

Key Features

✅ Folder-level test extraction: Tests defined at folder level are now included
✅ Recursive inheritance: Supports nested folder hierarchies with multiple levels
✅ Combined tests: Folder-level tests are merged with request-level tests
✅ Backward compatible: Existing collections without folder tests work exactly as before
✅ Comprehensive test coverage: 18 tests covering all scenarios including edge cases
✅ Robust edge case handling: Handles empty folders, deep nesting, null events, and mixed patterns

Testing

New Tests Added (18 total)

Original Tests (9):

• ✅ Extract folder-level tests and apply them to all requests
• ✅ Combine folder-level and request-level tests correctly
• ✅ Handle nested folders with tests at multiple levels
• ✅ Support multiple folders with different tests
• ✅ Extract status codes with alternative oneOf patterns
• ✅ Maintain backward compatibility for folders without tests

Edge Case Tests (9):

• ✅ Handle empty folders gracefully
• ✅ Handle folders containing only subfolders
• ✅ Handle requests with no event property
• ✅ Handle empty event arrays
• ✅ Ignore non-test events (prerequest events)
• ✅ Handle deeply nested folders (3+ levels)
• ✅ Handle various status code assertion patterns
• ✅ Handle mixed test patterns in same folder
• ✅ Handle folders with undefined or null event properties

Test Results

• All 198 tests pass (including 18 new tests for folder-level functionality)
• No regressions introduced
• CLI verified with real-world examples
• Integration tests confirm proper test inheritance and coverage calculation

CLI Verification

Tested with real Postman collection containing folder-level tests:

• ✅ Coverage: 70% calculated correctly
• ✅ 6 requests extracted with proper test inheritance
• ✅ Smart mapping functionality working
• ✅ HTML report generation successful

Documentation

Updated README.md with:

• Note about folder-level test support in the "Prepare Your Files" section
• Enhanced "Postman Collection" pros to highlight folder-level test support
• New comprehensive section "Folder-Level Tests in Postman Collections" including:
  • How it works explanation
  • Example JSON structure
  • Benefits and use cases
  • Supported test patterns
  • Edge cases handled (new section documenting robust handling of edge cases)

Updated .github/workflows/release.yml with:

• Folder-level tests feature description in release notes
• Updated test count from 183 to 198 tests
• Added folder tests coverage section
• Enhanced quality assurance details

Impact

This change provides:

• More accurate coverage calculations by including folder-level assertions
• Reduced duplication in Postman collections - define common tests once at folder level
• Better organization support for enterprise collections with folder structures
• Proper nested folder handling for complex collection hierarchies
• Robust edge case handling ensuring reliability across diverse collection structures

Files Changed

• lib/postman.js: Core implementation (103 lines modified)
• test/postman.test.js: Comprehensive test coverage (18 tests total - 9 original + 9 edge cases)
• readme.md: Documentation and examples including edge cases section
• .github/workflows/release.yml: Updated release notes with feature details

---

Fixes the issue where tests defined at the Postman collection folder level were not being included in coverage calculations.

Original prompt

It's important to note that tests may be located in Postman collection folders; they should also be included in the calculation and counting. For example, a test in a folder can be applied to all requests within it.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[dreamquality]

@copilot add tests, verify cli works with such collections where checks are collected in the folders, test edge cases, update readme.md and prepare release.yml file

[Copilot]

@copilot add tests, verify cli works with such collections where checks are collected in the folders, test edge cases, update readme.md and prepare release.yml file

All requested items completed in commit ee59206:

✅ Tests added: 18 total tests in postman.test.js (9 new edge case tests)

• Empty folders, deep nesting (3+ levels), null/undefined events
• Requests without events, empty event arrays, non-test events
• Various status code patterns, mixed patterns, folders with only subfolders

✅ CLI verified: Tested with real collection containing folder-level tests

• Coverage: 70% calculated correctly
• 6 requests extracted with proper test inheritance
• Smart mapping and HTML report generation working

✅ Edge cases tested: 9 comprehensive edge case tests covering:

• Empty folders, deep nesting, null/undefined handling
• Mixed test patterns, non-test events filtering
• Various assertion formats

✅ README updated: Added "Edge Cases Handled" section documenting all supported scenarios

✅ release.yml prepared: Updated with folder-level tests feature, test count (198 tests), and quality assurance details

All 198 tests pass with no regressions.

[Copilot]

Pull Request Overview

This PR implements support for folder-level tests in Postman collections to provide more accurate coverage calculations. Previously, tests defined at the folder level were ignored, leading to inaccurate coverage metrics and forcing users to duplicate common tests across requests.

• Implemented folder-level test extraction and inheritance through the collection hierarchy
• Enhanced regex patterns for better status code extraction from test scripts
• Added comprehensive test coverage with 18 new tests covering various scenarios and edge cases

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
lib/postman.js	Core implementation of folder-level test extraction and inheritance logic
test/postman.test.js	18 comprehensive tests covering folder-level functionality and edge cases
readme.md	Documentation updates including new section on folder-level tests with examples
.github/workflows/release.yml	Updated release notes with folder-level tests feature description
auto-detect-newman.html	Timestamp update from test execution

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Russian comments should be translated to English for consistency with the rest of the codebase. Consider translating 'Это папка - извлекаем тесты из папки' to 'This is a folder - extract tests from folder' and 'Объединяем тесты папки с родительскими тестами' to 'Combine folder tests with parent tests'.

[Copilot]

Russian comment should be translated to English. Consider translating 'Это запрос' to 'This is a request'.

Suggested change

	// Это запрос
	// This is a request

[Copilot]

Russian comments should be translated to English. Consider translating 'Извлекаем тесты из самого запроса' to 'Extract tests from the request itself' and 'Объединяем тесты запроса с тестами папки' to 'Combine request tests with folder tests'.

Suggested change

	// Извлекаем тесты из самого запроса
	const requestTestData = extractTestsFromEvents(item.event);
	// Объединяем тесты запроса с тестами папки
	// Extract tests from the request itself
	const requestTestData = extractTestsFromEvents(item.event);
	// Combine request tests with folder tests