Skip to content

talltechy/insightvm-python

Repository files navigation

InsightVM-Python

Version Python License Codacy Badge

A modern Python client library for Rapid7 InsightVM API. Built with industry-standard patterns, comprehensive type hints, and a clean, intuitive interface.

✨ Features

  • Modern Authentication - HTTPBasicAuth integration following industry standards
  • Unified Client Interface - Single entry point with organized sub-clients
  • Comprehensive Asset Management - Full CRUD operations with auto-pagination
  • Asset Group Operations - Create, update, delete, and manage dynamic groups
  • Site Management - Complete site lifecycle management and configuration
  • Scan Operations - Start, stop, pause, resume, and monitor vulnerability scans
  • Report Management - Generate, download, and manage security reports
  • Sonar Query Integration - Query and analyze Sonar data
  • Type-Safe - Complete type hints throughout the codebase
  • Self-Signed Certificate Support - Configurable SSL verification for enterprise environments
  • Context Manager Support - Proper resource cleanup
  • Well Documented - Comprehensive docstrings and examples

πŸš€ Quick Start

Installation

# Clone the repository
git clone https://github.com/talltechy/insightvm-python.git
cd insightvm-python

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Configuration

Create a .env file in the project root (see .env.example for template).

Do NOT commit real credentials. Use .env.example as a template for local development only, and use your platform's secret manager (GitHub Secrets, AWS Secrets Manager, Azure Key Vault, etc.) for CI and production settings.

Example placeholders (do not commit real values):

# Rapid7 InsightVM API (placeholders - DO NOT COMMIT)
INSIGHTVM_API_USERNAME=<INSIGHTVM_API_USERNAME>
INSIGHTVM_API_PASSWORD=<INSIGHTVM_API_PASSWORD>
INSIGHTVM_BASE_URL=https://your-console:3780

# SSL Configuration (optional)
INSIGHTVM_VERIFY_SSL=false  # Set to false only for trusted development/testing environments

See .github/copilot-instructions.md for Copilot-specific guidance on generating examples and handling secrets safely.

Basic Usage

from rapid7 import InsightVMClient

# Create client (loads credentials from environment)
with InsightVMClient() as client:
    # Asset Management
    assets = client.assets.list(page=0, size=100)
    print(f"Found {len(assets['resources'])} assets")
    
    # Scan Operations
    scan_id = client.scans.start_site_scan(
        site_id=123,
        scan_name="Security Audit"
    )
    print(f"Started scan: {scan_id}")
    
    # Report Generation
    content = client.reports.generate_and_download(
        report_id=42,
        timeout=3600
    )
    with open("security_report.pdf.gz", "wb") as f:
        f.write(content)
    
    # Asset Group Management
    group = client.asset_groups.create_high_risk(
        name="Critical Assets",
        threshold=25000
    )
    print(f"Created group: {group['name']}")

πŸ“š Documentation

Core Documentation

API Module Documentation

πŸ—οΈ Architecture

Project Structure

insightvm-python/
β”œβ”€β”€ src/rapid7/              # Main package
β”‚   β”œβ”€β”€ auth.py             # Authentication classes
β”‚   β”œβ”€β”€ client.py           # InsightVMClient
β”‚   β”œβ”€β”€ config.py           # Configuration management
β”‚   β”œβ”€β”€ constants.py        # API constants
β”‚   β”œβ”€β”€ ui.py              # User interface utilities
β”‚   └── api/               # API modules
β”‚       β”œβ”€β”€ base.py        # BaseAPI foundation
β”‚       β”œβ”€β”€ assets.py      # Asset operations
β”‚       β”œβ”€β”€ asset_groups.py # Asset group operations
β”‚       β”œβ”€β”€ scans.py       # Scan management
β”‚       β”œβ”€β”€ reports.py     # Report generation
β”‚       β”œβ”€β”€ sites.py       # Site management
β”‚       └── sonar_queries.py # Sonar integration
β”œβ”€β”€ docs/                  # Documentation
β”‚   β”œβ”€β”€ API_REFERENCE.md
β”‚   β”œβ”€β”€ SCANS_API.md
β”‚   β”œβ”€β”€ REPORTS_API.md
β”‚   └── ...
β”œβ”€β”€ requirements.txt       # Dependencies
β”œβ”€β”€ .env.example          # Configuration template
└── SECURITY.md           # Security policy

Design Patterns

BaseAPI Inheritance - All API modules inherit from a common base class:

from rapid7.api.base import BaseAPI

class ScansAPI(BaseAPI):
    MAX_PAGE_SIZE = 500  # Optimization constant
    
    def list(self, page=0, size=500):
        size = min(size, self.MAX_PAGE_SIZE)
        return self._request('GET', 'scans', params={'page': page, 'size': size})

Unified Client - Single entry point with sub-clients:

client = InsightVMClient()
client.assets.list()         # Asset operations
client.asset_groups.list()   # Asset group operations
client.scans.list()          # Scan operations
client.reports.list()        # Report operations
client.sites.list()          # Site operations
client.sonar_queries.list()  # Sonar operations

πŸ”§ Advanced Usage

Custom Configuration

from rapid7 import InsightVMClient

# Explicit credentials
client = InsightVMClient(
    username="admin",
    password="password",
    base_url="https://console:3780",
    verify_ssl=False,
    timeout=(10, 90)  # (connect, read) timeouts
)

Scan Management

# Start a scan for a site
scan_id = client.scans.start_site_scan(
    site_id=123,
    scan_name="Monthly Security Scan",
    scan_template_id="full-audit-without-web-spider"
)

# Monitor scan progress
scan = client.scans.get_scan(scan_id)
print(f"Status: {scan['status']}")
print(f"Progress: {scan.get('tasks', {}).get('pending', 0)} tasks pending")

# Wait for completion
final_scan = client.scans.wait_for_completion(
    scan_id,
    poll_interval=60,
    timeout=7200
)

# Stop a running scan if needed
client.scans.stop_scan(scan_id)

Report Generation

# List available report templates
templates = client.reports.get_templates()
for template in templates['resources']:
    print(f"{template['id']}: {template['name']}")

# Generate and download a report
content = client.reports.generate_and_download(
    report_id=42,
    poll_interval=30,
    timeout=3600
)

# Save the report (usually GZip compressed)
with open("vulnerability_report.pdf.gz", "wb") as f:
    f.write(content)

# Or manage report generation manually
instance_id = client.reports.generate(report_id=42)
client.reports.wait_for_completion(42, instance_id)
report_content = client.reports.download(42, instance_id)

Auto-Pagination

# Get all assets (handles pagination automatically)
all_assets = client.assets.get_all(batch_size=500)
print(f"Total assets: {len(all_assets)}")

# Get all scans across all pages
all_scans = client.scans.get_all_scans()
print(f"Total scans: {len(all_scans)}")

# Get all reports
all_reports = client.reports.get_all_reports()
print(f"Total reports: {len(all_reports)}")

Advanced Search

# Search for high-risk Windows servers
results = client.assets.search({
    "filters": [
        {"field": "risk-score", "operator": "is-greater-than", "value": 20000},
        {"field": "operating-system", "operator": "contains", "value": "Windows Server"}
    ],
    "match": "all"
})

# Filter scans by status
active_scans = client.scans.list(active=True)

Error Handling

import requests

try:
    client = InsightVMClient()
    
    # Start a scan
    scan_id = client.scans.start_site_scan(site_id=123)
    
    # Wait for completion with timeout
    result = client.scans.wait_for_completion(
        scan_id, 
        timeout=3600
    )
    
except ValueError as e:
    print(f"Configuration error: {e}")
except TimeoutError as e:
    print(f"Operation timed out: {e}")
except requests.exceptions.RequestException as e:
    print(f"API error: {e}")

πŸ” Security

Best Practices

  • No Hardcoded Credentials - All credentials loaded from environment variables
  • HTTPS Only - All API communication over secure connections
  • SSL Verification - Configurable for self-signed certificates (see SECURITY.md)
  • Secret Management - Use .env for development, secret management services for production
  • HTTPBasicAuth - Industry-standard authentication pattern
  • Timeout Configuration - Prevent hanging connections

Secrets Policy

This project enforces a strict secrets policy to avoid accidental credential leaks:

  • Never store API keys, passwords, tokens, or private keys in the repository or documentation examples.
  • Examples should reference environment variables or secret manager usage (e.g. os.getenv('INSIGHTVM_API_PASSWORD')) and use placeholders such as <INSIGHTVM_API_PASSWORD> when needed.
  • For development, use a local .env (gitignored). For CI and production, use a secret manager (GitHub Secrets, AWS Secrets Manager, Azure Key Vault, etc.).
  • See .github/copilot-instructions.md for additional guidance used by GitHub Copilot when generating or editing code.

Security Considerations

⚠️ Self-Signed Certificates: When using verify_ssl=False, you bypass SSL certificate validation. Only use this in trusted environments with self-signed certificates.

See SECURITY.md for complete security policy and vulnerability reporting.

πŸ§ͺ Testing

Running Tests

# Install test dependencies (included in requirements.txt)
pip install -r requirements.txt

# Run all tests
pytest

# Run with coverage report (local)
pytest --cov=src --cov-report=html

# Run specific test files
pytest tests/test_auth.py
pytest tests/test_client.py
pytest tests/test_rapid7/

# Run tests in verbose mode
pytest -v

# Run tests with coverage and open HTML report
pytest --cov=src --cov-report=html && open htmlcov/index.html

Docker Testing (Optional)

For testing in an environment matching CI/CD:

# Build and run tests in Docker
./.docker-test.sh

# Or manually:
docker build -f Dockerfile.test -t insightvm-test:local .
docker run --rm insightvm-test:local

This provides a consistent testing environment that matches the GitHub Actions workflow.

Test Coverage

This project includes automated test coverage reporting with Codacy integration:

  • CI/CD Coverage: Tests run automatically on push and PR to main/develop
  • Coverage Reporting: Uploaded to Codacy dashboard in real-time
  • Multi-Python Support: Tested across Python 3.8-3.12
  • Coverage Format: Cobertura XML for Codacy compatibility

Current Coverage Targets: 30-40% baseline with room for expansion

Test Structure

tests/
β”œβ”€β”€ __init__.py
β”œβ”€β”€ conftest.py                 # Shared fixtures and utilities
β”œβ”€β”€ test_auth.py                # Authentication module tests
β”œβ”€β”€ test_client.py              # Client initialization tests
└── test_rapid7/                # Rapid7 API module tests
    β”œβ”€β”€ __init__.py
    β”œβ”€β”€ test_base.py           # Base API functionality
    └── test_assets.py         # Assets API examples

Manual Testing

The v2.0 release has been tested against live InsightVM instances:

  • βœ… Authentication with HTTPBasicAuth
  • βœ… Asset retrieval and management (1182+ assets tested)
  • βœ… Asset group creation and management
  • βœ… Scan operations (start, stop, pause, resume)
  • βœ… Report generation and download
  • βœ… Site management operations
  • βœ… Self-signed certificate handling
  • βœ… Auto-pagination for large datasets

πŸ“‹ Requirements

  • Python 3.8+
  • requests >= 2.31.0
  • python-dotenv >= 1.0.0
  • urllib3 >= 2.0.0

πŸ”„ Version History

v2.0.0 (October 2025) - Major Refactoring & Sprint 3

Core Refactoring:

  • βœ… Replaced manual Base64 encoding with HTTPBasicAuth
  • βœ… Unified client interface with sub-clients
  • βœ… BaseAPI inheritance pattern for all modules
  • βœ… Type hints throughout
  • βœ… Context manager support
  • βœ… SSL verification configuration

Sprint 3: Core Operations (COMPLETE - 100%):

  • βœ… Scans API - Complete scan lifecycle management (Issue #66, PR #84)
    • Start, stop, pause, resume scans
    • Monitor scan progress and status
    • Site-based and adhoc scanning
    • Scan history and statistics
  • βœ… Reports API - Full report management (Issue #67, PR #84)
    • Report configuration CRUD
    • Report generation and monitoring
    • Download report content
    • Template and format discovery
  • βœ… Scan Engines API - Engine and pool management (Issue #68, PR #85)
    • CRUD operations for scan engines
    • Engine pool management
    • Health monitoring and load balancing
    • Site and scan associations
  • βœ… Scan Templates API - Template management (Issue #69, PR #86)
    • Template CRUD operations
    • Discovery configuration (asset, service, performance)
    • Service discovery settings
    • Performance optimization helpers
  • βœ… Sites API Standardization (Commit f5980df)
    • Refactored to follow standardized BaseAPI pattern
    • Created SiteManagementTools utility class for advanced operations
    • See SITE_MANAGEMENT.md for migration guide
  • βœ… Optimization Patterns
    • MAX_PAGE_SIZE constants for efficient pagination
    • Enhanced timeout validation
    • Type consistency improvements

Previously Supported:

  • βœ… Comprehensive asset and asset group operations
  • βœ… Sonar query integration

Breaking Changes:

  • ⚠️ Scan Engines API Method Renames (v2.0.x): Three methods renamed to avoid BaseAPI conflicts
    • get(engine_id) β†’ get_engine(engine_id)
    • update(engine_id) β†’ update_engine(engine_id)
    • delete(engine_id) β†’ delete_engine(engine_id)
  • ⚠️ Sites API Standardization: Custom helper methods moved to SiteManagementTools
  • ⚠️ See MIGRATION.md for complete upgrade guide from v1.0
  • ⚠️ See SITE_MANAGEMENT.md for Sites API migration details

v1.0.0 (Previous)

  • Initial release with basic functionality

🀝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md for guidelines.

Development Workflow

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/issue-XX-description)
  3. Follow the GitHub workflow (issues β†’ branches β†’ PRs)
  4. Commit with conventional commit format
  5. Push to the branch (git push origin feature/issue-XX-description)
  6. Open a Pull Request

Current Sprint Progress

Sprint 3: Core Operations βœ… COMPLETE (100%)

  • βœ… Issue #66: Scans API Module (COMPLETE - PR #84)
  • βœ… Issue #67: Reports API Module (COMPLETE - PR #84)
  • βœ… Issue #68: Scan Engines API Module (COMPLETE - PR #85)
  • βœ… Issue #69: Scan Templates API Module (COMPLETE - PR #86)
  • βœ… Sites API Standardization (COMPLETE - Commit f5980df)

Sprint 4: Vulnerabilities & Remediation (NEXT - High Priority)

  • ⏳ Issue #70: Vulnerabilities API Module
  • ⏳ Issue #71: Solutions API Module
  • ⏳ Issue #72: Vulnerability Exceptions API Module

πŸ“– API References

πŸ“ License

This project is licensed under the MIT License - see the LICENSE file for details.

πŸ™ Acknowledgments

  • Rapid7 for the InsightVM API
  • The Python requests library maintainers
  • Contributors to this project

πŸ’¬ Support

For issues, questions, or contributions, please:


Note: This is v2.0 with breaking changes from v1.0. See MIGRATION.md for upgrade instructions.

About

Rapid7 InsightVM Postgres Reporting built in Python and PostgreSQL Rapid7 InsightVM API tools using Python PaloAlto Cortex XDR API

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 6