Skip to content

Conversation

@Chaffelson
Copy link
Owner

@Chaffelson Chaffelson commented Dec 3, 2025

| GitHub CI/CD Integration - Native support for NiFi's GitHub Flow Registry Client

GitHub Flow Registry Support

  • Git-specific versioning helpers: New functions to work with NiFi's native GitHub Flow Registry Client
    • list_git_registry_buckets: List buckets (folders) in a Git-backed registry
    • get_git_registry_bucket: Get a specific bucket by name
    • list_git_registry_flows: List flows in a bucket
    • get_git_registry_flow: Get a specific flow by name
    • list_git_registry_flow_versions: List all versions (commits) of a flow
    • deploy_git_registry_flow: Deploy a versioned flow from GitHub to the NiFi canvas
    • update_git_flow_ver: Change version of an already-deployed Git-registry flow
  • Registry client management: ensure_registry_client and update_registry_client for idempotent registry configuration
  • Enhanced revert: revert_flow_ver now accepts wait=True parameter for synchronous operation

Profile System Enhancements

  • "env" profile for CI/CD: New special profile name that configures nipyapi entirely from environment variables
    • No profiles file required - ideal for GitHub Actions, containers, and CI/CD pipelines
    • Uses nipyapi.profiles.switch('env') to activate
    • All standard environment variable mappings (NIFI_API_ENDPOINT, NIFI_USERNAME, etc.) apply

Controller Service Management

  • Bulk controller service operations: schedule_all_controllers(pg_id, scheduled) to enable/disable all controller services in a process group
    • Uses NiFi's native bulk activation API
    • Handles all descendant controller services automatically
    • Simplifies flow start/stop operations in CI/CD workflows

Bug Fixes

  • Fixed test_create_controller leaving orphaned ADLS controller services after test runs
  • Improved test cleanup fixtures for better isolation
  • Fixed version sorting in deploy_git_registry_flow to correctly identify latest version

Infrastructure

  • Updated pylintrc for pylint 3.x compatibility
  • Increased module line limit for versioning.py to accommodate new functions
  • Added comprehensive test fixtures for Git registry integration testing

Related Projects

Add native support for NiFi's GitHub Flow Registry Client to enable
CI/CD workflows for versioned NiFi flows.

New Git-specific versioning helpers:
- list_git_registry_buckets, get_git_registry_bucket
- list_git_registry_flows, get_git_registry_flow
- list_git_registry_flow_versions, deploy_git_registry_flow
- ensure_registry_client, update_registry_client

Profile system enhancements:
- Add 'env' profile for pure environment variable configuration
- Ideal for GitHub Actions, containers, and CI/CD pipelines
- No profiles file required when using nipyapi.profiles.switch('env')

Controller service management:
- Add schedule_all_controllers for bulk enable/disable operations
- Uses NiFi's native bulk activation API with descendant support

Bug fixes:
- Fix test_create_controller leaving orphaned controller services

Documentation:
- Add 1.1.0 release notes to history
- Add env profile usage guide to profiles documentation

Related: nipyapi-actions and nipyapi-workflow companion repositories
@codecov-commenter
Copy link

codecov-commenter commented Dec 3, 2025

Codecov Report

❌ Patch coverage is 79.26267% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.03%. Comparing base (107e324) to head (5cb732a).

Files with missing lines Patch % Lines
nipyapi/versioning.py 79.65% 35 Missing ⚠️
nipyapi/canvas.py 80.76% 5 Missing ⚠️
nipyapi/profiles.py 73.68% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #393      +/-   ##
==========================================
+ Coverage   70.38%   71.03%   +0.65%     
==========================================
  Files           9        9              
  Lines        1830     2020     +190     
==========================================
+ Hits         1288     1435     +147     
- Misses        542      585      +43     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Add nifi-github, nifi-github.localdomain, host.docker.internal to certificate SANs
- Configure nifi-github service to mount generated certificates
- Add NIFI_WEB_PROXY_HOST for host.docker.internal access
- Enables act testing with proper SSL certificate validation
- update_processor now accepts name param for renaming processors
- add auto_stop param (default False) to handle running processors
- revert_flow_ver now refreshes revision internally to prevent stale errors
@Chaffelson Chaffelson marked this pull request as ready for review December 4, 2025 17:15
@Chaffelson Chaffelson self-assigned this Dec 4, 2025
Copy link
Collaborator

@ottobackwards ottobackwards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Minor version bump?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants