Skip to content

FLPATH-3402: add 11 cost_management resource type schemas#1243

Draft
jordigilh wants to merge 3 commits intoproject-kessel:mainfrom
jordigilh:FLPATH-3402/add-cost-management-resource-schemas
Draft

FLPATH-3402: add 11 cost_management resource type schemas#1243
jordigilh wants to merge 3 commits intoproject-kessel:mainfrom
jordigilh:FLPATH-3402/add-cost-management-resource-schemas

Conversation

@jordigilh
Copy link
Copy Markdown

@jordigilh jordigilh commented Mar 6, 2026

Summary

Register 11 resource types for the cost_management reporter so that the Kessel Inventory API can accept ReportResource gRPC calls from Cost Management (Koku).

Why this is needed

The Kessel Inventory API validates every ReportResource gRPC call against resource type schemas defined in data/schema/resources/. The validation flow is:

  1. Reporter calls ReportResource(resource_type=X, reporter_type=Y, ...)
  2. Inventory API looks up data/schema/resources/X/reporters/Y/config.yaml
  3. If found, validates the payload against the reporter-specific JSON schema and the common representation schema
  4. If not found, the call is rejected with an error

Cost Management (Koku) needs to report 11 resource types to the Inventory API as part of its Kessel/ReBAC integration for on-prem authorization. These resource types represent the entities that Koku manages and for which it needs to track authorization relationships (via SpiceDB tuples). Without these schemas registered, Koku cannot participate in the Kessel resource lifecycle — it cannot report, update, or delete resources through the Inventory API.

Currently, only 4 resource types exist in the upstream schema (host, k8s_cluster, k8s_policy, notifications_integration). None cover Cost Management's domain.

What this PR adds

44 new files — 11 resource type directories, each containing 4 files following the identical pattern established by existing resource types (host/reporters/hbi/, k8s_cluster/reporters/acm/, etc.):

File Purpose
<type>/config.yaml Declares resource_type and resource_reporters: [cost_management]
<type>/common_representation.json JSON Schema requiring workspace_id (same as host)
<type>/reporters/cost_management/config.yaml Declares reporter metadata (reporter_name, namespace)
<type>/reporters/cost_management/<type>.json Reporter-specific JSON Schema (minimal — no extra fields beyond common representation)

Resource types and what they represent

Resource Type Description Example resource ID
openshift_cluster OCP clusters reported by Koku cluster UUID
openshift_node OCP nodes within clusters node name
openshift_project OCP projects (namespaces) project name
cost_model Pricing/markup rules applied to sources cost model UUID
settings Tenant-level cost management settings settings key
aws_account AWS linked accounts AWS account ID
aws_organizational_unit AWS organizational units OU ID
azure_subscription_guid Azure subscriptions subscription GUID
gcp_account GCP billing accounts billing account ID
gcp_project GCP projects project ID
integration Data sources connecting cloud providers to Koku source UUID

Schema details

  • common_representation.json requires workspace_id (type: string) — identical to the existing host resource type schema. This field links the resource to an authorization workspace.
  • Reporter-specific JSON schemas are minimal ("properties": {}, "required": []) because Cost Management does not attach reporter-specific metadata beyond what the common representation provides. This matches the pattern used by other reporters with simple payloads.
  • Reporter name and namespace are both cost_management, following the convention where reporter name matches the service namespace.

Example: openshift_cluster (representative of all 11)

openshift_cluster/config.yaml:

resource_type: openshift_cluster
resource_reporters:
  - cost_management

openshift_cluster/common_representation.json:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "workspace_id": { "type": "string" }
  },
  "required": ["workspace_id"]
}

openshift_cluster/reporters/cost_management/config.yaml:

resource_type: openshift_cluster
reporter_name: cost_management
namespace: cost_management

openshift_cluster/reporters/cost_management/openshift_cluster.json:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {},
  "required": []
}

All 11 resource types follow this exact pattern, differing only in the resource_type name.

Validation

These schemas have been validated end-to-end on an on-prem deployment using local copies bundled in the ros-helm-chart kessel-resource-schemas ConfigMap. Koku successfully calls ReportResource, DeleteResource, and StreamedListObjects for all 11 resource types. This PR upstreams them so they are available in the canonical Inventory API definitions.

Related PRs

JIRA

FLPATH-3402

Made with Cursor

Summary by CodeRabbit

  • New Features
    • Added standardized resource definitions and cost-management reporter configurations for AWS, Azure, GCP, OpenShift, cost models, integrations, and settings, including workspace_id validation where applicable.
  • Chores
    • Expanded public schema mappings/cache for consistent reporting and updated packaged deployment payload.

Register 11 resource types for the cost_management reporter so that
the Kessel Inventory API can accept ReportResource gRPC calls from
Cost Management (Koku). Without these schemas, ReportResource calls
for cost management resource types are rejected.

Resource types: openshift_cluster, openshift_node, openshift_project,
cost_model, settings, aws_account, aws_organizational_unit,
azure_subscription_guid, gcp_account, gcp_project, integration.

Each follows the same 4-file pattern as existing resource types
(host/reporters/hbi, k8s_cluster/reporters/acm, etc.).

Made-with: Cursor
@app-sre-bot
Copy link
Copy Markdown
Collaborator

Can one of the admins verify this patch?

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 6, 2026

📝 Walkthrough

Walkthrough

Adds JSON Schema common representations and cost_management reporter schemas/configs for multiple resource types (AWS, Azure, GCP, OpenShift, cost_model, integration, settings); updates schema_cache.json and replaces a base64 payload in deploy/kessel-inventory-ephem.yaml.

Changes

Cohort / File(s) Summary
AWS
data/schema/resources/aws_account/common_representation.json, data/schema/resources/aws_account/reporters/cost_management/aws_account.json, data/schema/resources/aws_account/reporters/cost_management/config.yaml, data/schema/resources/aws_account/config.yaml, data/schema/resources/aws_organizational_unit/...
Added common_representation.json (requires workspace_id), empty/placeholder cost_management reporter schemas, and reporter/config YAML entries. Check reporter registration and schema paths.
Azure
data/schema/resources/azure_subscription_guid/common_representation.json, data/schema/resources/azure_subscription_guid/reporters/cost_management/azure_subscription_guid.json, .../config.yaml
Added common_representation.json, empty cost_management reporter schema, and reporter config YAML. Verify reporter namespace and resource_type values.
GCP
data/schema/resources/gcp_account/..., data/schema/resources/gcp_project/...
Added common_representation.json files, cost_management reporter schemas (empty) and reporter config YAMLs. Confirm consistent workspace_id requirement across common representations.
OpenShift
data/schema/resources/openshift_cluster/..., data/schema/resources/openshift_node/..., data/schema/resources/openshift_project/...
Added common representations requiring workspace_id, empty reporter schemas, and reporter config YAMLs for cluster/node/project.
Cost model / Integration / Settings
data/schema/resources/cost_model/..., data/schema/resources/integration/..., data/schema/resources/settings/...
Introduced resource configs, common representations (where present), cost_management reporter schemas (often placeholders) and reporter config YAMLs.
Repository index
schema_cache.json
Expanded schema cache with many new schema entries and base64-encoded config/schema blobs; review for correct keys and encoded payload consistency.
Deployment manifest
deploy/kessel-inventory-ephem.yaml
Replaced base64 payload for resources.tar.gz in ConfigMap binaryData; verify the tarball contents match the new schemas.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding 11 cost_management resource type schemas as referenced in FLPATH-3402.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check ✅ Passed The PR description is comprehensive and well-structured, covering the purpose, implementation details, examples, and validation. It includes all critical information and follows a logical flow.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@data/schema/resources/aws_account/config.yaml`:
- Around line 1-3: The new resource_type "aws_account" was added but
schema_cache.json wasn't updated; run the schema preloader and commit the
regenerated cache by executing the preload command (go run main.go
preload-schema) to rebuild schema_cache.json so it includes the aws_account
resource schema, then add and commit the updated schema_cache.json so the CI
verify-schema-tarball.yml check passes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8d67736b-6eaf-402a-8806-8963b6a3f86e

📥 Commits

Reviewing files that changed from the base of the PR and between e5b2ec8 and b827ccc.

📒 Files selected for processing (44)
  • data/schema/resources/aws_account/common_representation.json
  • data/schema/resources/aws_account/config.yaml
  • data/schema/resources/aws_account/reporters/cost_management/aws_account.json
  • data/schema/resources/aws_account/reporters/cost_management/config.yaml
  • data/schema/resources/aws_organizational_unit/common_representation.json
  • data/schema/resources/aws_organizational_unit/config.yaml
  • data/schema/resources/aws_organizational_unit/reporters/cost_management/aws_organizational_unit.json
  • data/schema/resources/aws_organizational_unit/reporters/cost_management/config.yaml
  • data/schema/resources/azure_subscription_guid/common_representation.json
  • data/schema/resources/azure_subscription_guid/config.yaml
  • data/schema/resources/azure_subscription_guid/reporters/cost_management/azure_subscription_guid.json
  • data/schema/resources/azure_subscription_guid/reporters/cost_management/config.yaml
  • data/schema/resources/cost_model/common_representation.json
  • data/schema/resources/cost_model/config.yaml
  • data/schema/resources/cost_model/reporters/cost_management/config.yaml
  • data/schema/resources/cost_model/reporters/cost_management/cost_model.json
  • data/schema/resources/gcp_account/common_representation.json
  • data/schema/resources/gcp_account/config.yaml
  • data/schema/resources/gcp_account/reporters/cost_management/config.yaml
  • data/schema/resources/gcp_account/reporters/cost_management/gcp_account.json
  • data/schema/resources/gcp_project/common_representation.json
  • data/schema/resources/gcp_project/config.yaml
  • data/schema/resources/gcp_project/reporters/cost_management/config.yaml
  • data/schema/resources/gcp_project/reporters/cost_management/gcp_project.json
  • data/schema/resources/integration/common_representation.json
  • data/schema/resources/integration/config.yaml
  • data/schema/resources/integration/reporters/cost_management/config.yaml
  • data/schema/resources/integration/reporters/cost_management/integration.json
  • data/schema/resources/openshift_cluster/common_representation.json
  • data/schema/resources/openshift_cluster/config.yaml
  • data/schema/resources/openshift_cluster/reporters/cost_management/config.yaml
  • data/schema/resources/openshift_cluster/reporters/cost_management/openshift_cluster.json
  • data/schema/resources/openshift_node/common_representation.json
  • data/schema/resources/openshift_node/config.yaml
  • data/schema/resources/openshift_node/reporters/cost_management/config.yaml
  • data/schema/resources/openshift_node/reporters/cost_management/openshift_node.json
  • data/schema/resources/openshift_project/common_representation.json
  • data/schema/resources/openshift_project/config.yaml
  • data/schema/resources/openshift_project/reporters/cost_management/config.yaml
  • data/schema/resources/openshift_project/reporters/cost_management/openshift_project.json
  • data/schema/resources/settings/common_representation.json
  • data/schema/resources/settings/config.yaml
  • data/schema/resources/settings/reporters/cost_management/config.yaml
  • data/schema/resources/settings/reporters/cost_management/settings.json

Run `go run main.go preload-schema` to include all 11 new
cost_management resource types in the JSON schema cache. Required
by the verify-schema-tarball CI workflow.

Made-with: Cursor
@jordigilh
Copy link
Copy Markdown
Author

Addressed in 42b8020 — ran go run main.go preload-schema to regenerate schema_cache.json with all 11 cost_management resource types.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
main 49.87% <ø> (ø)
v1beta2 65.07% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Adam0Brien
Copy link
Copy Markdown
Member

Adam0Brien commented Mar 9, 2026

Hey @jordigilh Looks like our workflow didnt add the comment on the schema tarball not generating!
You should be able to run

make build-schemas

and that should generate the new resource tarball file

Run `make build-schemas` to include the 11 new cost_management resource
types in the schema tarball and update the ephemeral deployment config.

Made-with: Cursor
@jordigilh
Copy link
Copy Markdown
Author

Thanks @Adam0Brien — done in bc42814. Ran make build-schemas to regenerate resources.tar.gz (now includes all 15 resource types) and the ephemeral deploy config was updated as well. CI runs are pending maintainer approval.

@Rajagopalan-Ranganathan Rajagopalan-Ranganathan marked this pull request as draft April 1, 2026 12:15
@Rajagopalan-Ranganathan
Copy link
Copy Markdown
Contributor

Marking this as draft to prevent any accidental merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants