Skip to content

Conversation

@alexreal1314
Copy link
Contributor

@alexreal1314 alexreal1314 commented Oct 30, 2025

This is a BREAKING CHANGE that updates the field mapping that aligns cnvm with other integrations - tenable_io, m365 etc.

Summary

This PR updates the vulnerability.published_date field mapping from keyword to date type in the Cloud Native Vulnerability Management (CNVM) integration. The change includes:

  1. modifying the field definition in fields/vulnerability.yml to use the date type
  2. adding a date processor to the ingest pipeline (default.yml) that parses the published_date field using ISO8601 format with proper error handling.
  3. bumping the package version from 3.1.1 to 4.0.0 following semantic versioning for breaking changes
  4. documenting the breaking change in changelog.yml. The ingest pipeline processor includes conditional logic to only parse the field when it exists and is not empty, with on_failure handling to append error messages without breaking the entire pipeline.

change was made to address this issue.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Breaking Change Impact

⚠️ Important: This is a breaking change that will result in mapping conflicts within the data stream during the upgrade process:

  • Existing indices will retain the old keyword mapping for vulnerability.published_date
  • New indices (created after rollover) will use the new date mapping
  • Multiple backing indices with different mappings will coexist under the same logs-cloud_security_posture.vulnerabilities-* data stream

User Impact:

  • Queries spanning both old and new indices may produce inconsistent results or errors due to field type conflicts
  • Sorting by vulnerability.published_date will work correctly only for data in new indices
  • Data views and dashboards may show mapping conflicts

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

https://github.com/elastic/sdh-security-team/issues/1477

Screenshots

Screenshots
image

Screen recording
https://github.com/user-attachments/assets/75a01ab4-c48a-4e5e-8fdb-c85b260eb229

@alexreal1314 alexreal1314 self-assigned this Oct 30, 2025
@alexreal1314 alexreal1314 added breaking change Team:Cloud Security Cloud Security team [elastic/cloud-security-posture] Integration:cloud_security_posture Security Posture Management labels Oct 30, 2025
@alexreal1314 alexreal1314 changed the title change vulnerability.published_date field type from keyword to date in cnvm integration [Contextual Security][CNVM] change vulnerability.published_date field type from keyword to date Oct 30, 2025
…n cnvm

This is a BREAKING CHANGE that updates the field mapping that aligns cnvm with other intengrations - tenable_io, m365 etc.
@alexreal1314 alexreal1314 force-pushed the 12860-vulnerability-published-date branch from 2e19760 to e8fd142 Compare October 30, 2025 12:57
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Oct 30, 2025

🚀 Benchmarks report

Package cloud_security_posture 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
findings 58823.53 50000 -8823.53 (-15%) 👍
vulnerabilities 125000 83333.33 -41666.67 (-33.33%) 💔

@alexreal1314
Copy link
Contributor Author

/test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @alexreal1314

@maxcold
Copy link
Contributor

maxcold commented Nov 3, 2025

From the discussion in Slack:
The main concern is that this breaking change forces users to manually recreate the latest index (and reindex data?). This might be too much to ask from our customers, and they may end up with mapping conflicts. If we are to do that, we need to agree on the strategy and make the trade-offs clear for the product folks.
An alternative is to implement first the approach we have for misconfigurations - have the destination index versioned and add an alias that won't change with breaking changes. Maybe doing that first would make the mapping change smoother

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Integration:cloud_security_posture Security Posture Management Team:Cloud Security Cloud Security team [elastic/cloud-security-posture]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants