Skip to content

Conversation

@npc-code
Copy link

The purpose of this feature is to prevent alterations to existing tables that would insert new columns before existing ones.

KUSTO_ENABLE_COLUMN_VALIDATION must be set to true in order for this check to come into play. This preserves the current default behavior to prevent existing workflows from being unexpectedly disrupted.

When set:

  • column order is checked
  • if the column order does not pass the validation check, the deployment is blocked

Included are unit tests, documentation, etc.

Copilot AI review requested due to automatic review settings October 27, 2025 18:37
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces an optional column order validation feature that prevents new columns from being inserted before existing columns in Kusto table definitions, which can cause update policy failures. The feature is disabled by default and can be enabled via the KUSTO_ENABLE_COLUMN_VALIDATION environment variable or programmatically.

Key Changes:

  • Added ValidationSettings class to manage feature flags with environment variable support
  • Implemented ColumnOrderValidator to detect when existing columns appear after new columns in table definitions
  • Modified DatabaseChanges.GenerateChanges to optionally apply validation and block deployments with violations

Reviewed Changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
README.md Documents the column order validation feature, configuration options, and example scenarios
KustoSchemaTools/Validation/ValidationResult.cs New class representing validation success/failure with error messages and severity levels
KustoSchemaTools/Validation/ColumnOrderValidator.cs Core validation logic that checks column ordering rules
KustoSchemaTools/Configuration/ValidationSettings.cs Configuration class with environment variable parsing for feature flags
KustoSchemaTools/Changes/DatabaseChanges.cs Updated to accept validation settings and apply column order checks to table changes
KustoSchemaTools.Tests/Parser/KustoClusterHandlerTests.cs Fixed null forgiving operator usage in test
KustoSchemaTools.Tests/Validation/ColumnOrderValidatorTests.cs Comprehensive unit tests for column order validation logic
KustoSchemaTools.Tests/Configuration/ValidationSettingsTests.cs Tests for configuration parsing and environment variable handling
KustoSchemaTools.Tests/Changes/DatabaseChangesValidationFlagTests.cs Tests verifying validation is only applied when enabled
KustoSchemaTools.Tests/Changes/DatabaseChangesColumnValidationTests.cs Integration tests for validation behavior with various table scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ianwinsemius
Copy link

There have been a couple of incidents related to the order of columns in Kusto not lining up correctly that have been able to merge in warehouse-config. Being able to validate the order of columns prior to merging would potentially prevent future incidents like this. It became clear that the intent of this update wasn't obvious during a discussion earlier.

Ref:
https://github.com/github/availability/issues/4583
https://github.com/github/availability/issues/4128

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants