feat: Feast proto & pydantic model changes to add defaultvalue to field#336
Open
vanitabhagwat wants to merge 53 commits intoregistry_defaults_featurefrom
Open
feat: Feast proto & pydantic model changes to add defaultvalue to field#336vanitabhagwat wants to merge 53 commits intoregistry_defaults_featurefrom
vanitabhagwat wants to merge 53 commits intoregistry_defaults_featurefrom
Conversation
vanitabhagwat
changed the title
vanitabhagwat
changed the title
Collaborator
|
added 16 commits
January 26, 2026 13:26
…east into registry_defaults
vanitabhagwat
changed the base branch from
registry_defaults_feature
to
master
vanitabhagwat
changed the base branch from
master
to
registry_defaults_feature
added 6 commits
February 27, 2026 00:30
…e tests - Add Float32 default_value test - Add Float64 (Double) default_value test - Add Bytes default_value test - Add String array default_value test - Add Float32 array default_value test - Add Bool array default_value test - Updated imports to include Float64 and Bytes types Completes coverage gaps identified in Task 1 and approved by user.
- Add field_serializer to convert proto Value to JSON dict with camelCase keys - Add field_validator to accept both proto Value and dict for default_value - Use google.protobuf.json_format (MessageToDict/ParseDict) for conversion - Enables HTTP Registry endpoints to return defaultValue in JSON responses
- 11 unit tests covering all serialization scenarios - Tests primitive types: Int64, String, Float64, Bool - Tests None/null handling - Tests deserialization from dict and proto Value - Tests roundtrip: serialize -> deserialize - Tests Field bridge methods: to_field/from_field - Tests full roundtrip: Field -> FieldModel -> JSON -> FieldModel -> Field
- Add serialization_alias='defaultValue' for camelCase JSON output - Add validation_alias='defaultValue' to accept camelCase JSON input - Import Pydantic Field for alias configuration Enables HTTP REST API to use camelCase 'defaultValue' in JSON responses while maintaining snake_case 'default_value' in Python code.
- test_feature_view_proto_roundtrip_with_defaults: FeatureView with defaults - test_feature_view_proto_roundtrip_without_defaults: Backwards compatibility - test_feature_view_proto_roundtrip_mixed_defaults: Mixed fields with/without defaults - test_sorted_feature_view_proto_roundtrip_with_defaults: SortedFeatureView preserves sort_keys and defaults - test_feature_view_proto_bytes_identity: Verify proto wire format contains default_value All 5 tests pass. Verifies that FeatureView.to_proto() / from_proto() preserves Field.default_value through serialization, which is the path used by Remote Registry (feast serve_registry) gRPC communication.
- Configure FastAPI endpoints to exclude None values from JSON - Prevents defaultValue: null from appearing for fields without defaults - Maintains backwards compatibility with existing API consumers
added 28 commits
February 27, 2026 01:49
- Use alias='defaultValue' with populate_by_name=True - Enables both Python (default_value) and JSON (defaultValue) field names - Field serializer now properly converts proto Value to camelCase JSON - All serialization/deserialization paths working correctly
- Remove alias to be consistent with other fields (vector_index, vector_length, etc.) - All FieldModel fields now use snake_case in JSON (name, dtype, default_value) - Proto value contents still use camelCase per proto3 JSON spec (int64Val, stringVal) - Simpler, more consistent with existing API patterns
- Add DefaultValue *types.Value field to Field struct - Update NewFieldFromProto to populate DefaultValue from proto - Add CreateFeatureWithDefault test helper function This enables the Field model to store default values defined in the registry, which will be used by the serving layer to replace NULL/NOT_FOUND values.
Add comprehensive table-driven test TestApplyDefaults covering: - OFF mode: leaves NOT_FOUND/NULL unchanged - FLEXIBLE mode: applies defaults when available, leaves when not - FLEXIBLE mode: keeps PRESENT values unchanged - UNSPECIFIED mode: behaves like OFF Tests both Arrow and non-Arrow paths. Currently fails as expected - TransposeFeatureRowsIntoColumns doesn't yet accept useDefaults parameter. TDD RED phase complete.
…owsIntoColumns GREEN phase implementation: 1. Updated TransposeFeatureRowsIntoColumns signature to accept UseDefaultsMode parameter 2. Added featureDefaults lookup map at function start 3. Implemented defaulting logic after status determination: - FLEXIBLE mode: replaces NOT_FOUND/NULL_VALUE with defaults when available - OFF/UNSPECIFIED modes: preserves existing behavior (no replacement) - Does NOT apply defaults to OUTSIDE_MAX_AGE status 4. Updated all callers (featurestore.go uses OFF mode for backward compatibility) 5. Updated existing tests and benchmarks to pass OFF mode 6. Fixed test to handle Arrow's nil value representation (empty Value with nil Val) 7. Added OUTSIDE_MAX_AGE test case to ensure defaults not applied to expired values All tests pass including: - 8 test cases with both Arrow and non-Arrow paths (16 total) - Existing transpose tests (backward compatibility) - All onlineserving package tests TDD GREEN phase complete.
- Add useDefaults parameter to FeatureStore.GetOnlineFeatures - Pass useDefaults to TransposeFeatureRowsIntoColumns (both v2 and v1 batching paths) - Extract use_defaults from gRPC request via GetUseDefaults() - Add UseDefaults field to HTTP JSON request struct - Implement parseUseDefaultsMode helper for HTTP (handles nil/OFF/FLEXIBLE/STRICT/invalid) - Add TestParseUseDefaultsMode unit test (5 cases covering all modes) - Update all 7 GetOnlineFeatures callers with OFF mode for backward compatibility Callers updated: - grpc_server.go: passes request.GetUseDefaults() - http_server.go: passes parseUseDefaultsMode(request.UseDefaults) - featurestore_test.go: 2 calls with OFF mode - embedded/online_features.go: 1 call with OFF mode
- Add TestFieldDefaultValueLoadedFromProto to validate FS-01 requirement - Test confirms proto FeatureSpecV2.DefaultValue populates Field.DefaultValue - Verifies feature with default: DefaultValue = 42 - Verifies feature without default: DefaultValue = nil - Validates proto -> model -> Field.DefaultValue chain works correctly
- Add TestApplyRangeDefaults with 9 test cases - Each case tests OFF/FLEXIBLE/UNSPECIFIED modes - Tests both Arrow and Proto value handling (18 total executions) - Tests NOT_FOUND, NULL_VALUE, PRESENT, OUTSIDE_MAX_AGE statuses - Tests cases with and without default values - Test fails as expected: function signature doesn't accept useDefaults yet
- Update TransposeRangeFeatureRowsIntoColumns to accept useDefaults parameter - Build featureDefaults map from sorted feature view fields - Update processFeatureRowData signature to accept defaulting parameters - Apply defaults for NOT_FOUND case when featureData.Values is nil (entity not found) - Apply defaults in per-value loop for nil values with NOT_FOUND/NULL_VALUE status - FLEXIBLE mode replaces NOT_FOUND and NULL_VALUE with defaults when available - OFF/UNSPECIFIED modes leave all statuses unchanged (backward compatible) - OUTSIDE_MAX_AGE values not replaced (correct TTL handling) - Update all callers (featurestore.go, tests) with OFF mode for backward compatibility - Tests pass: TestApplyRangeDefaults (9 cases × 2 modes = 18 executions) - Tests pass: TestTransposeRangeFeatureRowsIntoColumns (backward compat verified) - Full project compiles, no vet issues
- Add 04-01-SUMMARY.md documenting TDD implementation - Range queries support per-value defaulting with sort order preservation - FLEXIBLE mode replaces NOT_FOUND/NULL_VALUE in range arrays - All tests passing, backward compatible
… chain - Updated GetOnlineFeaturesRange signature to accept useDefaults parameter - Pass useDefaults to TransposeRangeFeatureRowsIntoColumns instead of hardcoded OFF - gRPC handler extracts request.GetUseDefaults() and passes to FeatureStore - HTTP handler added UseDefaults field to request struct and passes parseUseDefaultsMode(request.UseDefaults) - Updated testGetOnlineFeaturesRange helper to accept and pass through useDefaults - All existing callers use OFF mode for backward compatibility
- Add expectError and errorContains fields to TestApplyDefaults struct - Add 6 STRICT test cases to TestApplyDefaults (2 success, 2 error, 1 present, 1 OUTSIDE_MAX_AGE) - Add expectError, errorContains, entityNotFound fields to TestApplyRangeDefaults struct - Add 8 STRICT test cases to TestApplyRangeDefaults (2 success, 2 error, 1 present, 1 OUTSIDE_MAX_AGE, 2 entity-not-found) - Tests fail as expected (STRICT mode not yet implemented)
- Add STRICT mode to TransposeFeatureRowsIntoColumns - STRICT validates NULL/NOT_FOUND values have defaults before applying - Returns GrpcInvalidArgumentError if default missing - Add STRICT mode to processFeatureRowData for range queries - Handle entity-not-found case with STRICT validation - Handle per-value NULL/NOT_FOUND with STRICT validation - OUTSIDE_MAX_AGE values excluded from validation (consistent with FLEXIBLE) - All 28 new STRICT mode tests pass (14 regular + 14 range, x2 Arrow modes)
- STRICT mode validation and defaulting implemented - 28 new test cases passing (14 x 2 Arrow modes) - All existing tests continue to pass (backward compatibility) - Ready for API integration in 05-02
- Add feature_defaults_applied_total Prometheus counter with feature_view and feature_name labels - Add zerolog debug logging at all default application points - Instrumented FLEXIBLE mode in TransposeFeatureRowsIntoColumns - Instrumented STRICT mode in TransposeFeatureRowsIntoColumns - Instrumented FLEXIBLE mode entity-not-found case in processFeatureRowData - Instrumented STRICT mode entity-not-found case in processFeatureRowData - Instrumented FLEXIBLE mode per-value case in processFeatureRowData - Instrumented STRICT mode per-value case in processFeatureRowData
- Add TestDefaultsMetricRegistered to verify Prometheus metric is properly registered - Test confirms metric can be described and contains correct name - All existing tests pass with no duplicate registration panics
- SUMMARY.md documents Prometheus counter and zerolog logging implementation - 2 tasks completed in 3min 33sec - No deviations from plan - All verification passed
- Removed prometheus import - Removed featureDefaultsApplied counter declaration and init() - Removed all 6 metric increment calls (preserved all debug logging)
- Removed prometheus import - Removed TestDefaultsMetricRegistered function - All existing tests pass
- Created 05-03-SUMMARY.md with execution details - Removed featureDefaultsApplied metric and test - Preserved all 6 debug logging statements - All verification checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Proto & pydantic model changes to add defaultvalue to registry.
Misc