Feature processor v3 by BassemHalim · Pull Request #5565 · aws/sagemaker-python-sdk

BassemHalim · 2026-02-20T01:36:19Z

Port `feature_processor` module to SageMaker Python SDK v3

Migrates the feature_store.feature_processor module from sagemaker-python-sdk v2 to the v3 modular package structure (sagemaker-mlops), along with supporting changes in sagemaker-core.

Changes

Import path migration — All internal imports updated from sagemaker.feature_store.feature_processor → sagemaker.mlops.feature_store.feature_processor, and external SDK dependencies remapped to their v3 locations:

sagemaker.Session → sagemaker.core.helper.session_helper.Session
sagemaker.lineage → sagemaker.core.lineage
sagemaker.remote_function → sagemaker.core.remote_function
sagemaker.workflow → sagemaker.mlops.workflow
sagemaker.s3 / sagemaker.utils / sagemaker.vpc_utils → sagemaker.core.*

Estimator → ModelTrainer migration (feature_scheduler.py) — Replaced the v2 Estimator dict-based construction with the v3 ModelTrainer API:

Uses Compute, Networking, StoppingCondition, SourceCode, OutputDataConfig, and Tag config objects
Creates a PipelineSession for pipeline-aware execution
TrainingStep now uses step_args from ModelTrainer.train() instead of estimator+inputs

Input channel format change (_config_uploader.py) — prepare_step_input now returns List[Channel] (using Channel/DataSource/S3DataSource shapes) instead of Dict[str, TrainingInput].

Session helper additions (sagemaker-core) — Added Feature Store methods to Session: delete_feature_group, describe_feature_group, create_feature_group, update_feature_group, and related config schema imports.

Diff with V2

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: Add Feature Store Support to V3 * Add feature store tests --------- Co-authored-by: adishaa <adishaa@amazon.com>

…ondition Isolate Ivy cache per Spark session via spark.jars.ivy to prevent concurrent pytest-xdist workers from corrupting shared /root/.ivy2/cache during Maven dependency resolution in CI.

…oncurrent writes in CI

sagemaker-core/src/sagemaker/core/helper/session_helper.py

nargokul · 2026-03-03T19:42:14Z

...mlops/src/sagemaker/mlops/feature_store/feature_processor/lineage/_feature_group_contexts.py

Why do we have these file names starting with _

They are meant to be 'private' only imported from the feature_processor module. I ported them from v2 as-is. Do you prefer I rename them?

nargokul · 2026-03-03T19:43:40Z

sagemaker-mlops/tox.ini

 [testenv]
 setenv =
    PYTHONHASHSEED=42
+    JAVA_HOME={env:JAVA_HOME:/usr/lib/jvm/default-java}


Why do we need this ?
Is java a requirement for mlops package to work?

Java is a requirement for just feature-processor because it requires pySpark which needs java ref

sagemaker-mlops/pyproject.toml

nargokul · 2026-03-03T19:44:20Z

pyproject.toml

 train = ["sagemaker-train"]
 serve = ["sagemaker-serve"]
 mlops = ["sagemaker-mlops"]
+feature-processor = ["sagemaker-mlops", "pyspark==3.3.2", "sagemaker-feature-store-pyspark-3.3", "setuptools<82"]


Why is this needed ?

feature-processor used to require an extra pip install pip install sagemaker[feature-processor] this is because it requires extra dependencies that don't make sense to require all sagemaker users to install docs

V2 requirements: https://github.com/aws/sagemaker-python-sdk/blob/master-v2/requirements/extras/feature-processor_requirements.txt
The extra setuptools<82 is because setuptools removed a module named pkg_resources https://setuptools.pypa.io/en/latest/history.html#v82-0-0

- Replace sagemaker_session.describe_feature_group() calls with FeatureGroup.get() - Update _input_loader.py to use FeatureGroup resource attributes instead of dictionary access - Update feature_scheduler.py to use FeatureGroup.get() and access creation_time as attribute - Update _feature_group_lineage_entity_handler.py to return FeatureGroup resource instead of Dict - Remove unused imports (Dict, Any, FEATURE_GROUP, CREATION_TIME constants) - Replace dictionary key access with typed resource properties (offline_store_config, data_catalog_config, event_time_feature_name, etc.) - Update unit tests to reflect new FeatureGroup resource API usage - Improves type safety and reduces reliance on dictionary-based API responses

BassemHalim had a problem deploying to manual-approval February 20, 2026 01:36 — with GitHub Actions Error

BassemHalim had a problem deploying to manual-approval February 20, 2026 04:18 — with GitHub Actions Error

BassemHalim had a problem deploying to manual-approval February 25, 2026 20:12 — with GitHub Actions Error

BassemHalim changed the base branch from master to feature-store-v3 February 25, 2026 21:40

BassemHalim changed the base branch from feature-store-v3 to master February 27, 2026 05:19

BassemHalim closed this Feb 27, 2026

BassemHalim reopened this Feb 27, 2026

BassemHalim had a problem deploying to manual-approval February 27, 2026 05:32 — with GitHub Actions Error

Aditi2424 and others added 7 commits February 27, 2026 11:19

Feature store v3 (aws#5490)

56eef2d

* feat: Add Feature Store Support to V3 * Add feature store tests --------- Co-authored-by: adishaa <adishaa@amazon.com>

feat: feature_processor v3

d5fd783

integ tests

e91d8d2

fix

4079a5d

chore(docs): Add API docs

e6721ce

fix: Fix flaky integ tests

611f577

fix diff

6f4f490

BassemHalim force-pushed the feature-processor-v3 branch from b8a776b to 6f4f490 Compare February 27, 2026 19:30

BassemHalim had a problem deploying to manual-approval February 27, 2026 19:30 — with GitHub Actions Error

chore: rename parameter + cleanup comments

09abb01

BassemHalim had a problem deploying to manual-approval February 27, 2026 21:03 — with GitHub Actions Error

Merge branch 'master' into feature-processor-v3

a841d7d

mollyheamazon temporarily deployed to manual-approval February 27, 2026 21:40 — with GitHub Actions Inactive

Aditi2424 and others added 2 commits February 27, 2026 15:58

Feature store v3 (aws#5490)

aa654e4

* feat: Add Feature Store Support to V3 * Add feature store tests --------- Co-authored-by: adishaa <adishaa@amazon.com>

add pyspark to test deps

7cdd077

BassemHalim temporarily deployed to manual-approval February 28, 2026 00:03 — with GitHub Actions Inactive

add test deps

101824f

BassemHalim temporarily deployed to manual-approval February 28, 2026 01:49 — with GitHub Actions Inactive

fix unit test deps

1e2b5fe

BassemHalim temporarily deployed to manual-approval March 2, 2026 19:11 — with GitHub Actions Inactive

pin setuptools<82 for feature-processor and unit tests

bbf1a3f

BassemHalim temporarily deployed to auto-approve March 2, 2026 20:32 — with GitHub Actions Inactive

Set JAVA_HOME for integ tests which requires java

5ca86dc

BassemHalim temporarily deployed to auto-approve March 2, 2026 21:11 — with GitHub Actions Inactive

fix spark session bug

beb078d

BassemHalim temporarily deployed to auto-approve March 2, 2026 22:28 — with GitHub Actions Inactive

fix(feature-processor): Fix Spark session config and Ivy cache race c…

7a659af

…ondition Isolate Ivy cache per Spark session via spark.jars.ivy to prevent concurrent pytest-xdist workers from corrupting shared /root/.ivy2/cache during Maven dependency resolution in CI.

BassemHalim temporarily deployed to auto-approve March 2, 2026 23:27 — with GitHub Actions Inactive

revert previous change + create different ivy cache per test to fix c…

59b68e8

…oncurrent writes in CI

BassemHalim temporarily deployed to auto-approve March 3, 2026 00:13 — with GitHub Actions Inactive

Merge branch 'master' into feature-processor-v3

fa59f5e

BassemHalim marked this pull request as ready for review March 3, 2026 17:37

BassemHalim temporarily deployed to auto-approve March 3, 2026 17:37 — with GitHub Actions Inactive

mollyheamazon previously approved these changes Mar 3, 2026

View reviewed changes

nargokul reviewed Mar 3, 2026

View reviewed changes

bhhalim added 2 commits March 3, 2026 12:03

revert changes to sagemaker-core

a1b1bc3

BassemHalim dismissed mollyheamazon’s stale review via 3cd4686 March 3, 2026 21:38

BassemHalim temporarily deployed to auto-approve March 3, 2026 21:38 — with GitHub Actions Inactive

bhhalim added 2 commits March 3, 2026 14:24

add build to test_requirements

39f1be2

add upper bounds for test dependencies

4797633

BassemHalim deployed to auto-approve March 3, 2026 22:31 — with GitHub Actions Active

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature processor v3#5565

Feature processor v3#5565
BassemHalim wants to merge 23 commits intoaws:masterfrom
BassemHalim:feature-processor-v3

BassemHalim commented Feb 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

nargokul Mar 3, 2026

Uh oh!

BassemHalim Mar 3, 2026

Uh oh!

nargokul Mar 3, 2026

Uh oh!

BassemHalim Mar 3, 2026

Uh oh!

Uh oh!

nargokul Mar 3, 2026

Uh oh!

BassemHalim Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

BassemHalim commented Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Port feature_processor module to SageMaker Python SDK v3

Changes

Diff with V2

Uh oh!

Uh oh!

nargokul Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

BassemHalim Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

nargokul Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

BassemHalim Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nargokul Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

BassemHalim Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

BassemHalim commented Feb 20, 2026 •

edited

Loading

Port `feature_processor` module to SageMaker Python SDK v3