Multi Model Endpoint Deployment Issue

[nmakarun]

Describe the bug
When trying to deploy the multi model endpoint, it is not being deployed due to internal error.

To reproduce
print('Waiting for {} endpoint to be in service...'.format(endpoint_name))
waiter = sm_client.get_waiter('endpoint_in_service')
waiter.wait(EndpointName=endpoint_name)

Expected behavior
I expected the endpoint to be deployed and available for service .

Screenshots or logs

Additional context*
Add any other context about the problem here.

[preshen-goobiah]

Receiving the same error when trying to deploy multiple models locally.

I used this example

from sagemaker import multidatamodel

mdm = multidatamodel.MultiDataModel(name="MultipleModelsLocally",
                                    model_data_prefix="s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/multi_model_artifacts/", 
                                    image="<account_id>.dkr.ecr.eu-west-1.amazonaws.com/multi-model-sklearn:latest",
                                    role="arn:aws:iam::<account_id>:role/service-role/AmazonSageMaker-ExecutionRole-20190401T141434"
                                   )

predictor = mdm.deploy(initial_instance_count=1,instance_type='local', endpoint_name="MDMModel")

UPDATE: I was able to deploy locally by including the require env variables for the model

from sagemaker import multidatamodel

mdm = multidatamodel.MultiDataModel(name="MultipleModelsLocally2",
                                    model_data_prefix="s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/multi_model_artifacts/", 
                                    image="<account_id>.dkr.ecr.eu-west-1.amazonaws.com/multi-model-sklearn:latest",
                                    role="arn:aws:iam::<account_id>:role/service-role/AmazonSageMaker-ExecutionRole-20190401T141434",
 env={
            'SAGEMAKER_PROGRAM' : "script.py",
            'SAGEMAKER_SUBMIT_DIRECTORY' : "s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/inference-code/user_code.tar.gz"
        }
                                   )

predictor = mdm.deploy(initial_instance_count=1,instance_type='local', endpoint_name="MDMModel2")

[dlragha]

Thanks for reaching out! You are correct, if building an BYO image with SageMaker Scikit-learn image (SKLearn) as the base, the inference container expects SAGEMAKER_PROGRAM and SAGEMAKER_SUBMIT_DIRECTORY environment variables to be set explicitly while creating the model object. Creating an SKLearn estimator and deploying the model using the SageMaker Python SDK as seen in the example below sets these env variables automatically without having to pass them in explicitly.

AWS Managed Scikit-learn now provides native support for launching multi-model endpoints. Please see the updated example for usage.