DeepAR InternalServerError: We encountered an internal error. Please try again

[chaitanyasathi]

Here is the error I am getting

Error for Transform job forecasting-deepar-2020-07-28-12-59-06--2020-07-28-12-59-07-635: Failed. Reason: InternalServerError: We encountered an internal error. Please try again.

My DeepAR Batch Transformer jobs are working fine for small subset of the data. But however for larger set of data, I am getting above error. This has really been a show stopper not sure how to proceed further. I also tried with different combination of instance count, instance type and it is still failing.

Here is the transformer code I am using

deepar_transformer = estimator.transformer(instance_count = 4,
                                  instance_type = 'ml.m4.4xlarge',strategy='MultiRecord', output_path=batch_output , max_payload=100)
                                  
deepar_transformer.transform(data=data_channels["test"]) 

deepar_transformer.wait()

[laurenyu]

I've passed this along to the team that owns DeepAR to see if they have any insight (reference: P38158491). Thanks for using SageMaker!

[chaitanyasathi]

@laurenyu Any updates on the following issue?