Multi-platform builds

[danehans]

When building images, create a single image for each binary that can run on multiple platforms, such as linux/amd64, linux/arm64, etc.

[danehans]

The same should be done for the vLLM image. Currently https://gallery.ecr.aws/q9t5s3a7 contains GPU and CPU variants but not arm64.

[adarshagrawal38]

Hi @danehans, As part of the fix, I’ll be adding, additional platforms here, to address the issue.

Are there any other changes you'd recommend?

Also, do I need to update the ARCH value in the Dockerfile based on the target platform here.

[danehans]

@adarshagrawal38, thank you for helping with this issue. Based on your comments above, you are on the right track. This doc guides multi-platform builds with a good Go-based example here.

[chewong]

@adarshagrawal38 are you still working on this? We’d like to deploy to arm64 nodes for testing but are blocked by this. I could take a stab at it if you’re not actively working on it.

[adarshagrawal38]

Hi @chewong, @danehans , I have raised the PR Please review.

[danehans]

Reopened since #1010 caused CI to no longer build EPP image from main and was reverted. Not that #1286 was also intended to fix the EPP image build in CI but did not work and had to be reverted too.

[ahg-g]

GB200 machines will only have ARM and some setups may not have other machine types to run the epp image; this is important to address I believe.