Add AMD and Intel GPU/NPU Support

[korgano]

Problem Statement:

• Many users possess AMD or Intel hardware without NVIDIA GPUs.
• Current ONNX Runtime acceleration is limited to CUDA/DirectML.
• Lack of AMD/Intel support forces fallback to CPU inference—higher latency and lower throughput.

---

Proposal:

1. Integrate onnxruntime-rocm

   • Enable ROCm EP for AMD GPUs via ONNX Runtime
   • Optionally include MIGraphX EP for deep-graph optimization (ONNX Runtime

2. Integrate onnxruntime-openvino

   • Leverage OpenVINO EP to offload inference to Intel CPUs, integrated/discrete GPUs, and NPUs (ONNX Runtime, OpenVINO Documentation)

3. Update build scripts & documentation

   • Add EP-specific install instructions (pip wheels, environment setup)
   • Provide sample configs for AMD and Intel pipelines

---

Benefits:

• Reduced Latency

  • Hardware acceleration on AMD GPUs (ROCm) and Intel NPUs (OpenVINO) cuts inference time by 2–10× compared to CPU only.

• Broader Hardware Coverage

  • Supports a wider user base: gaming PCs (AMD/Intel), AI PCs with AMD Strix Point/Gorgon Point/Strix Halo APUs/NPUs and Intel Core Ultra NPUs.

---

References:

• ROCm Execution Provider (AMD) — ONNX Runtime
• MIGraphX Execution Provider (AMD) — ONNX Runtime
• OpenVINO Execution Provider (Intel) — ONNX Runtime
• Intel NPU Support — OpenVINO Documentation

---

Please consider this enhancement to unlock the full potential of local hardware acceleration across AMD and Intel platforms!

[raymondlo84]

:) Is it ready yet :)?

[mckaysalisbury]

Yes. I've got an NPU, and I'd love to be able to use it for better performance in Foundry.

[mbar63]

I am interested in this feature too, I use a Lenovo ThinkPad X1 2-in-1 Gen 9 featuring an AI-accelerating Neural Processing Unit (NPU) as part of its Intel Core Ultra processor that I cannot use.

[desirade-fm]

For NPU Intel you must install this driver before : https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html
I have a Intel(R) Core(TM) Ultra 7 155U and NPU is working with Foundry Local.