Add quick-tune support for Attention #2169
Conversation
mirza-halilcevic
requested review from
dhernandez0,
dorde-antic and
umangyadav
mlir/include/mlir/Dialect/Rock/Tuning/QuickTuningPerfconfigs.inc
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
nit: own file for attention/g+g?
|
|
||
| #ifdef Attn_LOOKUP_TABLE_GEN | ||
|
|
||
| {"gfx900_attention_f32", {PopulateParamsAttn::initParametersAttentionGfx900, PopulateParamsAttn::nInitParametersAttentionGfx900}}, |
There was a problem hiding this comment.
I guess temporarily, gemm+gemm kernels will use attention quick tuning list but eventually, we'll have a list for gemm_gemm? (and potentially conv+gemm). We already have a tier1-gemmgemm list
There was a problem hiding this comment.
Yes, the ticket is in progress https://github.com/ROCm/rocMLIR-internal/issues/2019
- Retire InitParamsNonAccel and InitParamsAccel - Combine MfmaGemmParamsAttr and WmmaGemmParamsAttr into AccelGemmParamsAttr
| tuningParams); | ||
| } else if (isWmma) { | ||
| WmmaGemmParamsAttr wmmaParams = cast<WmmaGemmParamsAttr>(tuningParams); | ||
| AccelGemmParamsAttr wmmaParams = cast<AccelGemmParamsAttr>(tuningParams); |
There was a problem hiding this comment.
do we need to cast this? I understand tuningParams is already AccelGemmParamsAttr?
There was a problem hiding this comment.
You're right, we don't. This was the result of a find and replace so I missed it. Thanks.
There was a problem hiding this comment.
Actually, tuningParams is RockAccelTuningParamAttrInterface, so we are downcasting.
Motivation
Implement proper quick tuning support for Attention operations.
Technical Details
Resolves https://github.com/ROCm/rocMLIR-internal/issues/1887
Resolves https://github.com/ROCm/rocMLIR-internal/issues/1447
Test Plan
Test Result
Submission Checklist