Skip to content

[ET-VK][qconv] Enable im2col to handle grouped convolution#17793

Merged
meta-codesync[bot] merged 1 commit intogh/SS-JIA/454/basefrom
gh/SS-JIA/454/head
Mar 3, 2026
Merged

[ET-VK][qconv] Enable im2col to handle grouped convolution#17793
meta-codesync[bot] merged 1 commit intogh/SS-JIA/454/basefrom
gh/SS-JIA/454/head

Conversation

@SS-JIA
Copy link
Contributor

@SS-JIA SS-JIA commented Mar 2, 2026

Stack from ghstack (oldest at bottom):

Previously, the im2col + pointwise GEMM path (q8ta_conv2d_im2col) only
supported non-grouped convolutions (groups=1). This diff extends it to handle
grouped convolutions as well, providing significant speedups on Mali GPUs.

The key changes are:

PW GEMM shader (q8ta_conv2d_pw.glsl): Added K4_per_group and
OC4_per_group as push constants. The shader now computes a group index from
the output channel block (group_idx = oc_block_idx / OC4_per_group) and
offsets the im2col input read by group_idx * K4_per_group. For non-grouped
cases (groups=1), group_idx is always 0, so behavior is unchanged.

PW node (Q8taConv2dPW.cpp): add_q8ta_conv2d_pw_node now accepts a
groups parameter (default=1) and computes K4_per_group and OC4_per_group
internally from the input/output tensor dimensions. K4_per_group and
OC4_per_group were previously specialization constants; they are now push
constants to avoid shader variant explosion when groups varies.

Im2col node (Q8taConv2dIm2Col.cpp): Removed the groups == 1 assertion
from add_q8ta_im2col_node. The im2col shader already handles groups correctly
(each group's K range is contiguous in the output buffer). The q8ta_conv2d_im2col
operator now passes the groups value through to the PW node.

Dispatch heuristic (Q8taConv2d.cpp): Updated q8ta_conv2d with
device-aware dispatch. On Mali, im2col is used for all eligible cases (grouped
and ungrouped) since it provides 1.2-3.6x speedups. On Adreno, im2col is only
used for ungrouped convolutions (groups=1) where in_channels_per_group >= 32 or
spatial_out <= 4096, since grouped convolutions show 0.7-0.95x regression with
im2col. The heuristic uses graph.device_is_mali() to select the path.

Tests (test_q8ta_conv2d.cpp): Updated im2col test eligibility from
groups == 1 && channels.in % 4 == 0 to in_channels_per_group % 4 == 0,
enabling im2col testing for grouped cases. Added SceneX v9 256x256 grouped
convolution configs.

Differential Revision: D94949480

Previously, the im2col + pointwise GEMM path (`q8ta_conv2d_im2col`) only
supported non-grouped convolutions (groups=1). This diff extends it to handle
grouped convolutions as well, providing significant speedups on Mali GPUs.

The key changes are:

**PW GEMM shader (`q8ta_conv2d_pw.glsl`)**: Added `K4_per_group` and
`OC4_per_group` as push constants. The shader now computes a group index from
the output channel block (`group_idx = oc_block_idx / OC4_per_group`) and
offsets the im2col input read by `group_idx * K4_per_group`. For non-grouped
cases (groups=1), `group_idx` is always 0, so behavior is unchanged.

**PW node (`Q8taConv2dPW.cpp`)**: `add_q8ta_conv2d_pw_node` now accepts a
`groups` parameter (default=1) and computes `K4_per_group` and `OC4_per_group`
internally from the input/output tensor dimensions. `K4_per_group` and
`OC4_per_group` were previously specialization constants; they are now push
constants to avoid shader variant explosion when groups varies.

**Im2col node (`Q8taConv2dIm2Col.cpp`)**: Removed the `groups == 1` assertion
from `add_q8ta_im2col_node`. The im2col shader already handles groups correctly
(each group's K range is contiguous in the output buffer). The `q8ta_conv2d_im2col`
operator now passes the groups value through to the PW node.

**Dispatch heuristic (`Q8taConv2d.cpp`)**: Updated `q8ta_conv2d` with
device-aware dispatch. On Mali, im2col is used for all eligible cases (grouped
and ungrouped) since it provides 1.2-3.6x speedups. On Adreno, im2col is only
used for ungrouped convolutions (groups=1) where in_channels_per_group >= 32 or
spatial_out <= 4096, since grouped convolutions show 0.7-0.95x regression with
im2col. The heuristic uses `graph.device_is_mali()` to select the path.

**Tests (`test_q8ta_conv2d.cpp`)**: Updated im2col test eligibility from
`groups == 1 && channels.in % 4 == 0` to `in_channels_per_group % 4 == 0`,
enabling im2col testing for grouped cases. Added SceneX v9 256x256 grouped
convolution configs.

Differential Revision: [D94949480](https://our.internmc.facebook.com/intern/diff/D94949480/)

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 2, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17793

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures, 2 Unrelated Failures

As of commit 0979460 with merge base ae41854 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions
Copy link

github-actions bot commented Mar 2, 2026

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@meta-codesync meta-codesync bot merged commit e754d74 into gh/SS-JIA/454/base Mar 3, 2026
192 of 211 checks passed
@meta-codesync meta-codesync bot deleted the gh/SS-JIA/454/head branch March 3, 2026 08:28
@meta-codesync meta-codesync bot temporarily deployed to cherry-pick-bot March 3, 2026 08:28 Inactive
SS-JIA pushed a commit that referenced this pull request Mar 3, 2026
Previously, the im2col + pointwise GEMM path (`q8ta_conv2d_im2col`) only
supported non-grouped convolutions (groups=1). This diff extends it to handle
grouped convolutions as well, providing significant speedups on Mali GPUs.

The key changes are:

**PW GEMM shader (`q8ta_conv2d_pw.glsl`)**: Added `K4_per_group` and
`OC4_per_group` as push constants. The shader now computes a group index from
the output channel block (`group_idx = oc_block_idx / OC4_per_group`) and
offsets the im2col input read by `group_idx * K4_per_group`. For non-grouped
cases (groups=1), `group_idx` is always 0, so behavior is unchanged.

**PW node (`Q8taConv2dPW.cpp`)**: `add_q8ta_conv2d_pw_node` now accepts a
`groups` parameter (default=1) and computes `K4_per_group` and `OC4_per_group`
internally from the input/output tensor dimensions. `K4_per_group` and
`OC4_per_group` were previously specialization constants; they are now push
constants to avoid shader variant explosion when groups varies.

**Im2col node (`Q8taConv2dIm2Col.cpp`)**: Removed the `groups == 1` assertion
from `add_q8ta_im2col_node`. The im2col shader already handles groups correctly
(each group's K range is contiguous in the output buffer). The `q8ta_conv2d_im2col`
operator now passes the groups value through to the PW node.

**Dispatch heuristic (`Q8taConv2d.cpp`)**: Updated `q8ta_conv2d` with
device-aware dispatch. On Mali, im2col is used for all eligible cases (grouped
and ungrouped) since it provides 1.2-3.6x speedups. On Adreno, im2col is only
used for ungrouped convolutions (groups=1) where in_channels_per_group >= 32 or
spatial_out <= 4096, since grouped convolutions show 0.7-0.95x regression with
im2col. The heuristic uses `graph.device_is_mali()` to select the path.

**Tests (`test_q8ta_conv2d.cpp`)**: Updated im2col test eligibility from
`groups == 1 && channels.in % 4 == 0` to `in_channels_per_group % 4 == 0`,
enabling im2col testing for grouped cases. Added SceneX v9 256x256 grouped
convolution configs.

Differential Revision: [D94949480](https://our.internmc.facebook.com/intern/diff/D94949480/)

ghstack-source-id: 346525921
Pull Request resolved: #17793
SS-JIA pushed a commit that referenced this pull request Mar 3, 2026
Previously, the im2col + pointwise GEMM path (`q8ta_conv2d_im2col`) only
supported non-grouped convolutions (groups=1). This diff extends it to handle
grouped convolutions as well, providing significant speedups on Mali GPUs.

The key changes are:

**PW GEMM shader (`q8ta_conv2d_pw.glsl`)**: Added `K4_per_group` and
`OC4_per_group` as push constants. The shader now computes a group index from
the output channel block (`group_idx = oc_block_idx / OC4_per_group`) and
offsets the im2col input read by `group_idx * K4_per_group`. For non-grouped
cases (groups=1), `group_idx` is always 0, so behavior is unchanged.

**PW node (`Q8taConv2dPW.cpp`)**: `add_q8ta_conv2d_pw_node` now accepts a
`groups` parameter (default=1) and computes `K4_per_group` and `OC4_per_group`
internally from the input/output tensor dimensions. `K4_per_group` and
`OC4_per_group` were previously specialization constants; they are now push
constants to avoid shader variant explosion when groups varies.

**Im2col node (`Q8taConv2dIm2Col.cpp`)**: Removed the `groups == 1` assertion
from `add_q8ta_im2col_node`. The im2col shader already handles groups correctly
(each group's K range is contiguous in the output buffer). The `q8ta_conv2d_im2col`
operator now passes the groups value through to the PW node.

**Dispatch heuristic (`Q8taConv2d.cpp`)**: Updated `q8ta_conv2d` with
device-aware dispatch. On Mali, im2col is used for all eligible cases (grouped
and ungrouped) since it provides 1.2-3.6x speedups. On Adreno, im2col is only
used for ungrouped convolutions (groups=1) where in_channels_per_group >= 32 or
spatial_out <= 4096, since grouped convolutions show 0.7-0.95x regression with
im2col. The heuristic uses `graph.device_is_mali()` to select the path.

**Tests (`test_q8ta_conv2d.cpp`)**: Updated im2col test eligibility from
`groups == 1 && channels.in % 4 == 0` to `in_channels_per_group % 4 == 0`,
enabling im2col testing for grouped cases. Added SceneX v9 256x256 grouped
convolution configs.

Differential Revision: [D94949480](https://our.internmc.facebook.com/intern/diff/D94949480/)

ghstack-source-id: 346525921
Pull Request resolved: #17793
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants