Bug Report: SAM2 crashes with Unsupported number of channels: 512 and multiple attention warnings

[1wangjianchen]

I’m using SAM2 to predict masks, but it crashes with the error Unsupported number of channels: 512. Before the crash, there are multiple warnings about attention kernels not being used and dtype mismatches.

Full Log：

Predicting 1 masks
/root/autodl-tmp/venv/lib/python3.10/site-packages/torchvision/transforms/functional.py:154: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:203.)
img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
/root/autodl-tmp/venv/lib/python3.10/site-packages/sam2/modeling/sam/transformer.py:270: UserWarning: Memory efficient kernel not used because: (Triggered internally at /pytorch/aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:838.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/root/autodl-tmp/venv/lib/python3.10/site-packages/sam2/modeling/sam/transformer.py:270: UserWarning: Memory Efficient attention has been runtime disabled. (Triggered internally at /pytorch/aten/src/ATen/native/transformers/sdp_utils_cpp.h:548.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/root/autodl-tmp/venv/lib/python3.10/site-packages/sam2/modeling/sam/transformer.py:270: UserWarning: Flash attention kernel not used because: (Triggered internally at /pytorch/aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:840.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/root/autodl-tmp/venv/lib/python3.10/site-packages/sam2/modeling/sam/transformer.py:270: UserWarning: Expected query, key and value to all be of dtype: {Half, BFloat16}. Got Query dtype: float, Key dtype: float, and Value dtype: float instead. (Triggered internally at /pytorch/aten/src/ATen/native/transformers/sdp_utils_cpp.h:90.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/root/autodl-tmp/venv/lib/python3.10/site-packages/sam2/modeling/sam/transformer.py:270: UserWarning: CuDNN attention kernel not used because: (Triggered internally at /pytorch/aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:842.)
out = F.scaled_dot_product_attention(q, k, v, dropout_p=dropout_p)
/root/autodl-tmp/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1762: UserWarning: Flash Attention kernel failed due to: No available kernel. Aborting execution.
Falling back to all available kernels for scaled_dot_product_attention (which may have a slower speed).
return forward_call(*args, **kwargs)
Predicted 1 masks
Unsupported number of channels: 512