Enable RWX support for raw block volumes

[Joffref]

Hello everyone, this is two questions in one.

I've gone through the SPDK-CSI design document today, and I've read that support for raw block volume is present in the to-do tasks. Is there any plan for adding this inside spdk-csi?

Plus, if this is planned, do you plan to support RWX?

Kudos who those who'll answer :)

[cyb70289]

hi @Joffref , AFAIK, there's no plan to implement raw block volume recently.

[Joffref]

Hi @cyb70289, thanks for answering. Do you think it is possible to implement this?

[cyb70289]

Yes. It's possible, and I believe it's useful.

[Joffref]

I'm happy to hear that, don't you mind if I keep this issue open to keep tracking further investigations on this subject?

[cyb70289]

Sure. We can keep this issue open.

[dong-liuliu]

hi, @Joffref
I've writen a patch for RWO raw block volume enablement before. Push it here.
But it was based on a commit in May 2023. And it couldn't be applied to latest master commit automatically.

Another note is that RWX support for raw block volume is something bitter both from the perspectives of implementation and usage.
So I'm curious what's the workload? or how will you use RWX raw block volume?

[Joffref]

Hi @dong-liuliu!
Thanks for pointing this out, do you know if in the future conflicts will be resolved and the patch will be pushed on the upstream?

The use case is quite simple, we're using kube-virt solution, and live migration of VM is doable only with RWX Volume. At the moment we can use an NFS Provisioner on top of our storage to make volumes RWX, but as a matter of performance, this is not good as we'd like it to be.

[dong-liuliu]

Hah, @Joffref
I've met the same situation with KubeVirt before. It's another block based CSI driver OpenEBS and it claimed that if user wants to use RWX volume, then NFS should be built on top of block volume.
But for live migration, it is a RWX PV but won't be accessed at the same time.
I've considered that KubeVirt could simplify its live-migration RWX requirment, because it works more like that one RWO PV, first claimed by source VMI, then claimed by target VMI.

Yeah, it is much easier for CSI to enable RWX block volume, since in other words, the uplevel app -- KubeVirt will coordinate multiple access. CSI drivers don't need to consider cosistency. So I think it's OK to add RWX on raw block volume.

[dong-liuliu]

Hi @dong-liuliu! Thanks for pointing this out, do you know if in the future conflicts will be resolved and the patch will be pushed on the upstream?

Sorry, I did it for an performance analysis requirment. Just checked master commit, it already changed a lot on related functions/structures. Not easy to resolve the conflict quickly.
Just like Yibo said, still need to keep the issue open for anyone who interested to solve...

[xinydev]

Yeah, it is much easier for CSI to enable RWX block volume, since in other words, the uplevel app -- KubeVirt will coordinate multiple access. CSI drivers don't need to consider cosistency. So I think it's OK to add RWX on raw block volume.

If we add RWX, it's fine for this KubeVirt use case. But for other use cases, there should be conflicts when multiple pods are writing at the same time.
If we can't guarantee that all use cases will support RWX, should we still consider including RWX options?