Improved CUDA device selection (computer capability >= 700, pick card if several)

[coezbek]

When running Zonos with CPU on Laptops/PCs with old CUDA hardware (before RTX 2000) then Zonos will not work. The patch only enables CUDA if compute capabilities are 7.0 or higher.

[Ph0rk0z]

Did anyone test on P40 and P100 to push such a change? Mobile GPU are in general not suited to this. Plus you can get rid of BF16 use.

[coezbek]

Improved the patch to include support for picking either the faster GPU or the one with more VRAM.

[tjameswilliams]

This is a good idea, but for my use case, I would not want it to be the default. For instance I feel like Zonos will be used frequently alongside an LLM, in which case you want the LLM to take the faster / larger compute GPU by default.

[coezbek]

I have added the two options because these two options for me allow me to select my two cards (faster 4060, more RAM 3090). But you are right, that smallest and slowest also should be options to select. I guess you can always just override with "cuda:0" and "cuda:1".

[Ph0rk0z]

Have you ever heard of CUDA_VISIBLE_DEVICES?

[coezbek]

I have not until now.

https://stackoverflow.com/questions/39649102/how-do-i-select-which-gpu-to-run-a-job-on

Thanks for the pointer.