Currently I observe some inconsistency for fp8 and int4.
- binary / unary / computation ops do not support either of them
- Some ops (flatten, reshape etc) support int4 but not fp8
- Allocation ops (full, arange) support both (well it's because we cast in the end)