SGLang int8 kernels #2196
                  
                    
                      vadimkantorov
                    
                  
                
                  started this conversation in
                General
              
            Replies: 0 comments
  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment
  
        
    
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I wonder if these Triton kernels are any relevant for wider torchao / pytorch usage (and if this Triton impl is also any portable for CPU):
and if not - I wonder why sglang does not use the quant triton kernels/bindings from ao?
(Also similar question on liger / unsloth kernels - including the notorious rmsnorm kernels - any plans to upstream their main components like linear + chunked cross entropy some place upstream?)
Beta Was this translation helpful? Give feedback.
All reactions