Improved quant
#2
by
distantquant
- opened
Here is a most likely improved quant that rotates the shapes better: https://huggingface.co/152334H/miqu-1-70b-sf
Why is this only 48Gb or less.
If it was full upscaled to full float16 wouldn't it be 140Gb?