Evaluation results?
#2
by
LuckiestOne
- opened
Thanks a lot for quantizing with an imatrix!
I would be very interested in any evaluations regarding the performance of the smaller models to observe the impact of the imatrix in this use case.
Did you evaluate the model sizes and if so, would you consider adding this to the model card?
Since this is a highly specialized model the best route to go for any kind of meaningful score would be to port the Berkeley Function-Calling Leaderboard to llama-cpp-python and see how each quant scores against the original weights.
Unless someone else wants to do that I might look into it once issues with IQ1_S are resolved and I've requantized it.