3b, 8b, and 34b versions of GGUF?

#1
by tombenninger - opened

Are currently I see the tensor data version of Granite for 3b, 8b, 20b, and 34b, but only a 20b version packaged in GGUF. Will the other size models be packaged in GGUF?

IBM Granite org

Ill be releasing 34b today abd imthe instruct of 20b
Some folks are working on making the 3 and 8b work, more details here: https://github.com/ggerganov/llama.cpp/issues/7116#issuecomment-2100061526

mayank-mishra changed discussion status to closed

Sign up or log in to comment