Output limitation
Hello! Me and my Uni colleagues are developing a chatbot for our Uni using this llm. The problem that we find is that the ouput has a chars limitation. We found that the LLM doesnt give answers with more than 64 chars. We already checked on the model but we dont find where is that limitation (codewise). Any way to change that?
Could we still use it this way and train it to give answers with more than 64 chars?
Obrigado! :)
Hi.
BERT is not a LLM. It Is a encoder LM. It means it is limited in terms of tokens number both in its input and output (512 tokens, https://jalammar.github.io/illustrated-bert/).
If you want to get outputs up to x tokens (x <= 512 tokens) , you must finetune BERT with a dataset that includes outputs up to x tokens (https://discuss.huggingface.co/t/squad-bert-why-max-length-384-by-default-and-not-512/11693).