Sample code is not functional

#2
by ecdouglas - opened

RuntimeError: The expanded size of the tensor (64) must match the existing size (2) at non-singleton dimension 0. Target sizes: [64]. Tensor sizes: [2]

Owner

RuntimeError: The expanded size of the tensor (64) must match the existing size (2) at non-singleton dimension 0. Target sizes: [64]. Tensor sizes: [2]

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline

tokenizer = AutoTokenizer.from_pretrained("ccdv/lsg-bart-base-4096-booksum", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ccdv/lsg-bart-base-4096-booksum", trust_remote_code=True)

text = "Replace by what you want."*512
pipe = pipeline("text2text-generation", model=model, tokenizer=tokenizer)
generated_text = pipe(
  text, 
  truncation=True, 
  max_length=256, 
  no_repeat_ngram_size=7,
  num_beams=2,
  early_stopping=True
  )

Works fine on colab (transformers 4.38.2)

Ah I wasn't actually using the sample code exactly, as I had added device="mps" to pipe = pipeline("text2text-generation", model=model, tokenizer=tokenizer, device="mps") on MacOS.

Unsure if the above is a model error or user error, as I'm new to transformers and still learning, but can confirm the above works with no device given.

Owner

Didn't test on mac

Sign up or log in to comment