ehartford TheBloke commited on
Commit
278bf3c
1 Parent(s): 3534761

Upload fast tokenizer, which is the recommended and default for transformers now (#1)

Browse files

- Upload fast tokenizer, which is the recommended and default for transformers now (4245ca6a111b18e9d95cc3b1317a483c53927921)


Co-authored-by: Tom Jobbins <[email protected]>

Files changed (1) hide show
  1. tokenizer.json +0 -0
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff