SFTTrainer.__init__() got an unexpected keyword argument 'tokenizer'

[h3clikejava]

[kmrasmussen]

Use the parameter processing_class=tokenizer instead of tokenizer=tokenizer

https://stackoverflow.com/questions/79546910/typeerror-in-sfttrainer-initialization-unexpected-keyword-argument-tokenizer

Maybe someone can do a fix

[jpodivin]

This should be easy enough to do. But it should go with pinning the library, so that we don't have more issues caused by unstable API.

[michael-haroon]

Use the parameter processing_class=tokenizer instead of tokenizer=tokenizer

https://stackoverflow.com/questions/79546910/typeerror-in-sfttrainer-initialization-unexpected-keyword-argument-tokenizer

Maybe someone can do a fix

Thank you!

[jpodivin]

Issues should only be closed when fixes are merged. Actually only if the solution has been tested, but this is not a production software. Still, while the #220 remains open, this ticket must as well.

[michael-haroon]

Oh sorry! Thought that was merged. Didn't realize it was an open pull request. I deleted my message.

[steveepreston]

Agree, to Solve SFTTrainer.__init__() got an unexpected keyword argument 'tokenizer', rename parameter:

SFTTrainer(
      ...
-     tokenizer=tokenizer
+     processing_class=tokenizer
)