Pedro Cuenca 6c270a9d12 Remove Tokenizer wrapper (#28) ...

* Remove Tokenizer wrapper

Depends on https://github.com/huggingface/swift-transformers/pull/70

* Update swift-transformers revision

2024-03-18 09:07:52 -07:00

7.5 KiB

Raw Blame History

View Raw