0af2efe8a0
* switch swift-tokenizers to main, remove some workarounds - swift-tokenizers is getting a lot of updates and fixes, let's track main for now - remove some workarounds that are no longer needed - https://github.com/huggingface/swift-transformers/issues/63
Llama
This is a port of several models from:
using the Hugging Face swift transformers package to provide tokenization:
https://github.com/huggingface/swift-transformers
The Models.swift provides minor overrides and customization -- if you require overrides for the tokenizer or prompt customizations they can be added there.
This is set up to load models from Hugging Face, e.g. https://huggingface.co/mlx-community
The following models have been tried:
- mlx-community/Mistral-7B-v0.1-hf-4bit-mlx
- mlx-community/CodeLlama-13b-Instruct-hf-4bit-MLX
- mlx-community/phi-2-hf-4bit-mlx
- mlx-community/quantized-gemma-2b-it
Currently supported model types are:
- Llama / Mistral
- Gemma
- Phi
See Configuration.swift for more info.
See llm-tool