zhangshuai/mlx-swift-examples
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add notes about building Release

Browse Source
This commit is contained in:
David Koski
2024-02-26 10:34:59 -08:00
parent 5a83d7d92b
commit 8870b0d386
1 changed files with 15 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
Tools/llm-tool/README.md
View File

@@ -43,3 +43,18 @@ The easiest way to do this is drag the Products/llm-tool into Terminal to get th
DYLD_FRAMEWORK_PATH=~/Library/Developer/Xcode/DerivedData/mlx-examples-swift-ceuohnhzsownvsbbleukxoksddja/Build/Products/Debug ~/Library/Developer/Xcode/DerivedData/mlx-examples-swift-ceuohnhzsownvsbbleukxoksddja/Build/Products/Debug/llm-tool --prompt "swift programming language"
```
### Troubleshooting
If the program crashes with a very deep stack trace you may need to build
in Release configuration. This seems to depend on the size of the model.
There are a couple options:
- build Release
- force the model evaluation to run on the main thread, e.g. using @MainActor
- build `Cmlx` with optimizations by modifying `mlx/Package.swift` and adding `.unsafeOptions(["-O3"]),` around line 87
Building in Release / optimizations will remove a lot of tail calls in the C++
layer. These lead to the stack overflows.
See discussion here: https://github.com/ml-explore/mlx-swift-examples/issues/3