diff --git a/Tools/llm-tool/README.md b/Tools/llm-tool/README.md index 0efd0c9..259c6f0 100644 --- a/Tools/llm-tool/README.md +++ b/Tools/llm-tool/README.md @@ -43,3 +43,18 @@ The easiest way to do this is drag the Products/llm-tool into Terminal to get th DYLD_FRAMEWORK_PATH=~/Library/Developer/Xcode/DerivedData/mlx-examples-swift-ceuohnhzsownvsbbleukxoksddja/Build/Products/Debug ~/Library/Developer/Xcode/DerivedData/mlx-examples-swift-ceuohnhzsownvsbbleukxoksddja/Build/Products/Debug/llm-tool --prompt "swift programming language" ``` +### Troubleshooting + +If the program crashes with a very deep stack trace you may need to build +in Release configuration. This seems to depend on the size of the model. + +There are a couple options: + +- build Release +- force the model evaluation to run on the main thread, e.g. using @MainActor +- build `Cmlx` with optimizations by modifying `mlx/Package.swift` and adding `.unsafeOptions(["-O3"]),` around line 87 + +Building in Release / optimizations will remove a lot of tail calls in the C++ +layer. These lead to the stack overflows. + +See discussion here: https://github.com/ml-explore/mlx-swift-examples/issues/3