You can run in a variety of ways, I went with llama.cpp,
If my notes are correct:
Clone and build
git clone https://github.com/ggerganov/llama.cpp cd llama.cpp mkdir build && cd build cmake .. -DLLAMA_AVX2=ON -DLLAMA_F16C=ON -DLLAMA_FMA=ON cmake --build . --config Release