Add llama.cpp GPU offload option (#2060)

docs/llama.cpp-models.md

@@ -1,16 +1,31 @@
-## Using llama.cpp in the web UI
+# Using llama.cpp in the web UI
 
-#### Pre-converted models
+## Setting up the models
+
+#### Pre-converted
 
 Place the model in the `models` folder, making sure that its name contains `ggml` somewhere and ends in `.bin`.
 
 #### Convert LLaMA yourself
 
-Follow the instructions in the llama.cpp README to generate the `ggml-model-q4_0.bin` file: https://github.com/ggerganov/llama.cpp#usage
+Follow the instructions in the llama.cpp README to generate the `ggml-model.bin` file: https://github.com/ggerganov/llama.cpp#usage
+
+## GPU offloading
+
+Enabled with the `--n-gpu-layers` parameter. If you have enough VRAM, use a high number like `--n-gpu-layers 200000` to offload all layers to the GPU.
+
+Note that you need to manually install `llama-cpp-python` with GPU support. To do that:
+
+```
+pip uninstall -y llama-cpp-python
+CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python --no-cache-dir
+```
+
+Here you can find the different compilation options for OpenBLAS / cuBLAS / CLBlast: https://pypi.org/project/llama-cpp-python/
 
 ## Performance
 
-This was the performance of llama-7b int4 on my i5-12400F:
+This was the performance of llama-7b int4 on my i5-12400F (cpu only):
 
 > Output generated in 33.07 seconds (6.05 tokens/s, 200 tokens, context 17)