mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Bump llama-cpp-python

Browse source
This commit is contained in:
oobabooga 2023-08-17 21:39:26 -07:00
parent 3e7c624f8e
commit 6170b5ba31
7 changed files with 21 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -262,6 +262,7 @@ Optionally, you can use the following command-line flags:
| `--no-mmap` | Prevent mmap from being used. |
| `--mlock` | Force the system to keep the model in RAM. |
| `--cache-capacity CACHE_CAPACITY` | Maximum cache capacity. Examples: 2000MiB, 2GiB. When provided without units, bytes will be assumed. |
| `--tensor_split TENSOR_SPLIT` | Split the model across multiple GPUs, comma-separated list of proportions, e.g. 18,17 |
| `--llama_cpp_seed SEED` | Seed for llama-cpp models. Default 0 (random). |
| `--n_gqa N_GQA` | grouped-query attention. Must be 8 for llama-2 70b. |
| `--rms_norm_eps RMS_NORM_EPS` | 5e-6 is a good value for llama-2 models. |