mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Remove GGML support

Browse source
This commit is contained in:
oobabooga 2023-09-11 07:30:56 -07:00
parent cc7b7ba153
commit ed86878f02
15 changed files with 24 additions and 123 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/llama.cpp.md
View file

@ -9,16 +9,14 @@ llama.cpp is the best backend in two important scenarios:
#### Pre-converted
Download the GGUF or GGML models directly into your `text-generation-webui/models` folder. It will be a single file.
Download the GGUF models directly into your `text-generation-webui/models` folder. It will be a single file.
* For GGUF models, make sure its name contains `.gguf`.
* For GGML models, make sure its name contains `ggml` and ends in `.bin`.
`q4_K_M` quantization is recommended.
* Make sure its name ends in `.gguf`.
* `q4_K_M` quantization is recommended.
#### Convert Llama yourself
Follow the instructions in the llama.cpp README to generate a ggml: https://github.com/ggerganov/llama.cpp#prepare-data--run
Follow the instructions in the llama.cpp README to generate a GGUF: https://github.com/ggerganov/llama.cpp#prepare-data--run
## GPU acceleration