mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update README.md

Browse source
This commit is contained in:
oobabooga 2023-05-25 01:23:46 -03:00 • committed by GitHub
parent 361451ba60
commit a04266161d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -101,7 +101,7 @@ pip install -r requirements.txt
The base installation covers [transformers](https://github.com/huggingface/transformers) models (`AutoModelForCausalLM` and `AutoModelForSeq2SeqLM` specifically) and [llama.cpp](https://github.com/ggerganov/llama.cpp) (GGML) models. The base installation covers [transformers](https://github.com/huggingface/transformers) models (`AutoModelForCausalLM` and `AutoModelForSeq2SeqLM` specifically) and [llama.cpp](https://github.com/ggerganov/llama.cpp) (GGML) models.
To use 4-bit GPU models, the additional installation steps below are necessary: To use GPTQ models, the additional installation steps below are necessary:
[GPTQ models (4 bit mode)](https://github.com/oobabooga/text-generation-webui/blob/main/docs/GPTQ-models-(4-bit-mode).md) [GPTQ models (4 bit mode)](https://github.com/oobabooga/text-generation-webui/blob/main/docs/GPTQ-models-(4-bit-mode).md)
@ -223,6 +223,8 @@ Optionally, you can use the following command-line flags:
#### Accelerate 4-bit #### Accelerate 4-bit
⚠️ Not supported on Windows at the moment.
| Flag | Description | | Flag | Description |
|---------------------------------------------|-------------| |---------------------------------------------|-------------|
| `--load-in-4bit` | Load the model with 4-bit precision (using bitsandbytes). | | `--load-in-4bit` | Load the model with 4-bit precision (using bitsandbytes). |