mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add AutoGPTQ wheels to requirements.txt

Browse source
This commit is contained in:
oobabooga 2023-06-02 00:47:11 -03:00
parent f344ccdddb
commit d0aca83b53
2 changed files with 4 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/GPTQ-models-(4-bit-mode).md
View file

@ -18,7 +18,7 @@ There are two ways of loading GPTQ models in the web UI at the moment:
* supports more models
* standardized (no need to guess any parameter)
* is a proper Python library
* no wheels are presently available so it requires manual compilation
* ~no wheels are presently available so it requires manual compilation~
* supports loading both triton and cuda models
For creating new quantizations, I recommend using AutoGPTQ: https://github.com/PanQiWei/AutoGPTQ
@ -175,7 +175,7 @@ python server.py --model llama-7b-4bit-128g --listen --lora tloen_alpaca-lora-7b
### Installation
To load a model quantized with AutoGPTQ in the web UI, you need to first manually install the AutoGPTQ library:
No additional steps are necessary as AutoGPTQ is already in the `requirements.txt` for the webui. If you still want or need to install it manually for whatever reason, these are the commands:
```
conda activate textgen