mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Improved instructions for AMD/Metal/Intel Arc/CPUs without AVCX2

Browse source
This commit is contained in:
oobabooga 2023-08-30 09:37:53 -07:00
parent b2f7ca0d18
commit bc4023230b
2 changed files with 42 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

26
README.md
View file

@ -75,11 +75,6 @@ conda activate textgen
The up-to-date commands can be found here: https://pytorch.org/get-started/locally/.
#### 2.1 Additional information
* MacOS users: https://github.com/oobabooga/text-generation-webui/pull/393
* AMD users: https://rentry.org/eq3hg
#### 3. Install the web UI
```
@ -88,17 +83,26 @@ cd text-generation-webui
pip install -r requirements.txt
```
#### llama.cpp on AMD, Metal, and some specific CPUs
#### AMD, Metal, Intel Arc, and CPUs without AVCX2
Precompiled wheels are included for CPU-only and NVIDIA GPUs (cuBLAS). For AMD, Metal, and some specific CPUs, you need to uninstall those wheels and compile llama-cpp-python yourself.
To uninstall:
1) Replace the last command above with
```
pip uninstall -y llama-cpp-python llama-cpp-python-cuda
pip install -r requirements_nocuda.txt
```
To compile: https://github.com/abetlen/llama-cpp-python#installation-with-openblas--cublas--clblast--metal
2) Manually install llama-cpp-python using the appropriate command for your hardware: [Installation from PyPI](https://github.com/abetlen/llama-cpp-python#installation-from-pypi).
3) AMD: Manually install AutoGPTQ: [Installation](https://github.com/PanQiWei/AutoGPTQ#installation).
4) AMD: Manually install ExLlama by simply cloning it into the `repositories` folder (it will be automatically compiled at runtime after that):
```
cd text-generation-webui
mkdir repositories
cd repositories
git clone https://github.com/turboderp/exllama
```
#### bitsandbytes on older NVIDIA GPUs