mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add a proper documentation (#3885)

Browse source
This commit is contained in:
oobabooga 2023-10-21 19:15:54 -03:00 • committed by GitHub
parent 5a5bc135e9
commit 6efb990b60
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
30 changed files with 707 additions and 932 deletions
Download patch file Download diff file Expand all files Collapse all files

23
docs/What Works.md Normal file
View file

@ -0,0 +1,23 @@
## What Works
| Loader | Loading 1 LoRA | Loading 2 or more LoRAs | Training LoRAs | Multimodal extension | Perplexity evaluation |
|----------------|----------------|-------------------------|----------------|----------------------|-----------------------|
| Transformers | ✅ | ❌ | ✅* | ✅ | ✅ |
| ExLlama_HF | ✅ | ❌ | ❌ | ❌ | ✅ |
| ExLlamav2_HF | ✅ | ✅ | ❌ | ❌ | ✅ |
| ExLlama | ✅ | ❌ | ❌ | ❌ | use ExLlama_HF |
| ExLlamav2 | ✅ | ✅ | ❌ | ❌ | use ExLlamav2_HF |
| AutoGPTQ | ✅ | ❌ | ❌ | ✅ | ✅ |
| GPTQ-for-LLaMa | ✅** | ❌ | ✅ | ✅ | ✅ |
| llama.cpp | ❌ | ❌ | ❌ | ❌ | use llamacpp_HF |
| llamacpp_HF | ❌ | ❌ | ❌ | ❌ | ✅ |
| ctransformers | ❌ | ❌ | ❌ | ❌ | ❌ |
| AutoAWQ | ? | ❌ | ? | ? | ✅ |
❌ = not implemented
✅ = implemented
\* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.
\*\* Requires the monkey-patch. The instructions can be found [here](https://github.com/oobabooga/text-generation-webui/wiki/08-%E2%80%90-Additional-Tips#using-loras-with-gptq-for-llama).