mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Recommend ExLlama_HF and ExLlamav2_HF

Browse source
This commit is contained in:
oobabooga 2023-12-04 15:28:46 -08:00
parent 3f993280e4
commit 7fc9033b2e
2 changed files with 5 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

2
modules/loaders.py
View file

@ -71,7 +71,6 @@ loaders_and_params = OrderedDict({
'compress_pos_emb',
'cfg_cache',
'no_use_fast',
'exllama_HF_info',
],
'ExLlamav2_HF': [
'gpu_split',
@ -133,6 +132,7 @@ loaders_and_params = OrderedDict({
'cache_8bit',
'alpha_value',
'compress_pos_emb',
'exllamav2_info',
],
'ctransformers': [
'n_ctx',