mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Disable logits_all in llamacpp_HF (makes processing 3x faster)

Browse source
This commit is contained in:
oobabooga 2023-11-07 14:35:48 -08:00
parent 5c3eb22ce6
commit af3d25a503
5 changed files with 5 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
modules/ui.py
View file

@ -87,6 +87,7 @@ def list_model_elements():
'alpha_value',
'rope_freq_base',
'numa',
'logits_all',
]
if is_torch_xpu_available():
for i in range(torch.xpu.device_count()):