mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Make llama.cpp use tfs parameter

Browse source
This commit is contained in:
oobabooga 2023-06-17 19:08:25 -03:00
parent e19cbea719
commit 05a743d6ad
2 changed files with 12 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

11
docs/Generation-parameters.md
View file

@ -10,10 +10,21 @@ llama.cpp only uses the following parameters:
* top_p * top_p
* top_k * top_k
* repetition_penalty * repetition_penalty
* tfs
* mirostat_mode * mirostat_mode
* mirostat_tau * mirostat_tau
* mirostat_eta * mirostat_eta
### ExLlama
ExLlama only uses the following parameters:
* temperature
* top_p
* top_k
* repetition_penalty
* typical_p
### RWKV ### RWKV
RWKV only uses the following parameters when loaded through the old .pth weights: RWKV only uses the following parameters when loaded through the old .pth weights:

1
modules/llamacpp_model.py
View file

@ -68,6 +68,7 @@ class LlamaCppModel:
top_p=state['top_p'], top_p=state['top_p'],
top_k=state['top_k'], top_k=state['top_k'],
repeat_penalty=state['repetition_penalty'], repeat_penalty=state['repetition_penalty'],
tfs_z=state['tfs'],
mirostat_mode=int(state['mirostat_mode']), mirostat_mode=int(state['mirostat_mode']),
mirostat_tau=state['mirostat_tau'], mirostat_tau=state['mirostat_tau'],
mirostat_eta=state['mirostat_eta'], mirostat_eta=state['mirostat_eta'],