mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Autodetect rms_norm_eps and n_gqa for llama-2-70b

Browse source
This commit is contained in:
oobabooga 2023-07-24 15:26:29 -07:00
parent a07d070b6c
commit 08c622df2e
2 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
models/config.yaml
View file

@ -277,3 +277,6 @@ TheBloke_WizardLM-30B-GPTQ:
.*llama-(2|v2).*chat:
mode: 'instruct'
instruction_template: 'Llama-v2'
.*llama.*70b.*ggml.*\.bin:
n_gqa: 8
rms_norm_eps: 1.0e-5