mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add rope_freq_base parameter for CodeLlama

Browse source
This commit is contained in:
oobabooga 2023-08-25 06:53:37 -07:00
parent feecd8190f
commit 52ab2a6b9e
10 changed files with 26 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -337,8 +337,9 @@ Optionally, you can use the following command-line flags:
| Flag | Description |
|------------------|-------------|
|`--alpha_value ALPHA_VALUE` | Positional embeddings alpha factor for NTK RoPE scaling. Use either this or compress_pos_emb, not both. |
|`--compress_pos_emb COMPRESS_POS_EMB` | Positional embeddings compression factor. Should typically be set to max_seq_len / 2048. |
| `--alpha_value ALPHA_VALUE` | Positional embeddings alpha factor for NTK RoPE scaling. Use either this or compress_pos_emb, not both.
| `--rope_freq_base ROPE_FREQ_BASE` | If greater than 0, will be used instead of alpha_value. Those two are related by rope_freq_base = 10000 * alpha_value ^ (64 / 63)
| `--compress_pos_emb COMPRESS_POS_EMB` | Positional embeddings compression factor. Should be set to (context length) / (model's original context length). Equal to 1/rope_freq_scale.
#### Gradio