mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Jinja templates for Instruct and Chat (#4874)

Browse source
This commit is contained in:
oobabooga 2023-12-12 17:23:14 -03:00 • committed by GitHub
parent aab0dd962d
commit 39d2fe1ed9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
71 changed files with 1774 additions and 518 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/01 - Chat Tab.md
View file

@ -97,10 +97,7 @@ The "Chat" option should typically be used only for base models or non-instruct
Used for talking to an instruction-following model using the prompt format defined under "Parameters" > "Instruction template". Think of this option as an offline ChatGPT.
The prompt format is defined by the following adjustable parameters in "Parameters" > "Instruction template":
* **Context**: appears at the top of the prompt exactly as it is written, including the newline characters at the end (if any). Often the context includes a customizable system message. For instance, instead of "Answer the questions." for Llama-2-chat, you can write "Answer the questions as if you were a pirate.", and the model will comply.
* **Turn template**: defines a single input/reply turn. In this string, `<|user|>` and `<|bot|>` are placeholders that get replaced with whatever you type in the **User string** and **Bot string** fields respectively; they are mandatory and should be present even if those fields are empty. `<|user-message|>` and `<|bot-message|>` get replaced with the user and bot messages at that turn. If the prompt format uses newline characters, they should be written inline as `\n` in the turn template.
The prompt format is defined by the **Instruction template string** parameter in "Parameters" > "Instruction template", which represents a A Jinja2 template.
Note that when you load a model in the "Model" tab, the web UI will try to automatically detect its instruction template (if any), and will update the values under "Parameters" > "Instruction template" accordingly. This is done using a set of regular expressions defined in `models/config.yaml`. This detection is not guaranteed to be accurate. You should check the model card on Hugging Face to see if you are using the correct prompt format.

8
docs/03 - Parameters Tab.md
View file

@ -101,13 +101,9 @@ So you can use those special placeholders in your character definitions. They ar
Defines the instruction template that is used in the Chat tab when "instruct" or "chat-instruct" are selected under "Mode".
* **Instruction template**: A dropdown menu where you can select from saved templates, save a new template (💾 button), and delete the currently selected template (🗑️).
* **Instruction template**: A dropdown menu where you can load a saved template, save a new template (💾 button), and delete the currently selected template (🗑️).
* **Custom system message**: A message that defines the personality of the chatbot, replacing its default "System message" string. Example: "You are a duck."
* **Turn template**: Defines the positioning of spaces and new line characters in a single turn of the dialogue. `<|user-message|>` gets replaced with the user input, `<|bot-message|>` gets replaced with the bot reply, `<|user|>` gets replaced with the "User string" below, and `<|bot|>` gets replaced with "Bot string" below. The `<|user|>` and `<|bot|>` placeholders must be included even if "User string" and "Bot string" are empty, as they are used to split the template in parts in the backend.
* **User string**: Replaces `<|user|>` in the turn template.
* **Bot string**: Replaces `<|bot|>` in the turn template.
* **Context**: A string that appears as-is at the top of the prompt, including the new line characters at the end (if any). The `<|system-message|>` placeholder gets replaced with the "System message" string below, unless "Custom system message" is not empty, in which case it is used instead.
* **System message**: A default message recommended by the model creator(s) to define the personality of the chatbot.
* **Instruction template string**: A Jinja2 template that defines the prompt format for the instruction-following conversation.
* **Send to default**: Send the full instruction template in string format to the Default tab.
* **Send to notebook**: Send the full instruction template in string format to the Notebook tab.
* **Send to negative prompt**: Send the full instruction template in string format to the "Negative prompt" field under "Parameters" > "Generation".