mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add /v1/internal/logits endpoint (#4650)

Browse source
This commit is contained in:
oobabooga 2023-11-18 23:19:31 -03:00 • committed by GitHub
parent 8f4f4daf8b
commit 0fa1af296c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 71 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

23
docs/12 - OpenAI API.md
View file

@ -97,6 +97,29 @@ curl http://127.0.0.1:5000/v1/chat/completions \
}'
```
#### Logits
```
curl -k http://127.0.0.1:5000/v1/internal/logits \
-H "Content-Type: application/json" \
-d '{
"prompt": "Who is best, Asuka or Rei? Answer:",
"use_samplers": false
}'
```
#### Logits after sampling parameters
```
curl -k http://127.0.0.1:5000/v1/internal/logits \
-H "Content-Type: application/json" \
-d '{
"prompt": "Who is best, Asuka or Rei? Answer:",
"use_samplers": true,
"top_k": 3
}'
```
#### Python chat example
```python