mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Lint the openai extension

Browse source
This commit is contained in:
oobabooga 2023-09-15 20:11:16 -07:00
parent 760510db52
commit 8f97e87cac
12 changed files with 79 additions and 69 deletions
Download patch file Download diff file Expand all files Collapse all files

5
extensions/openai/tokens.py
View file

@ -1,6 +1,5 @@
from extensions.openai.utils import float_list_to_base64
from modules.text_generation import encode, decode
import numpy as np
from modules.text_generation import decode, encode
def token_count(prompt):
tokens = encode(prompt)[0]