Extensions performance & memory optimisations

Reworked remove_surrounded_chars() to use regular expression ( https://regexr.com/7alb5 ) instead of repeated string concatenations for elevenlab_tts, silero_tts, sd_api_pictures. This should be both faster and more robust in handling asterisks.

Reduced the memory footprint of send_pictures and sd_api_pictures by scaling the images in the chat to 300 pixels max-side wise. (The user already has the original in case of the sent picture and there's an option to save the SD generation).
This should fix history growing annoyingly large with multiple pictures present
This commit is contained in:
Φφ 2023-03-22 07:47:54 +03:00
parent 45b7e53565
commit 5389fce8e1
4 changed files with 39 additions and 27 deletions

View file

@ -4,6 +4,7 @@ from io import BytesIO
import gradio as gr
import torch
from transformers import BlipForConditionalGeneration, BlipProcessor
from PIL import Image
import modules.chat as chat
import modules.shared as shared
@ -25,10 +26,20 @@ def caption_image(raw_image):
def generate_chat_picture(picture, name1, name2):
text = f'*{name1} sends {name2} a picture that contains the following: "{caption_image(picture)}"*'
# lower the resolution of sent images for the chat, otherwise the log size gets out of control quickly with all the base64 values in visible history
width, height = picture.size
if (width > 300):
height = int(height * (300 / width))
width = 300
elif (height > 300):
width = int(width * (300 / height))
height = 300
newsize = (width, height)
picture = picture.resize(newsize, Image.LANCZOS)
buffer = BytesIO()
picture.save(buffer, format="JPEG")
img_str = base64.b64encode(buffer.getvalue()).decode('utf-8')
visible_text = f'<img src="data:image/jpeg;base64,{img_str}">'
visible_text = f'<img src="data:image/jpeg;base64,{img_str}" alt="{text}">'
return text, visible_text
def ui():