mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add ExLlamaV2 and ExLlamav2_HF loaders (#3881)

Browse source
This commit is contained in:
oobabooga 2023-09-12 14:33:07 -03:00 • committed by GitHub
parent a821928877
commit c2a309f56e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 295 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

4
modules/shared.py
View file

@ -219,6 +219,10 @@ def fix_loader_name(name):
return 'ExLlama'
elif name in ['exllama-hf', 'exllama_hf', 'exllama hf', 'ex-llama-hf', 'ex_llama_hf']:
return 'ExLlama_HF'
elif name in ['exllamav2', 'exllama-v2', 'ex_llama-v2', 'exlamav2', 'exlama-v2']:
return 'ExLlamav2'
elif name in ['exllamav2-hf', 'exllamav2_hf', 'exllama-v2-hf', 'exllama_v2_hf', 'exllama-v2_hf']:
return 'ExLlamav2_HF'
elif name in ['ctransformers', 'ctranforemrs', 'ctransformer']:
return 'ctransformers'