mmarch/text-generation-webui
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

substitu superboog Beatiful Soup Parser (#2996)

Browse source 
* add lxml to requirments

add lxml to requirments

* Change Beaitful Soup Parser

"lxml" parser which might be more tolerant of certain kinds of parsing errors than "html.parser" and quicker at the same time.
This commit is contained in:
Juliano Henriquez 2023-07-11 18:02:49 -04:00 • committed by GitHub
parent ab044a5a44
commit 1fc0b5041e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
extensions/superbooga/script.py
View file

@ -69,7 +69,7 @@ def feed_url_into_collector(urls, chunk_len, chunk_sep, strong_cleanup, threads)
cumulative += 'Processing the HTML sources...'
yield cumulative
for content in contents:
soup = BeautifulSoup(content, features="html.parser")
soup = BeautifulSoup(content, features="lxml")
for script in soup(["script", "style"]):
script.extract()