Skip to content

harden: disable external XML entity processing in benchmarks.py... #415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
anupamme wants to merge 1 commit into from
Closed

harden: disable external XML entity processing in benchmarks.py... #415

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions benchmarks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
import requests
from autoscraper import AutoScraper
from bs4 import BeautifulSoup
from lxml import etree, html
import defusedxml.lxml

from mechanicalsoup import StatefulBrowser
from parsel import Selector
from pyquery import PyQuery as pq
Expand Down Expand Up @@ -47,10 +48,10 @@ def wrapper(*args, **kwargs):
def test_lxml():
return [
e.text
for e in etree.fromstring(
for e in defusedxml.lxml.fromstring(
large_html,
# Scrapling and Parsel use the same parser inside, so this is just to make it fair
parser=html.HTMLParser(recover=True, huge_tree=True),
parser=defusedxml.lxml._etree.HTMLParser(recover=True, huge_tree=True),
).cssselect(".item")
]

Expand Down Expand Up @@ -135,7 +136,7 @@ def test_autoscraper(request_html):

display(results1)
print("\n" + "=" * 25)
req = requests.get("https://books.toscrape.com/index.html")
req = requests.get("https://books.toscrape.com/index.html", timeout=30)
print(
" Benchmark: Speed of searching for an element by text content, and retrieving the text of similar elements\n"
)
Expand Down