Skip to content

Commit 4947152

Browse files
committed
docs: change docstring style for some methods
1 parent 9742b23 commit 4947152

4 files changed

Lines changed: 156 additions & 164 deletions

File tree

scrapling/engines/_browsers/_controllers.py

Lines changed: 24 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -89,19 +89,18 @@ def fetch(self, url: str, **kwargs: Unpack[PlaywrightFetchParams]) -> Response:
8989
"""Opens up the browser and do your request based on your chosen options.
9090
9191
:param url: The Target url.
92-
:param kwargs: Additional keyword arguments including:
93-
- google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
94-
- timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
95-
- wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
96-
- page_action: Added for automation. A function that takes the `page` object and does the automation you need.
97-
- extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
98-
- disable_resources: Drop requests for unnecessary resources for a speed boost.
99-
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
100-
- wait_selector: Wait for a specific CSS selector to be in a specific state.
101-
- wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
102-
- network_idle: Wait for the page until there are no network connections for at least 500 ms.
103-
- load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
104-
- selector_config: The arguments that will be passed in the end while creating the final Selector's class.
92+
:param google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
93+
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
94+
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
95+
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
96+
:param extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
97+
:param disable_resources: Drop requests for unnecessary resources for a speed boost.
98+
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
99+
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
100+
:param wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
101+
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
102+
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
103+
:param selector_config: The arguments that will be passed in the end while creating the final Selector's class.
105104
:return: A `Response` object.
106105
"""
107106
params = _validate(kwargs, self, PlaywrightConfig)
@@ -219,19 +218,18 @@ async def fetch(self, url: str, **kwargs: Unpack[PlaywrightFetchParams]) -> Resp
219218
"""Opens up the browser and do your request based on your chosen options.
220219
221220
:param url: The Target url.
222-
:param kwargs: Additional keyword arguments including:
223-
- google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
224-
- timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
225-
- wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
226-
- page_action: Added for automation. A function that takes the `page` object and does the automation you need.
227-
- extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
228-
- disable_resources: Drop requests for unnecessary resources for a speed boost.
229-
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
230-
- wait_selector: Wait for a specific CSS selector to be in a specific state.
231-
- wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
232-
- network_idle: Wait for the page until there are no network connections for at least 500 ms.
233-
- load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
234-
- selector_config: The arguments that will be passed in the end while creating the final Selector's class.
221+
:param google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
222+
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
223+
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
224+
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
225+
:param extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
226+
:param disable_resources: Drop requests for unnecessary resources for a speed boost.
227+
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
228+
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
229+
:param wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
230+
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
231+
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
232+
:param selector_config: The arguments that will be passed in the end while creating the final Selector's class.
235233
:return: A `Response` object.
236234
"""
237235
params = _validate(kwargs, self, PlaywrightConfig)

scrapling/engines/_browsers/_stealth.py

Lines changed: 26 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -175,20 +175,19 @@ def fetch(self, url: str, **kwargs: Unpack[StealthFetchParams]) -> Response:
175175
"""Opens up the browser and do your request based on your chosen options.
176176
177177
:param url: The Target url.
178-
:param kwargs: Additional keyword arguments including:
179-
- google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
180-
- timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
181-
- wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
182-
- page_action: Added for automation. A function that takes the `page` object and does the automation you need.
183-
- extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
184-
- disable_resources: Drop requests for unnecessary resources for a speed boost.
185-
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
186-
- wait_selector: Wait for a specific CSS selector to be in a specific state.
187-
- wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
188-
- network_idle: Wait for the page until there are no network connections for at least 500 ms.
189-
- load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
190-
- solve_cloudflare: Solves all types of the Cloudflare's Turnstile/Interstitial challenges before returning the response to you.
191-
- selector_config: The arguments that will be passed in the end while creating the final Selector's class.
178+
:param google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
179+
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
180+
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
181+
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
182+
:param extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
183+
:param disable_resources: Drop requests for unnecessary resources for a speed boost.
184+
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
185+
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
186+
:param wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
187+
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
188+
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
189+
:param solve_cloudflare: Solves all types of the Cloudflare's Turnstile/Interstitial challenges before returning the response to you.
190+
:param selector_config: The arguments that will be passed in the end while creating the final Selector's class.
192191
:return: A `Response` object.
193192
"""
194193
params = _validate(kwargs, self, StealthConfig)
@@ -388,20 +387,19 @@ async def fetch(self, url: str, **kwargs: Unpack[StealthFetchParams]) -> Respons
388387
"""Opens up the browser and do your request based on your chosen options.
389388
390389
:param url: The Target url.
391-
:param kwargs: Additional keyword arguments including:
392-
- google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
393-
- timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
394-
- wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
395-
- page_action: Added for automation. A function that takes the `page` object and does the automation you need.
396-
- extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
397-
- disable_resources: Drop requests for unnecessary resources for a speed boost.
398-
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
399-
- wait_selector: Wait for a specific CSS selector to be in a specific state.
400-
- wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
401-
- network_idle: Wait for the page until there are no network connections for at least 500 ms.
402-
- load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
403-
- solve_cloudflare: Solves all types of the Cloudflare's Turnstile/Interstitial challenges before returning the response to you.
404-
- selector_config: The arguments that will be passed in the end while creating the final Selector's class.
390+
:param google_search: Enabled by default, Scrapling will set the referer header to be as if this request came from a Google search of this website's domain name.
391+
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
392+
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
393+
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
394+
:param extra_headers: A dictionary of extra headers to add to the request. _The referer set by the `google_search` argument takes priority over the referer set here if used together._
395+
:param disable_resources: Drop requests for unnecessary resources for a speed boost.
396+
Requests dropped are of type `font`, `image`, `media`, `beacon`, `object`, `imageset`, `texttrack`, `websocket`, `csp_report`, and `stylesheet`.
397+
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
398+
:param wait_selector_state: The state to wait for the selector given with `wait_selector`. The default state is `attached`.
399+
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
400+
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
401+
:param solve_cloudflare: Solves all types of the Cloudflare's Turnstile/Interstitial challenges before returning the response to you.
402+
:param selector_config: The arguments that will be passed in the end while creating the final Selector's class.
405403
:return: A `Response` object.
406404
"""
407405
params = _validate(kwargs, self, StealthConfig)

0 commit comments

Comments
 (0)