Share your use case #661
Replies: 8 comments
|
Thanks for sharing! |
|
Speaking of GUI, there is Dashboard UI for Ferret Server, it's still WIP, but you are welcome to contribute. |
|
We are using it to scrape home showing data for our company’s real estate listings because they provide no API to access it. Overall it’s been a pretty solid solution save for issues where the sites we scrape are slow to load or have issues due to lots of SSO redirection. A nice to have feature would be something like “navigate to a URL and wait for it to get to this other URL after a series of redirections” |
|
@ap0 thank you for sharing! |
|
@ap0 how did you integrate it? Are you using it via script as a CLI or embedded into your scraping system? |
We embed it into the scraping system. |
|
I would like to extract data from a each row of a given listing. Extract the links if they exist in the rows, follow these links, extract content form these pages. This must be done for all pages of the listing. I personally would be very happy for any direction that would make scraping fast and mostly automatized. |
|
Structured scrapping The following is the approach that I had been needing the most (adapted from https://github.com/andrew-d/goscrape): The architecture of scraping is roughly as follows:
Almost every single task that I need to do falls within such description. Like in https://www.montferret.dev/cookbook/google-search/, but need an outer loop to walk page by page until limit is reached. |
Uh oh!
There was an error while loading. Please reload this page.
Hello folks!
I would like to ask you to share your use cases of using Ferret in order to better understand in which direction the project should move.
All reactions