Key takeaways
- Scrapers break when a website changes its layout. Scrapling says its element tracking can relocate the same elements afterwards.
- One Python library covers plain requests, stealth browsers and full crawls, so you do not stitch several tools together.
- Long crawls can be paused and resumed, and blocked requests are detected and retried, so a stoppage need not mean starting over.
- Its own parser micro-benchmark shows 1.99 ms against 1562.1 ms for BS4 with Lxml. These are the project's figures, not ours.
- The catch: robots.txt support is opt-in, and the README says you must respect site terms and the law.
What it is
Scrapling is a Python web scraping framework that brings fast HTTP requests, stealth browsers, parsing and concurrent crawling into one library. It is open source under the BSD-3-Clause licence and is trending on GitHub.
The problem it targets
Anyone who collects data from websites knows the routine. A page is redesigned, a selector stops matching, and a working script quietly returns nothing. Sites that block automated traffic add a second kind of failure.
Scrapling is aimed at both. Its README describes element tracking that relocates elements after website changes, and spiders that detect blocked requests and retry them.
What you get
- Fewer broken scripts: smart element tracking uses similarity algorithms to find elements again after a change.
- Fewer tools to juggle: plain requests that can impersonate browser fingerprints, and a stealth browser mode, share one interface.
- Long crawls that survive interruption: pause and resume saves pending requests and seen URLs.
- Politer crawling: AutoThrottle slows the spider when a site starts blocking or rate-limiting it.
- Agent-ready: an MCP server lets AI agents scrape through the library.
The README also says it can bypass Cloudflare Turnstile and Interstitial challenges. That is the project's claim and we have not checked it.
How it compares
In the project's own text-extraction test on 5000 nested elements, the parser takes 1.99 ms. BS4 with Lxml takes 1562.1 ms in the same table. This is a parser micro-benchmark from the project, not a measure of a whole scraping job, where network time usually dominates.
The catches
- Legal and ethical: the README says the library is for educational and research purposes only, and that users must always respect website terms of service and robots.txt files.
- robots.txt support is an optional flag for spiders, not the default.
- Bypassing bot protection does not make scraping permitted. Check a site's terms first, and take advice where personal data or commercial use is involved.
- The base package is parser-only; fetching and browser features need extras.
What we did not test
We did not run Scrapling on any live site. We did not test the element tracking after a layout change, the Cloudflare handling, the MCP server or the benchmark. Every claim is tied to a quote from the README or the GitHub API.
Verdict
Editors' view: teams that maintain scrapers against sites they are allowed to scrape, and tire of repairing selectors, have reason to try it. Anyone hoping it removes the need to read a site's terms should look elsewhere.
Sources
Need help adopting this, or have a software project in mind?
Tell us what you are building. The initial consultation is free. Email [email protected] or follow @zolnaltd.
Start a conversation →