Scrapy is the free leading open source and collaborative framework for extracting the data you need from websites in a fast, simple, yet extensible way.
It allows you to build and run your web spiders and deploy them to Scrapy Cloud (via a Scrapinghub API Key) or use Scrapyd to host the spiders on your own server.
All you have to do is write the rules to extract the data and let Scrapy do the rest. If you need additional functionality you can easily do so without having to touch the Scrapy-core. Scrapy has a healthy developer community that contribute actively to maintain and extend the software.
Scrappy is written in Python and runs on Linux, Windows, Mac and BSD.