The goal of the project was to build scrapers for regular scraping of real estate properties from Redfin and Zillow. The data must be updated twice a day to keep the database relevant. The scraper grabs all property details including images.
For scraping we developed multi-thread scraper, one for Zillow and one for Redfin. The scrapers used our cheap and effective custom proxy application in order to get data quickly and not be banned by the sites.