We need a Python developer to build and maintain web scrapers to collect data from various websites. This contract role involves writing scripts using libraries like Scrapy or Beautiful Soup to extract specific data and store it in our database. You will be responsible for handling challenges like JavaScript-rendered pages, anti-scraping measures, and data cleaning. This is a great project for a developer who enjoys solving puzzles and working with web data.

As a Web Scraper Developer, you will be responsible for our data acquisition efforts. You will be building the tools that gather the raw data that fuels our business. This is a challenging role that requires a combination of coding skills, problem-solving, and persistence.

Your Day-to-Day Tasks:

• Analyze target websites to understand their structure and identify the data to be extracted.
• Write robust and efficient web scrapers using Python and libraries like Scrapy, Beautiful Soup, and Selenium.
• Develop strategies to handle challenges like CAPTCHAs, IP blocks, and dynamic content loaded with JavaScript.
• Clean, parse, and structure the scraped data.
• Store the extracted data in our database or in a structured file format like JSON or CSV.
• Maintain and update existing scrapers to adapt to changes in the target websites.

Who you are:

You are a curious and tenacious developer. You are not deterred by obstacles and enjoy the cat-and-mouse game of web scraping. You have a good understanding of how the web works (HTTP, HTML, CSS, JavaScript) and you are proficient in Python. This is a great opportunity to specialize in the field of data acquisition and work on a series of interesting and varied challenges.

• Strong proficiency in Python.
• Experience with web scraping libraries (e.g., Scrapy, Beautiful Soup, Selenium).
• Good understanding of HTML, CSS, and web protocols.
• Strong problem-solving skills and persistence.
• Basic knowledge of databases or data storage is a plus.

• Work on a variety of interesting web scraping challenges
• Become an expert in data acquisition techniques
• Flexible, fully remote work environment
• See the direct impact of your work on the company's data assets