Job Summary: Develop advanced bots for web scraping, building robust, secure, and scalable solutions in the automotive market. Key Highlights: 1. Development of bots for automated data collection 2. Solutions to overcome barriers such as CAPTCHAs and IP blocking 3. Integration of scraping practices with legal and ethical guidelines **COME MEET CARBIGDATA!** We are the largest big data platform in the vehicle market. Our products focus on locating stolen and hijacked vehicles, as well as determining the true status of located assets. Specializing in big data analytics, we serve major players in the automotive industry, including banks, finance companies, leasing firms, and insurance providers. **JOB DESCRIPTION** Your mission will be to develop advanced bots for web scraping, creating robust, secure, and scalable solutions. Day-to-day challenges will include CAPTCHA solving, strategic use of proxies, and simulating human interactions within highly protected environments. **DAILY RESPONSIBILITIES:** * Develop bots for automated data collection, ensuring efficiency and resilience. * Design solutions to overcome barriers such as CAPTCHAs, IP blocking, and anti-bot verification. * Implement and optimize proxy routing and management systems (residential, datacenter, rotating, etc.). * Integrate scraping practices with legal and ethical guidelines. * Monitor bots under high-volume data scenarios, ensuring performance and scalability. * Perform logging and debugging for continuous bot analysis and improvement. * Apply agile development methodologies (SCRUM or similar). **REQUIREMENTS AND QUALIFICATIONS:** * Programming language: Python. * Proven experience of over 4 years in development, with emphasis on automation and scraping. * Scraping frameworks and libraries: Scrapy, Selenium. * Experience with Playwright or Puppeteer for browser-based scraping. * CAPTCHA solving: Knowledge of OCR (Tesseract) and integration with services such as 2Captcha, Anti-Captcha, DeathByCaptcha. * Familiarity with machine learning methodologies for solving custom CAPTCHAs. * Proxy management: Experience with rotating proxies and proxy pools. * Header and cookie management: To simulate human-like requests. * Familiarity with protection mechanisms (e.g., Cloudflare) and strategies to bypass them. * Experience with WebSockets and real-time scraping. * Use of containers (Docker) for bot deployment and management. * Development in Unix/Linux environments. **PREFERRED QUALIFICATIONS** * Knowledge of the JAVA programming language. * Experience with HTTP traffic analysis tools such as Fiddler, Wireshark, or Burp Suite. * Basic understanding of information security and strategies to circumvent anti-scraping measures. * Familiarity with distributed crawling and queue systems such as RabbitMQ, Kafka, or Celery. * Experience with cloud computing platforms (AWS, Azure, GCP) for bot hosting and scalability. * Development of RESTful APIs for integration with external systems. **OUR BENEFITS** * Meal/Voucher allowance * Health insurance (Sulamerica) * Dental insurance * Birthday day off * Life insurance * Remote work Join the transformation in Brazil’s automotive market! If you’re passionate about data and technology, join our team and be part of the automotive industry revolution. **#VemSerCarBigData** Employment type: Full-time CLT Benefits: * Medical assistance * Dental assistance * Childcare allowance * Meal voucher Selection question(s): * What is your expected salary (CLT)? * Are you available to work remotely? Work location: Remote