Data Engineer - Just Transition

Description

Job Summary: This role involves automating and optimizing the discovery and mapping of key stakeholders in the Just Transition ecosystem using data engineering and AI. Key Highlights: 1. Develop and manage automated data collection flows (web scraping) 2. Implement AI agents and Machine Learning systems for entity identification 3. Structure algorithms to recognize relationships between actors **Introduction** The transition to a sustainable economy presents the country with a structural challenge: ensuring that decarbonization, climate adaptation, and productive modernization efforts result in sustainable economic development without exacerbating social vulnerabilities or generating disproportionate impacts on workers, communities, and territories. In this context, the principle of just transition guides the need to integrate economic, social, and environmental dimensions in a balanced manner, guaranteeing that the benefits of the new economy are accessible to all and that any adverse effects are adequately mitigated. This project aims to support a partner of the Cordial Institute in establishing an initial Resilience and Just Transition pillar through a robust process of discovery and knowledge systematization, with the objective of consolidating the conceptual, thematic, and strategic foundations that will guide the organization’s future work in this field. The core of this phase consists of developing the Network Analysis: a structured document that organizes and synthesizes the main relevant stakeholders, their agendas, conflicts, convergences, and cooperation potentials. This process will enable the client to gain deep insight into the ecosystem—its scale, critical actors, and potential future pathways of action. **Opportunity Description** The Data Engineering and AI Specialist role within this project aims to automate and optimize the discovery and mapping process of key actors in the Just Transition ecosystem. This person will be responsible for developing large-scale data collection architectures and implementing Artificial Intelligence agents capable of identifying relevant events, organizations, and individuals, while establishing logical connections among them. The work will serve as a structured foundation enabling the Network Analysis researcher to conduct sociocentric and strategic analysis of the ecosystem. **Candidate Profile** * Bachelor’s degree in Computer Engineering, Data Science, Software Engineering, Statistics, or related fields; * Proven experience in data engineering and process automation projects; * Advanced proficiency in Python (focused on web scraping, APIs, and AI integration); * Practical experience with scraping tools such as Apify, SerpApi, Selenium, BeautifulSoup, or Scrapy; * Solid knowledge of implementing and orchestrating AI Agents (e.g., LangChain, AutoGPT, or API-based model integrations); * Proficiency in cloud databases (Google Cloud Platform / BigQuery) and SQL; * Experience with Git/GitHub for code documentation and collaborative development; * Ability to structure datasets for multivariate and graph-based analysis; * Advanced English proficiency for technical reading and integration of international APIs; * Prior knowledge or interest in sustainability and climate agenda topics is desirable. **Responsibilities** * Develop and manage automated data collection flows (web scraping) using tools such as SerpApi and Apify to extract information from social media platforms (LinkedIn, Instagram, Twitter) and search engines; * Implement and configure AI agents and Machine Learning systems for autonomous identification of entities (individuals, organizations, events) and classification of topics related to Just Transition; * Structure relationship recognition algorithms to automatically establish connections among identified actors (e.g., participation in common events, institutional partnerships, cross-citations); * Develop and maintain the structured database of the network mapping, ensuring data integrity, necessary anonymization (LGPD), and quality for the analysis phase; * Collaborate with the Analysis and Research team and the Network Analysis Specialist to ensure extracted data meets topological requirements for Social Network Analysis (SNA); * Document all scraping processes and AI agent logic in code repositories (GitHub); * Support the integration of AI tools for generating preliminary syntheses and insights regarding mapped stakeholders. **Contract Format** * Remote work; * Engagement will be conducted via service provision, scoped and time-bound; * The start date for counting deliverables (“kick-off”) will be established by email upon commencement of the research. **Diversity Incentive** The Cordial Institute is committed to promoting inclusion and diversity within its team. Therefore, we encourage applications from women, Black individuals, Indigenous peoples, LGBTQ+, persons with disabilities, and persons with special needs.