Requirements

Expected technologies

Python

SQL

PySpark

Azure Data Lake Gen2

Azure Data Factory

Databricks

GitHub VCS

GitHub Actions

Azure DevOps Pipelines

Our requirements

• Bachelor’s/Masters degree in Computer Science, Engineering, or a related field.
• 3+ years of working experience in the field of Data & Analytics, preferably in the CPG industry
• 3+ years of proficient coding experience with Python for data engineering, including SQL and PySpark (DataFrame API, Spark SQL, MLlib), with hands-on experience in various databases (SQL/NoSQL), key libraries (e.g., pandas, SQLAlchemy), parallel processing, and advanced data transformation and performance optimization techniques.
• Ability to develop, and maintain data engineering applications using Python classes and PySpark, ensuring code modularity, reusability, and maintainability.
• Advanced data engineering & technology knowledge (Azure Data Lake Gen2, Azure Data Factory and Databricks as well as data management knowhow (data cataloguing, data quality management).
• Solid understanding of data modeling, ETL processing and lakehouse concepts.
• Solid knowledge of CI/CD processes and tools (GitHub VCS, GitHub Actions, Azure DevOps Pipelines)
• Solid data content knowledge of market data (Sell-out, Sell-through) for the CPG or health care industry which is licensable from data providers like IQVIA, Nielsen and IRI.
• Good understanding of data governance processes to support mapping of data from multiple sources.
• Experience working within international/intercultural teams and using Agile methodologies (Scrum, Kanban) to organize the work.
• Strong problem solving and analytical skills. Excellent interpersonal and communication skills, active listening, consulting, challenging in a constructive way.
• Fluent in English, both written & spoken, intercultural awareness and willingness to travel.

Your responsibilities

• Implement efficient data pipelines that integrate data from different sources and business domains to develop globally harmonized data models and KPI calculations by following data management and data quality standards.
• Make sure you follow data management and data quality standards. Ensure that data is well-managed to build stable, reusable and quality assured data assets.
• Ensure that data products adhere to the data protection & compliance standards. Implement effective data access management policies, data privacy policies and secure data provisioning based on corporate guidelines.
• Contribute to the enhancement of implementation frameworks based on the needs of the analytics products consuming your data asset and based on your own needs to process the data.
• Collaborate with other IT functions (enabling functions data asset teams, analytics teams, platform product managers & integration architects) to ensure the aforementioned activities are executed effectively.
• Collaborate with other data engineers in your team (internal or external engineers) on solving tasks and implement work packages.
• Estimate owned product backlog items, ensure that own deliverables are properly tested, documented and handed over to the operations team.