What will you do:

• Build and maintain robust data pipelines (batch, micro-batch, real-time)
• Work with both structured and unstructured data from diverse sources
• Transform raw data into trusted, value-adding assets for business use
• Implement data quality frameworks (validation, profiling, cleansing)
• Manage storage, metadata and master data for the entire platform
• Support modern orchestration processes and pipeline automation

Requirements:

• Strong skills in SQL, Python, and Spark
• Comfortable with JVM languages (e.g. Java, Scala)
• Experience with CLI environments (bash, PowerShell)
• Familiar with source code management (Git) and dependency management
• Hands-on with ETL/ELT processes (batch, micro-batch, real-time)
• Deep understanding of the data lifecycle
• Solid experience with data quality (validation, profiling, cleansing & conforming)
• Working knowledge of data formats and serialization
• Familiarity with OLTP, OLAP, and NoSQL databases
• Experience with Azure data platforms
• Knowledge of data lake architecture and cloud-native tools
• Skilled in orchestration and distributed systems
• Experience with MPP frameworks, database management, and change control
• Confident with logging, debugging, and monitoring data workflows