Requirements

Expected technologies

SQL

Operating system

Windows

Our requirements

• Strong hands-on experience with PySpark and Apache Spark
• Proficiency with Airflow for orchestrating data pipelines
• Solid SQL skills for data transformation and validation
• Experience working with Databricks in a production environment
• Familiarity with data modeling concepts (desired, but not required)

Your responsibilities

• Take ownership of existing data pipelines, migrating them from Databridge to our new team
• Address and resolve high-priority technical debt in Spark-based ETL systems
• Build and maintain scalable, production-grade data pipelines using PySpark, SQL, and Airflow
• Work within Databricks to optimize workflows and data processing
• Collaborate with ML and product teams to ensure reliable delivery of high-quality data
• Contribute to establishing best practices, documentation, and engineering processes within the team
• Support data modeling efforts to improve downstream data usability