Must have

• Spark

• Java

• Kubernetes

• Docker

• AWS

• English (B2)

Nice to have

• Python

• PySpark

• Databricks

Requirements description

We expect from our candidates:

• 4-7 years of experience in data engineering or big data roles
• Expert knowledge of Apache Spark
• Strong experience in deploying and managing Spark clusters on open-source Kubernetes(preferably on AWS with EKS )
• Deep understanding of Docker and containerization for Spark applications
• Experience with Spark on K8s operator, resource scaling, and job orchestration
• Solid programming skills in Java and (Python preferably)
• Good problem-solving skills with a focus on performance optimization
• Experience with distributed systems and cloud-native infrastructure
• English — B2+ level

Nice to have:

• Experience with Databricks and Amazon EMR
• Knowledge of CI/CD pipelines for data deployments
• Understanding of monitoring/observability tools for distributed systems

Offer description

We are looking for a Data Engineer for our client the US technology company, specializing in digital solutions for roadway and infrastructure management. Their platforms provide government agencies and engineering firms with integrated tools for real-time pavement performance analytics, facilitating a proactive approach to roadway maintenance.

Your responsibilities

1. Design and implement scalable data processing pipelines with Apache Spark
2. Deploy and manage Spark clusters on open-source Kubernetes infrastructure
3. Optimize data storage and access with Iceberg and Parquet formats
4. Collaborate with data scientists and engineers to productionize ML workflows
5. Apply best practices for building and running containerized big data workloads
6. Contribute to a platform used for real-time analysis of transportation data and mobility optimization