Requirements

Expected technologies

Linux

Spark

Iceberg

Java

Our requirements

• Very good understanding of Linux internals (processes/threads, service management, resource allocation)
• Linux as a daily driver of operations
• Good understanding of Spark internals and mechanics
• Knowledge of Iceberg
• 4+ years of commercial experience with Java
• Proven ability to debug JVM/Java (hopefully in Spark ecosystem)
• Ability to design high performance processing pipelines
• Hands-on, proactive approach
• Familiarity with CI/CD pipelines and DevOps practices
• Proficiency in English enables effective communication in an international environment

Your responsibilities

• Taking care of our high-volume (tens of billions of events daily), multi-tenant, and distributed data processing pipelines
• Develop and maintain Java-based and Spark-based Big Data processing pipelines, ensuring scalability and performance
• Work with distributed systems, contributing to the design and implementation of backend solutions
• Collaborate with cross-functional teams to integrate different platforms and ensure smooth data flow
• Troubleshoot and debug issues to improve overall system stability
• Participate in code reviews and offer guidance on best practices
• Debug java/aws/airflow pipelines
• Help team to produce better solutions (teaching/code reviews/showing directions)
• Design high efficient processing pipelines