Requirements

Expected technologies

Hive

Spark

HDFS

Kafka

Linux

Hadoop

Operating system

Linux

Our requirements

• 5+ years of experience with Big Data solutions in on-prem or cloud environments
• Hands-on expertise with Cloudera tools including Hive, Spark, HDFS, and Kafka
• Proficiency in Linux system administration and shell scripting
• Strong understanding of Hadoop platform security and encryption practices
• Experience implementing CI/CD pipelines using Jenkins and Ansible
• Ability to troubleshoot performance and infrastructure issues
• Knowledge of Agile and DevOps principles
• Strong problem-solving skills and independent thinking
• Excellent communication and collaboration skills
• Familiarity with capacity planning and operational support in data platforms

Optional

• Exposure to centralized S3 data storage systems like VAST
• Experience optimizing Spark workloads for large-scale processing
• Background in working with Apache Ranger and Knox for advanced security
• Familiarity with Agile and Kanban project methodologies

Your responsibilities

• Developing and testing Big Data components in a Cloudera-based environment
• Creating and configuring new Hadoop users and managing Kerberos authentication
• Setting up and maintaining Spark, HDFS, and MapReduce access
• Monitoring and tuning performance of Apache Spark jobs
• Executing end-to-end Cloudera cluster installation and capacity planning
• Automating deployments using Jenkins and Ansible
• Managing platform security using Apache Ranger, Knox, and Kerberos
• Troubleshooting and resolving system-level issues and incidents
• Collaborating with data engineers to optimize processing pipelines
• Incorporating centralized S3-based storage across the processing stack