Must have

• Apache Hadoop

• Scala

• Apache Spark

• Yarn

• Kafka

• Hive

• Python

• ETL

• SQL

• REST API

• Unix

• Linux

• Data pipelines

• Hadoop

• Spark

• Spark SQL

• Git

• GitHub

• Ansible

• Jenkins

• Jira

• Big Data

• Airflow

• Control-M

• Cloud

• Design Patterns

• Kanban

• English

Requirements description

• Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Sparkstreaming, YARN, Kafka, Hive, Python, ETL frameworks, Map Reduce, SQL, RESTful services).

•  Sound knowledge on working Unix/Linux Platform.

•  Hands-on experience building data pipelines using Hadoop components - Hive, Spark, Spark SQL.

•  Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible& Jenkins) and requirement management in JIRA.

• Understanding of big data modelling techniques using relational and non-relational techniques.

•  Experience on Debugging the Code issues and then publishing the highlighted differences to thedevelopment team/Architects.

• Experience with: time-series/analytics dB’s such as Elastic search, scheduling tools such as Airflow,Control-M, understanding of Cloud design patterns, developing Hive QL, UDF’s for analysing semi structured/structured datasets.

• Exposure to DevOps & Agile Project methodology such as Scrum and Kanban.

Offer description

Some careers shine brighter than others.

If you’re looking for a career that will help you stand out, join HSBC, and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.

Your career opportunity

The Data Engineer will be responsible for designing, building, and managing the data infrastructure and data

pipeline processes for the bank. The ideal candidate will have a strong background in data engineering, excellent

leadership skills, and a thorough understanding of the banking industry's data requirements.

What we offer

• Competitive salary

• Annual performance-based bonus

• Additional bonuses for recognition awards

• Multisport card

• Private medical care

• Life insurance

• One-time reimbursement of home office set-up (up to 800 PLN).

• Corporate parties & events

• CSR initiatives

• Financial support with trainings and education

• Nursery discounts

• Social fund

• Flexible working hours

• Free parking

Your responsibilities

1. Spark development and design using Scala 2.10+ or Java development or Python development and design using Java 1.8+, automated testing of new and existing components in an Agile, DevOps anddynamic environment.
2. Promote development standards, code reviews, mentoring, knowledge sharing.
3. Production support & troubleshooting.
4. Implement the tools and processes, handling performance, scale, availability, accuracy and monitoring
5. Liaison with BAs to ensure that requirements are correctly interpreted and implemented.
6. Participate in regular planning and status meetings. Input to the development process – through theinvolvement in Sprint reviews and retrospectives. Input into system architecture and design.

show all (7)