Data Engineer - Remote Sensing

Digital Hub Warsaw at Bayer

Warszawa, Ochota
21000 zł/mth.
Hybrydowa
🐍 Python
Go
Kafka
☁️ AWS SQS
🔍 Google Cloud Pub/Sub
GitHub
☁️ AWS
GDAL
STAC
📊 Geospatial data
Hybrydowa

Requirements

Expected technologies

Python

Go

Kafka

AWS SQS

Google Cloud Pub/Sub

GitHub

AWS

GDAL

STAC

Geospatial data

Optional technologies

GCP

Operating system

Windows

macOS

Our requirements

  • Bachelor's degree in Computer Science, Geography, Engineering, or relevant job experience
  • Strong proficiency in Python and/or GO programming languages
  • Strong experience with PostgreSQL/PostGIS, including schema design and query optimization.
  • Hands-on experience with event-driven and streaming data architectures using platform and services such as Kafka, AWS SQS, and Google Cloud Pub/Sub
  • Hands-on experience with large, complex geospatial data sets (vector or raster) and geospatial processing
  • Strong understanding of geospatial concepts (projections, datums), geospatial data types, and the practical requirements associated with processing and analyzing geospatial data
  • Experience with programmatic use of Geospatial Data Abstraction Library (GDAL) to manipulate geospatial data
  • Experience with building RESTful APIs using common frameworks
  • Strong understanding of asynchronous programming patterns and best practices, including coroutines, event loops, and concurrent execution models for optimizing data processing pipelines and API integrations
  • Experience with utilizing Docker to build and deploy containerized applications
  • Experience in cloud platforms such as AWS and GCP, including native data and compute services such as Bigquery, Aurora, GCS/S3, GKE/EKS, GCE/EC2, Cloud Functions/Lambda
  • Experience with code versioning and dependency management systems such as GitHub
  • Excellent problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.
  • Experience applying test-driven development principles to data engineering workflows, including unit testing, integration testing, and mocking of data dependencies.
  • Strong communication skills and the ability to articulate technical concepts to non-technical stakeholders.
  • Preferred:
  • Highly proficient in Python or Golang with a strong track record of building and maintaining production data pipelines and backend systems
  • Hands-on experience working with Kubernetes(K8S) for orchestrating and managing containerized data services and workflows.
  • Experience with automating pipelines with OGC (WFS, WPS, WMS, WTMS) and SpatioTemporal Asset Catalog (STAC) specifications in production
  • Familiarity with CI/CD practices and tools such as GitHub Actions, Terraform, Google Cloud Build, ArgoCD
  • Experience with integrating geospatial compute platforms like Google Earth Engine / Planet Insights Platform into enterprise workflows
  • Experience with object-oriented design, coding and testing patterns, and implementing complex data projects in a large-scale data infrastructure.
  • Solid understanding of geospatial data concepts. Experience with data processing and analysis using geospatial libraries and tools.
  • Experience with monitoring and logging tools such as Grafana, Prometheus, ELK stack, or equivalent.
  • Familiarity with cloud-based machine learning services and platforms such as Google Cloud Vertex AI or AWS SageMaker. Experience with deploying and invoking model endpoints.
  • Solid understanding of networking concepts, security principles, and best practices for cloud environments.
  • Experience working with customers and developers to deliver full-stack development solutions; the ability to translate customer requirements into technical requirements in an Agile environment.

Your responsibilities

  • Design, build, deploy and support cloud-based and open-source solutions that ingest and process satellite data and associated metadata for a global enterprise
  • Implement scalable data pipelines for ingestion, transformation, and delivery of structured and unstructured geospatial data.
  • Build highly scalable APIs for accessing geospatial data and initiating geospatial processing and analysis.
  • Develop event-driven data processing solutions using Kafka, AWS SQS, and Google Cloud Pub/Sub to orchestrate multi-stage spatial workflows.
  • Integrate and manage data flows across cloud platforms such as AWS and GCP, databases such as PostgreSQL/PostGIS and BigQuery, and cloud storage such as AWS S3 and Google Cloud Storage
  • Leverage Kubernetes (K8s) for deploying and managing containerized applications and workflows.
  • Collaborate with data engineers and SREs to optimize and monitor data pipelines and services for performance, reliability, scalability, and cost-effectiveness.
  • Provide technical support, including incident response, troubleshooting and resolution for production issues in data pipelines and API services.
  • Integrate company and industry standards and best practices for data security and regulatory requirements to stay compliant with established policies.
Wyświetlenia: 4
Opublikowana10 dni temu
Wygasaza 3 dni
Tryb pracyHybrydowa
Źródło
Logo
Logo

Podobne oferty, które mogą Cię zainteresować

Na podstawie "Data Engineer - Remote Sensing"