Data Tool Stack Engineer

Description

You will play a crucial role in shaping the future of Data Engineering and actively contribute to the success of the digital transformation at ZEISS.

In this challenging function, you will design, implement, and maintain our robust on-premises and hybrid data tool stack, with a strong focus on Kubernetes-based environments. As a Data Tool Stack Engineer at ZEISS, you will work in a supportive, team-oriented environment that enables the co-design and development of scalable, high-performance data platforms. Together with your colleagues, you will build modern data ingestion, streaming, and integration solutions, ensuring seamless data flows across our ecosystems. You will automate operations, establish modern CI/CD pipelines, and leverage cutting-edge agentic DevOps tools to elevate engineering efficiency and empower data-driven decisions across the company.

Design, implement, and maintain our on-premises data tool stack, with a strong focus on Kubernetes-based environments, while supporting hybrid cloud integrations (e.g., MS Azure).

Build and operate scalable data ingestion, messaging, and Change Data Capture (CDC) solutions using technologies such as MQTT, Kafka, Kafka Connect, and Debezium.

Develop, optimize, and orchestrate robust ETL/ELT pipelines and data processing workflows using Python, PySpark, SQL, dbt, and Apache Airflow.

Integrate and operate modern distributed query engines like Trino/Starburst to enable fast and efficient data access.

Establish and continuously improve CI/CD pipelines for data and platform components, automating deployment, configuration, and infrastructure management.

Design and support networking and connectivity concepts, including Nginx, virtual networks (VNets), and hybrid cloud/on-prem connectivity.

Leverage AI agents and agentic engineering approaches in DevOps to increase automation, operational efficiency, and platform reliability.

Ensure system reliability and transparency by implementing observability, logging, and search solutions using Grafana, Elasticsearch, or OpenSearch.

University degree, preferably computer science, natural sciences, or engineering, paired with a passion for data platforms and automation

Strong software engineering skills with Python along with very good SQL skills

Hands-on experience with Kubernetes and Helm, as well as practical experience in implementing and maintaining CI/CD pipelines

Experience with ETL/ELT processing, data pipelines, PySpark, and workflow orchestration or data transformation technologies such as Apache Airflow, dbt, or comparable solutions

Experience with messaging and streaming technologies such as Kafka, MQTT, Kafka Connect, and Debezium or comparable change data capture technologies

Knowledge of distributed SQL/query engines such as Trino or Starburst, alongside observability and search technologies like Grafana, Elasticsearch, or OpenSearch

Good understanding of networking concepts, reverse proxies (Nginx), network routing, VNets, and hybrid connectivity (e.g., Azure ExpressRoute)

Beneficial experience with Microsoft Azure, Databricks (including automation and deployment approaches), Azure Data Factory (ADF), Terraform or other Infrastructure-as-Code technologies, hybrid on-premises/cloud architectures, and interest in applying AI agents and agentic engineering approaches to DevOps and platform operations

Your ZEISS Recruiting Team:

Wenner Lili

Responsibilities

• You will play a crucial role in shaping the future of Data Engineering and actively contribute to the success of the digital transformation at ZEISS.
• In this challenging function, you will design, implement, and maintain our robust on-premises and hybrid data tool stack, with a strong focus on Kubernetes-based environments. As a Data Tool Stack Engineer at ZEISS, you will work in a supportive, team-oriented environment that enables the co-design and development of scalable, high-performance data platforms. Together with your colleagues, you will build modern data ingestion, streaming, and integration solutions, ensuring seamless data flows across our ecosystems. You will automate operations, establish modern CI/CD pipelines, and leverage cutting-edge agentic DevOps tools to elevate engineering efficiency and empower data-driven decisions across the company.
• Design, implement, and maintain our on-premises data tool stack, with a strong focus on Kubernetes-based environments, while supporting hybrid cloud integrations (e.g., MS Azure).
• Build and operate scalable data ingestion, messaging, and Change Data Capture (CDC) solutions using technologies such as MQTT, Kafka, Kafka Connect, and Debezium.
• Develop, optimize, and orchestrate robust ETL/ELT pipelines and data processing workflows using Python, PySpark, SQL, dbt, and Apache Airflow.
• Integrate and operate modern distributed query engines like Trino/Starburst to enable fast and efficient data access.
• Establish and continuously improve CI/CD pipelines for data and platform components, automating deployment, configuration, and infrastructure management.
• Design and support networking and connectivity concepts, including Nginx, virtual networks (VNets), and hybrid cloud/on-prem connectivity.
• Leverage AI agents and agentic engineering approaches in DevOps to increase automation, operational efficiency, and platform reliability.
• Ensure system reliability and transparency by implementing observability, logging, and search solutions using Grafana, Elasticsearch, or OpenSearch.
• University degree, preferably computer science, natural sciences, or engineering, paired with a passion for data platforms and automation
• Strong software engineering skills with Python along with very good SQL skills
• Hands-on experience with Kubernetes and Helm, as well as practical experience in implementing and maintaining CI/CD pipelines
• Experience with ETL/ELT processing, data pipelines, PySpark, and workflow orchestration or data transformation technologies such as Apache Airflow, dbt, or comparable solutions
• Experience with messaging and streaming technologies such as Kafka, MQTT, Kafka Connect, and Debezium or comparable change data capture technologies
• Knowledge of distributed SQL/query engines such as Trino or Starburst, alongside observability and search technologies like Grafana, Elasticsearch, or OpenSearch
• Good understanding of networking concepts, reverse proxies (Nginx), network routing, VNets, and hybrid connectivity (e.g., Azure ExpressRoute)
• Beneficial experience with Microsoft Azure, Databricks (including automation and deployment approaches), Azure Data Factory (ADF), Terraform or other Infrastructure-as-Code technologies, hybrid on-premises/cloud architectures, and interest in applying AI agents and agentic engineering approaches to DevOps and platform operations
• Your ZEISS Recruiting Team:
• Wenner Lili