Senior Data Engineer, Unified Platform

Description

The company is a diversified trading firm with more than 30 years of experience. It trades on its own capital and at its own risk, and it moves quickly to act on opportunities.

Headquartered in Chicago, the firm has offices across the U.S., Canada, Europe, and Asia. It trades Fixed Income, ETFs, Equities, FX, Commodities, and Energy in all major global markets, and it also runs three non-traditional strategies: real estate, venture capital, and cryptoassets.

The Data Ecosystem team sits within the Unified Platform (UP) organization. It builds the platform capabilities, managed infrastructure, and metadata management systems behind a common query layer that supports thousands of batch and streaming pipelines across the firm.

As a Senior Data Engineer, you will design, build, and run the data platform and managed services used by Traders, Quantitative Researchers, and Back-Office staff. You will own engineering projects and service components, work closely with platform users on technical problems, and help build metadata management and governance capabilities. The role is hands-on and combines technical design ownership, software development, production support, and mentoring within the wider platform architecture.

Responsibilities

• Design, build, and run managed platform services for data storage, distributed querying, batch and streaming processing, and workflow orchestration.
• Own the technical design and implementation choices for services and components in the wider architecture, and lead them from requirements through deployment and support.
• Automate provisioning, configuration, upgrades, and monitoring of managed services to cut manual work and make deployments more consistent.
• Build and connect metadata tools, such as data catalogs and lineage tracking, so users can find datasets and understand their structure, owners, and dependencies.
• Put schema governance, data quality checks, access controls, and usage tracking in place under established policies, working with security and data stakeholders on the technical requirements.
• Work with managed service users to diagnose integration and performance problems, clarify what they need technically, and improve the services their workloads rely on.
• Raise the reliability and performance of managed services and shared processing infrastructure through observability, capacity planning, and testing. Take part in production support and incident response, fix root causes, and keep operational runbooks current.
• Carry out platform upgrades and data migrations, with validation and compatibility checks to protect downstream workloads.
• Lead design and code reviews for the services you own, record technical trade-offs, and propose improvements to architectural patterns shared by the team.
• Mentor less experienced engineers through hands-on work and feedback, encouraging maintainable code, automated testing, and sound operational habits.

Requirements

• 7+ years of experience with modern data technologies, building data-intensive distributed systems, or both.
• Strong skill in Java and/or Python, with a record of delivering high-quality, maintainable production software. Solid familiarity with SQL, Bash, and Linux.
• Experience building and running managed data or infrastructure services used by other teams, including operational automation and production support.
• Proven ownership of engineering projects or service components from technical design through deployment and ongoing operation.
• Solid grasp of distributed systems trade-offs, including fault tolerance, concurrency, partitioning, and recovering from partial failures.
• Production experience with batch and streaming systems, including practical handling of state, retries, replay, and data consistency.
• Experience with processing or query technologies such as Flink, Spark, or ClickHouse, and the ability to assess their performance and operational trade-offs.
• Experience with several storage technologies, including object storage, relational or NoSQL databases, and event streaming systems such as Kafka. Familiarity with open table formats such as Iceberg or Delta Lake is a plus.
• Practical experience with data modeling, schema governance and evolution, and data quality controls. Experience integrating metadata management, data cataloging, or lineage into data systems.
• Experience with data formats and serialization systems such as Arrow, Parquet, Protobuf, or Avro.
• Experience with cloud-native data infrastructure, including Kubernetes, workflow orchestration, automated deployment, and production monitoring.
• Strong communication and problem-solving skills. You can troubleshoot with platform users, collaborate across teams, set priorities across competing work, and deliver in a fast-moving trading environment.

Conditions

Optional: experience extending or scaling data catalogs or lineage tracking tools across several storage and compute technologies.

Optional: experience implementing access controls, usage tracking, or other governance features in shared data environments.

Optional: familiarity with financial datasets and the needs of trading, quantitative research, or back-office users.

The company is an equal opportunity employer.

Benefits

• Annual base salary of $160,000 to $250,000, depending on experience, qualifications, and relevant skill set.
• Eligible for an annual discretionary bonus.
• Group medical, pharmacy, dental, and vision insurance, plus short- and long-term disability, life and AD&D insurance, health savings accounts, and flexible spending accounts.
• 401k with a discretionary employer match.