Senior Data Engineer, Unified Platform
drwBrooklyn, NY
About the role
Overview:
As a Senior Data Engineer in DRW’s Unified Platform, you will design, build, and operate the data platform and managed services used by traders, researchers, and back-office teams. You will own engineering projects, drive complex technical decisions, and mentor teammates within the platform architecture. You’ll contribute to metadata governance and data catalogs to improve discoverability and trust in datasets. This role combines hands-on development with production support in a fast-paced trading environment, offering opportunities to shape scalable data infrastructure across DRW’s global platforms. Compensation / Benefits group medical, pharmacy, dental and vision insurance 401k with discretionary employer match short and long-term disability life and AD&D insurance health savings accounts (HSA) flexible spending accounts (FSA)
Responsibilities:
Design, build, and operate managed platform services for data storage, distributed query execution, batch/streaming processing, and workload orchestration Own technical designs and lead delivery of services from requirements to deployment and support Automate provisioning, configuration, upgrades, and monitoring of managed services to reduce manual ops Develop and integrate metadata management tools (data catalogs, lineage) for dataset understanding and ownership Implement schema governance, data quality checks, access controls, and usage tracking with security teams Collaborate with service users to diagnose integration and performance issues and refine requirements Improve reliability and performance via observability, capacity planning, and testing Lead platform upgrades and data migrations with validation to protect downstream workloads Lead design and code reviews, document trade-offs, and contribute to architectural patterns Mentor engineers to promote maintainable code, testing, and operational excellence
Key requirements:
7+ years of experience with modern data technologies or distributed systems Expertise in Java and/or Python, strong SQL, Bash, and Linux fundamentals Experience building/operating managed data or infrastructure services with production support Proven ownership of projects from design to deployment and operation Solid understanding of distributed systems trade-offs (fault tolerance, concurrency, partitioning, partial failure recovery) Experience with batch and streaming systems, including state management, retries, replay, and data consistency Experience with data processing/query tech such as Flink, Spark, or ClickHouse and evaluating performance Experience with multiple storage technologies (object storage, relational/NoSQL, Kafka); familiarity with Iceberg or Delta Lake Practical data modeling, schema governance/evolution, and data quality controls; integrating metadata/catalog/lineage Experience with data formats/serialization (Arrow, Parquet, Protobuf, Avro) Cloud-native data infrastructure experience (Kubernetes, workflow orchestration, automated deployment, production monitoring) Strong communication and problem-solving skills in a fast-paced trading environment strong communication problem-solving cross-team collaboration Java Python SQL
Matching similar jobs
JOB OVERVIEW
Experience level
Lead
Location
Brooklyn, NY
Occupation
Data Warehousing Specialists
Industry
Computer Systems Design Services
Posted
today
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)