Machine Learning Engineer – GPU Performance & Low-Latency Inference
fintal partnersNew York, NY
Machine Learning Engineer – GPU Performance & Low-Latency Inference
Industries
Computer Systems Design ServicesCustom Computer Programming ServicesElectronic Computer ManufacturingAbout the role
We’re partnering with a leading global trading firm to hire a Machine Learning Engineer to make ML models run as fast as possible. You’ll work close to the hardware, squeezing latency out of real-time inference and speeding up training, in an environment where microseconds matter.
What you’ll do:
- Build and optimize low-latency inference pipelines that deliver real-time predictions in production
- Profile and optimize ML workloads at the device level: memory, compute, kernels and compilers
- Write and tune custom GPU code (CUDA, Triton or equivalent) where libraries fall short
Apply and validate low-precision techniques (FP8/INT8, quantization), knowing which optimizations stay bit-exact, Optimize batching, KV caching and serving paths for latency-critical workloads.
Extend the internals of PyTorch and inference frameworks to raise performance on specific workloads
What you’ll bring 3+ years in machine learning, focused on inference or GPU performance
Hands-on experience optimizing ML at the GPU level: CUDA, Triton, C++, TensorRT or similar
Experience with profiling tools (Nsight, torch.profiler) and reasoning about memory bandwidth, latency and throughput
A strong grasp of modern ML architectures, attention in particular, and how they map onto hardware
Production-quality Python (C++ a strong plus), with testing discipline and ownership of the systems you ship
A habit of measuring before optimizing, and the judgment to know when to build versus buy
Nice to have:
- Experience building real-time, low-latency prediction systems in high-performance environments
- Experience with numerics: mixed precision, quantization and correctness at scale
- Contributions to open-source projects such as PyTorch, vLLM, Triton or Flash
AttentionNo trading experience is required. What matters is deep, hands-on performance engineering and a drive to make things faster.
Matching similar jobs
JOB OVERVIEW
Experience level
Lead
Location
New York, NY
Occupation
Computer Systems Engineers/Architects
Industry
Computer Systems Design Services
Posted
2 days ago
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)