Skip to main content

About the role

Overview: In this role you will shape a scalable, customizable retrieval platform used by product teams, AI agents, and internal systems. You’ll translate customer needs into reusable platform capabilities, build and operate APIs and services, and drive secure, multi-tenant retrieval at enterprise scale. You will collaborate with product, research engineering, and applied science to bring new retrieval techniques into production. This position combines technical leadership with hands-on delivery to improve retrieval quality and customization. Compensation / Benefits Hybrid work model flexible vacation and work-life balance tuition reimbursement mental health days and Headspace access 401k with company match employee incentive programs
Responsibilities: Translate customer and product needs into reusable platform capabilities in collaboration with product and client teams Design retrieval APIs and search services with consistent contracts, tunable ranking, and low-latency payloads for agent contexts Make retrieval customizable via pluggable components (chunking, enrichment, embedding, ranking, models) accessible through self-service APIs Own data ingestion and indexing on the platform, including streaming pipelines and zero-downtime index changes Partner with Research Engineering and Applied Science to move retrieval techniques from experiment to production-ready capabilities Collaborate with product and client teams to prioritize roadmaps and improve platform adoption Incorporate evaluation into the platform (offline/online evaluation, LLM-as-judge scoring, experiments) Own security posture, multi-tenant isolation, entitlements at query time, and telemetry for user content protection Own SLOs, observability, on-call responsibilities, and tracing for agentic retrieval paths Set architectural direction across retrieval, search services, and APIs; mentor engineers and promote AI-assisted development
Key requirements: 8+ years of software engineering with progression to staff-level scope Experience building search, retrieval, or data-intensive platforms at scale; strong distributed systems fundamentals; Python/Java/Go Experience translating customer needs into platform capabilities across product, engineering, and science Experience designing APIs and services consumed by multiple teams with contracts and versioning considerations Operational rigor: SLOs, observability, on-call ownership, incident learning Fluency with AI-assisted development tools cross-functional collaboration technical leadership mentorship search engines (Vespa, OpenSearch, Elasticsearch) hybrid (dense and sparse) retrieval, ranking, embedding pipelines agentic retrieval patterns and MCP protocols

Matching similar jobs

JOB OVERVIEW

Experience level

Lead

Location

Brooklyn, NY

Occupation

Software Developers

Industry

Software Publishers

Posted

today

Tired of running searches?

Rank the roles you'd take once, and matches like these arrive on their own.

CREATE PROFILE (opens in a new tab)