Full-Stack Engineer, Agent Evaluation & Reliability (San Francisco, CA, In-Person)
idelsoftMillbrae, CA
Full-Stack Engineer, Agent Evaluation & Reliability (San Francisco, CA, In-Person)
idelsoftMillbrae, CA
APPLY NOW (opens in a new tab)3 days ago
About the role
Full-Stack Engineer, Agent Evaluation & Reliability - SF, CA (In-Person) San Francisco, California, United States
We're hiring a backend-leaning Full-Stack Engineer to build the infrastructure that makes enterprise AI agents reliable. The platform evaluates how agents perform in production, finds what went wrong, and keeps improving both the agents and the knowledge they rely on. Most of the hard problems are in the backend: data modeling, queues, execution pipelines, integrations, observability and debugging distributed workflows. You'll also take features through the full stack and ship the product that makes the system useful. You'll work directly with the founders, the engineering lead and enterprise customers, with real ownership of architecture and product decisions from day one.
What you'll do:
Build and own the backend systems behind an agent evaluation and continuous-improvement platform
Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings and customer-defined quality criteria
Own enterprise integrations end to end: authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, schema changes, retries, monitoring and production support
Take features from technical design through deployment, monitoring, customer feedback and iteration
Work directly with customers to understand their systems, diagnose failures and turn one-off requests into reusable product features
You may be a fit if
You have 2 to 4 years of building and running production software, including time at a fast-paced startup
You have strong backend skills in production TypeScript and Node.js (or Bun) and can ship across the full stack
You know PostgreSQL well and have hands-on experience with Redis, job queues, background workers or workflow systems
You've owned projects and features end to end and pick things up quickly
You understand why evaluating nondeterministic agent systems is different from regular unit testing
You can work in person in San Francisco
Nice to have:
Founding or early engineer experience at a YC or other early-stage startup
APIs, integrations, webhooks and data pipelines for multi-tenant enterprise software
A background in enterprise AI, agent infrastructure, observability or distributed systems
Experience at a high-growth AI startup
What we offer:
Competitive salary and meaningful equity, with flexibility on the mix
Visa sponsorship available
Direct work with the founders and enterprise customers
Or refer someone
Matching similar jobs
JOB OVERVIEW
Experience level
Senior
Location
Millbrae, CA
Occupation
Software Developers
Industry
Custom Computer Programming Services
Posted
3 days ago
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)