Evaluation Engineer
reinforced learningTonawanda, NY
Evaluation Engineer
reinforced learningTonawanda, NY
yesterday
Industries
Software PublishersOther Scientific and Technical Consulting ServicesCustom Computer Programming ServicesAbout the role
The role:
Research designs evaluations; you make them run reliably. You’ll build the harnesses, graders and reporting that let a model be evaluated on hundreds of tasks across our environments, with results anyone can reproduce.
What you’ll own
The evaluation harness: task loading, environment orchestration, model adapters, scoring.
Graders and verifiers implemented with the environment and verifier engineers.
Runs that are deterministic, resumable and cheap to repeat.
Dashboards and reports that researchers and customers read.
Discrepancies between runs, chased until they’re explained.
What you bring
Strong software engineering in Python with an eye for reliability.
Model APIs, batch systems and asynchronous processing.
You care about reproducibility and know how to achieve it.
Comfort reading research code and making it production-grade.
Helpful, not required
Open evaluation frameworks.
Containers and cloud infrastructure.
Matching similar jobs
JOB OVERVIEW
Experience level
Lead
Location
Tonawanda, NY
Occupation
Software Developers
Industry
Software Publishers
Posted
yesterday
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)