AI Bioinformatics Benchmarking Engineer

The Position

The Impact

• Build and run automated evaluation pipelines for AI outputs end-to-end.
• Expand regression datasets that cover variant normalization, transcript ambiguity, and disease-level mapping.
• Execute large-scale validation runs across curated VCF datasets, track performance over time, and surface accuracy regressions the moment they appear.
• Grow a structured question bank using DV3 and related datasets, encoding the edge cases that matter most.
• Maintain ground truth datasets for pathogenic and benign variants so our benchmarks reflect real-world genomic complexity.
• Ship automated regression tests for every AI release and build dashboards that make variant-level and case-level accuracy visible to the whole team.
• Partner with engineering to wire validation directly into CI/CD, and diagnose and document failure modes as they emerge.

Dominant and Expressed Traits

• Degree in Bioinformatics, Computational Biology, Genetics, or a related field.
• Deep, hands-on experience with VCFs and real-world genomic datasets.
• Fluency with ClinVar, dbSNP, HGVS standards, and transcript mapping.
• A track record building evaluation frameworks or testing pipelines, Langfuse or similar.
• Experience evaluating LLM-based systems and standing up automated QA at scale.
• Comfort living inside modern data engineering workflows.