Internal Medicine AI Evaluation Specialist
Weekday AI
- Location
- Remote (United States)
- Compensation
- $130 - $180/hr
- Employment
- Part-time
- Level
- Senior Level
About the Role
Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. This role involves developing realistic clinical scenarios and evaluating AI-generated medical responses to train and improve next-generation AI models.
Skills
Perks
- Remote work
- Flexible schedule
- Weekly payments
Full job details
This role is for one of our clients
Compensation: $130-$180 per hour
Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.
In this role, you will develop realistic clinical scenarios, evaluate AI-generated responses against evidence-based standards, and help improve how AI systems reason about diagnosis, treatment planning, and patient management within Internal Medicine.
We welcome Internal Medicine attending physicians, hospitalists, fellows, final-year residents, and other qualified Internal Medicine professionals who are passionate about applying their expertise to the future of healthcare AI.
Key ResponsibilitiesDesign Clinical Scenarios- Create realistic Internal Medicine case studies based on everyday clinical practice.
- Develop prompts covering:
- Diagnostic reasoning
- Differential diagnosis
- Treatment planning
- Risk stratification
- Evidence-based guideline adherence
- Long-term patient management
- Produce attending physician-level reference solutions that reflect current clinical best practices.
- Ensure recommendations align with established evidence-based guidelines.
- Review and assess AI-generated clinical responses using structured evaluation frameworks.
- Identify reasoning gaps, inaccuracies, omissions, and opportunities for improvement.
- Provide detailed written feedback to improve AI model performance and clinical reliability.
- Participate in onboarding sessions and specialty calibration meetings.
- Work with multidisciplinary teams to ensure consistency, accuracy, and quality across medical evaluations.
Applicants should meet one of the following criteria:
Attending Physicians- Board-certified in Internal Medicine.
- Current, active, and unrestricted medical license.
- Final-year Internal Medicine resident.
- Recent residency graduates must be board-eligible.
- Board-certified or board-eligible in Internal Medicine or a related subspecialty.
- Current, active, and unrestricted medical license.
Professionals with expertise in Internal Medicine subspecialties are encouraged to apply, including:
- Cardiology
- Hematology & Oncology
- Endocrinology
- Nephrology
- Gastroenterology
- Pulmonology
- Other Internal Medicine subspecialties
- Strong knowledge of evidence-based medicine and clinical practice guidelines.
- Experience in clinical teaching, medical research, or quality improvement initiatives.
- Excellent written communication and analytical skills.
- Interest in artificial intelligence, clinical decision support, or healthcare technology.
- Fully remote with 100% asynchronous work.
- Standard commitment of approximately 20 hours per week, with opportunities for additional hours based on project demand.
- Flexible working schedule.
- Weekly payments.
- Rolling onboarding with ongoing engagement opportunities.
- Compensation is determined by clinical experience, expertise, and seniority.
- Help shape the future of AI-powered clinical decision-making.
- Apply your Internal Medicine expertise to improve advanced healthcare AI systems.
- Collaborate with leading AI researchers and experienced clinicians.
- Contribute to high-impact projects that advance safe, accurate, and evidence-based medical AI.
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Contract Information- Independent contractor engagement.
- Fully remote work completed on your own schedule.
- Projects may be extended, modified, or concluded based on business needs and individual performance.
- Work does not involve access to confidential or proprietary information from any employer, client, or institution.
- Weekly payments are processed through supported payment platforms based on approved work completed.
- Please note that visa sponsorship is not available for this opportunity.