Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

eUtyMlk4UUZ6Q2VwZUxEeEZjeWhxdEovVnc9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Penguin Random House LLC

Assistant Marketing Director, Ballantine Bantam Dell (Hybrid) Job at Penguin Random House LLC

 ...The marketing team at Ballantine Bantam Dell is seeking an experienced, creative, detail-oriented and strategic marketer to join our innovative team as Assistant Director of Marketing. This position reports to the Director of Marketing and includes managing a direct... 

Yoh, A Day & Zimmermann Company

***ONLY W2/NO C2C***Camunda Developer Hybrid MA NH W2 Only Job at Yoh, A Day & Zimmermann Company

Please feel free to send your updated resume to ****@*****.*** Rekhu Chhetri, Sr. Recruiter, YOH-Day & Zimmerman Inc LinkedIn Profile: Camunda Developer Hybrid MA NH W2 Only Hybrid Boston MA Nashua NH area W2 Only Experience with Camunda...

H. Theophile

Mechanical Engineer - Design and Manufacturing Job at H. Theophile

Company Description H. Theophile designs and manufactures door and cabinet hardware which is the leading choice for top architects and designers globally. Our New York studio works with projects that range from one-off, highly engineered hardware solutions to historically...

Vista Prairie Communities

Part-Time Housekeeper Job at Vista Prairie Communities

 ...Start a new career as a Part-Time Housekeeper at Red Cedar Canyon, a Senior Living Facility! Join Vista Prairie Communities and elevate your career while making a meaningful impact. Enjoy our supportive culture, outstanding benefits, and the opportunity to build lasting... 

Rock Health

Weekend Nurse Job at Rock Health

We are seeking a Full-Time Weekend Nurse (36 hours) to join our hospice and palliative care team, working Saturdays, Sundays, and Mondays from 9 am to 9 pm. The role involves providing care in patients' homes and residential facilities in Greater Boston and parts of Metrowest...