Research Scientist - Model Evaluation Job at Lumicity, Santa Rosa, CA

eUtyMlk4UUZ6Q2VwZUxEeEZjeWhxdEovVnc9PQ==
  • Lumicity
  • Santa Rosa, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Cassia CONNECT

Nursing Assistant - Certification Provided Job at Cassia CONNECT

: Provide hands-on personal care Assist with nurse-delegated tasks such as recording vital signs and operating mechanical lifts. Enhance residents' quality of life through communication, interaction, and support with daily activities + more! All-inclusive Comprehensive... 

Kyyba Inc.

Occupational Health Nurse Job at Kyyba Inc.

Occupational Health Occupational Health Nurse -School nurses also work well for the position Moving between Plymouth and Minnetonka MN sites 34287 38-42.00 an hour w2 contract onsite 04/08/2024 to 04/07/20257+years experience Education: Bachelor's degree and... 

SCA Health

Credentialing Coordinator - Credentialing Resource Office Job at SCA Health

 ...hospital policies and procedures, while maintaining internal controls and protecting the hospitals assets. The credentialing coordinator works closely with the hospital leadership to ensure the proper credentialing of physicians, allied health practitioners while managing... 

Apex Systems

Electrician Job at Apex Systems

 ...Title: Electrician Location: Landover, MD Training: Monday-Friday, 6am to 2:15pm After Training: Sunday-Thursday or Monday-Friday Shift: 3rd, 9:30pm to 6am (MUST BE OPEN TO THIS SHIFT) You will be held accountable for: Troubleshooting, inspecting,... 

Keyes Coverage Insurance Services

Personal Lines Insurance Producer Job at Keyes Coverage Insurance Services

 ...Murray Keyes and his sons, Carey and Kenneth. Since then, this third-generation, family-owned agency has grown into one of the leading insurance agencies in South Florida. The agency specializes in three main areas of practice including Personal Lines Insurance, Employee...