Research Scientist, Human-AI Interaction

About The Team

OpenAI's Human Data Team creates custom data solutions driving groundbreaking research. Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-4, and Sora, and contributes to safety initiatives through collaboration with our Preparedness and Safety Systems teams.

We work with AI trainers to gather specialized data for training and evaluating our models across modalities such as video, audio, text, and tool actions. Our goal is to develop scalable methods, tools, and platforms to generate and evaluate high-quality data from both synthetic sources and human experts in various fields, including mathematics, sciences, creative writing, programming, art, and safety. We leverage OpenAI models to improve and streamline our data collection and quality processes.

About The Role

In this role, you will work side by side with research teams to iteratively design and evaluate new methods for collecting high quality human data. You’ll use your knowledge of human cognition and data quality to co-design experiments, write instructions and assess the efficacy of our data collections. In parallel, you’ll partner with engineering and operations to implement changes and iteratively improve how we run human data campaigns.

This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.

Responsibilities

• Collaborate closely with researchers to understand, predict, and design tasks and quality rubrics to collect great human data.
• Engage with data labelers and vendors to evolve training and feedback mechanisms.
• Create hiring assessments and brainstorm acquisition strategies for high skill data labelers.
• Design and conduct experiments to assess the impact of interventions on human data quality.
• Come up with creative strategies for collecting and assessing high-quality data.
• Generate golden datasets and evaluation techniques.
• Work on multiple projects with tight timelines.
• Do whatever needs to be done to make our models better.

You Might Thrive In This Role If You

• Have a background in cognitive science, computational linguistics, human-computer interaction.
• Enjoy tackling big questions in an ambiguous and fast-paced environment.
• Love being at the intersection of research and operational execution.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.