LLM Red Team Specialist

Join the LLM Red Team at Google DeepMind

About the Role

Google DeepMind is seeking a creative and skilled LLM Red Team Specialist to join our team in Mountain View, California. This role involves working at the forefront of AI safety and security, focusing on identifying and mitigating risks associated with large language models (LLMs) and other generative models. You will be responsible for discovering and evaluating vulnerabilities in our AI systems, enabling other teams to implement approaches that mitigate these risks.

Key Responsibilities

• Identify Vulnerabilities: Expose issues in existing and future LLMs and other generative models related to risk areas such as deceptive content, toxicity, bias, plagiarism, weapons development, cyber attacks, and other criminal activities.
• Develop and Evaluate Attacks: Create and evaluate attacks on LLMs for various threat models, including adversarial attacks, jailbreaks, and prompt injection attacks.
• Model Analysis: Evaluate models for falsehoods, verbal manipulation, and dangerous scientific capabilities.
• Collaboration: Work closely with ML and SWE engineering teams to develop vulnerability testing scripts in Python and share insights for internal development.
• Research and Development: Develop new prompt engineering methods and support the development of scalable tooling in collaboration with other technical staff.
• Training and Documentation: Create training materials for future red teaming efforts and adversarial analysis.

About You

We are looking for individuals who thrive in ambiguity and are willing to tackle any challenge that moves prototypes forward. Flexibility and adaptability are crucial as we often need to invent novel solutions and change course if ideas don’t work.

Required Skills and Experience

• Educational Background: BSc/BA, MSc, or PhD/DPhil in computer science, mathematics, applied statistics, machine learning, or similar experience in the industry.
• Technical Expertise: Proven knowledge and experience with Python and machine learning.
• Analytical Skills: Ability to conduct analysis of model performance via statistics and data visualization.

Preferred Skills

• Academic Work: Experience in language models, particularly on adversarial robustness.
• Security Experience: Previous disclosures of vulnerabilities or security issues in LLMs.
• ML Frameworks: Experience with TensorFlow, PyTorch, or JAX.
• Cross-functional Collaboration: Prior experience collaborating with researchers.

Benefits

At Google DeepMind, we offer a range of benefits to support you and your family, including enhanced maternity, paternity, adoption, and shared parental leave, private medical and dental insurance, and flexible working options. We also provide excellent facilities such as healthy food, an on-site gym, and more.

Application Process

If you are passionate about AI safety and security and have the skills and experience we are looking for, we encourage you to apply. Please note that any offer of employment will be conditional on the results of a background check.

Join us in advancing the state of the art in artificial intelligence and ensuring its safe and ethical deployment.