About the Role
As a Senior Prompt Engineer at Scale AI, you will be at the forefront of developing and implementing advanced LLM applications. Your main responsibilities will include evaluating data quality, solving complex problems related to synthetic data capabilities, and executing robust business solutions. You will also be involved in 'jailbreaking' and finding vulnerabilities in LLMs and other generative AI models, developing training materials for red teaming and adversarial analysis, and conducting training with operational staff.
Responsibilities
- Develop LLM applications to evaluate data quality and solve complex problems.
- 'Jailbreak' and find vulnerabilities in LLMs and other generative AI models.
- Develop training materials for red teaming and adversarial analysis.
- Conduct training with operational staff.
- Develop data creation and vulnerability testing scripts in Python/TypeScript.
- Work with ML Engineers and software engineers for robust implementation and in-platform deployment.
- Analyze and visualize experimental prompt results.
- Develop and teach new prompt engineering methods to ML engineers and software engineering staff.
Qualifications
- B.S. or higher in a quantitative major or one related to language models (linguistics, philosophy, etc.).
- Proficiency writing Python or TypeScript for API-enabled scripts.
- Ability to conduct analysis of model performance via statistics and data visualization.
- Experience using frontier-scale LLMs (GPT, Claude, PaLM) programmatically towards specific applications or implementations.
- Demonstrated ability to have built an LLM-based application or hacking an LLM to serve some novel purpose.
- Familiarity with LLM evaluation metrics (BLUE, ROUGE, MAUVE, etc.) and statistics.
Nice to Haves
- Previous experience with AI/LLM developmental work or red teaming a strong plus.
- Academic work in language models, particularly on adversarial robustness.
- Previous disclosures of vulnerabilities or security issues in LLMs or other AI models.
- Experience working with cloud technology stack (e.g., AWS or GCP) and developing machine learning models in a cloud environment.
Compensation and Benefits
Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The salary range for this full-time position in locations like San Francisco, New York, Seattle is $140,000 — $210,000 USD. Scale employees in eligible roles are also granted equity-based compensation, subject to Board of Director approval. Additionally, this role may be eligible for additional benefits such as a commuter stipend.
Benefits Extracted with AI
- Comprehensive health, dental, and vision coverage
- Retirement benefits
- Learning and development stipend
- Generous PTO
- Commuter stipend
Similar jobs
Last update: 23 minutes ago
Prompt Engineer with Python and TypeScript
Join Scale AI as a Prompt Engineer in San Francisco, focusing on LLMs and generative AI with Python and TypeScript.
Senior Software Engineer, Generative AI
Senior Software Engineer role at Scale AI, focusing on Generative AI technologies, requiring skills in Python, Node.js, React, and more.
Lead Software Engineer, Generative AI
Lead Software Engineer role at Scale AI, focusing on generative AI, requiring skills in Python, Next.js, and AI technologies.
Senior Software Engineer, GenAI Safety & Evaluation
Senior Software Engineer role in AI safety and evaluation, requiring skills in AI, backend development, and system performance.
Senior Software Engineer - Generative AI Operator
Senior Software Engineer for Generative AI, focusing on building advanced LLMs and generative models with technologies like MongoDB, React, and Node.js.
Senior Software Engineer, Generative AI
Senior Software Engineer for Generative AI at Scale AI, focusing on backend development and AI technologies.
Software Engineer, GenAI Growth
Join Scale AI as a Software Engineer in GenAI Growth, developing AI applications with a dynamic team in NYC.
Senior Fullstack Software Engineer, GenAI Horizontal Task Tooling
Join Scale AI as a Senior Fullstack Software Engineer to build web-based applications for AI data annotation.
Senior Software Engineer, GenAI Safety & Evaluation
Join Scale AI as a Senior Software Engineer in GenAI Safety & Evaluation, shaping AI model evaluation.
Contributor Engagement Lead, Generative AI
Lead initiatives in Generative AI, manage communications, and drive strategy at Scale AI in San Francisco. Experience with AI and LLMs preferred.
Manager, Machine Learning Research Engineer, Generative AI
Lead a team in developing generative AI models at Scale AI, focusing on LLMs and MLOps.
Senior Prompt Engineer
Join Accrete AI as a Senior Prompt Engineer to design and optimize prompts for AI agents, enhancing NLP applications.
Senior Software Engineer, GenAI Model Evaluation
Join Scale AI as a Senior Software Engineer in GenAI Model Evaluation, focusing on AI model safety and performance.
Senior Software Engineer, Machine Learning Infrastructure
Join Scale AI as a Senior Software Engineer in Machine Learning Infrastructure, focusing on backend system design and ML Infrastructure.
Senior Software Engineer - Frontend, Generative AI
Senior Software Engineer for Frontend in Generative AI at Scale AI, focusing on React.js, Node.js, and performance optimization.
Fullstack Software Engineer, GenAI Quality
Fullstack Software Engineer role focusing on GenAI Quality, involving end-to-end feature development and system design.
Staff AI Product Manager, Generative AI
Join Scale AI as a Staff AI Product Manager to lead the development of ML-powered products in a hybrid role in San Francisco.
Software Engineer, GenAI Growth
Join Scale AI as a Software Engineer in GenAI Growth, focusing on AI applications and platform growth.
Engineering Manager, Generative AI
Lead a team in Generative AI at Scale AI, focusing on engineering management and technical leadership.
Director of Contributor Growth
Lead contributor growth at Scale AI, driving Generative AI business with strategic leadership and operational excellence.
Senior Fullstack Software Engineer, GenAI Allocation
Senior Fullstack Engineer role focusing on AI applications, requiring AWS, OOP, and problem-solving skills.
Chief of Staff, Generative AI
Chief of Staff role at Scale AI, focusing on Generative AI and Large Language Models, offering a comprehensive benefits package.
Senior Fullstack Software Engineer, Pay & Incentives
Senior Fullstack Engineer role focusing on payment systems, requiring skills in both front-end and back-end development.
Director of Contributor Growth - Generative AI
Lead the growth of contributor workforce in Generative AI, driving strategies and operational excellence at Scale AI.
