Senior Data Platform Engineer
Stability AIAbout the Role
Stability AI is seeking a Senior Data Platform Engineer to enhance and expand our data platform using AWS and GCP services. This role involves understanding our needs, selecting the appropriate tech stack and tools, gaining buy-in from stakeholders, and choosing the best long-term solutions. As a technical thought leader, you will help hire and mentor other members of the team.
Responsibilities
- Architect, implement, and monitor data pipelines.
- Conduct knowledge-sharing sessions with peers to ensure effective data use.
- Work with team members and leaders to further develop our data strategy.
- Develop and maintain a data catalog for a comprehensive understanding of the data and events within our platform.
- Maintain infrastructure-as-code (AWS CDK) provisioning, spin up new and/or revised data platform resources and environments to enhance data collection and processing from our applications and data sources.
- Act as the primary technical decision-maker for the data platform and help hire and support analytical data personnel (e.g., Data Scientist, Data Analyst, BI Developer).
Technical Experience
- Containerized compute: Kubernetes (EKS, GKS), ECS, Docker
- Data ingestion: Airbyte, Mage AI, AWS Glue
- Pipeline orchestration: Airflow, Mage AI
- Data manipulation: Pandas, Numpy, Polars
- Data lake: S3, Iceberg, Delta Lake
- Data warehouse: BigQuery, Redshift
- Database: PostgreSQL
- Data visualization: Looker, Superset/Preset
- Data quality (optional): Great Expectations, Monte Carlo
- Data catalog (optional): Amundsen, AWS Glue Catalog
- Event streaming (optional): Kafka, Flink, Redis, Pub/Sub
- Infrastructure-as-code: AWS CDK, Terraform
- CI/CD: GitHub Actions
Qualifications
- Must have 7+ years of hands-on experience working with data platforms.
- Experience being involved in the design and implementation of at least one data platform.
- Experience in streaming and/or batch analytics.
- Experience with scalable distributed data processing, management, and visualization tools.
- Experience using business intelligence tools and data frameworks.
- Strong working knowledge of Python for both provisioning cloud infrastructure and working with data.
Compensation
The salary range for this role is between $130,000 and $190,000. Individual pay within the range is based on factors like job-related skills and experience. Total compensation also includes stock options and benefits.
Equal Employment Opportunity
We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or other legally protected statuses.
Benefits Extracted with AI
- Remote work flexibility
- Stock options
Similar jobs
Last update: 23 minutes ago
Remote Data Engineer - Research
Join Stability AI as a Remote Data Engineer to build scalable data infrastructure for AI models.
Lead Architect - Gen AI API Platform
Lead Architect for Gen AI API platform, focusing on AWS, REST APIs, and AI/ML infrastructure. Remote role with competitive salary.
Senior Backend Engineer (AI)
Join Stability AI as a Senior Backend Engineer to develop REST APIs and AI/ML services for Generative AI models.
Senior Data Engineer
Join us as a Senior Data Engineer in Lisbon to design and maintain data infrastructure. Hybrid role with flexible benefits.
Senior Cloud Engineer
Join as a Senior Cloud Engineer to architect and deploy cloud solutions using Azure, AWS, and GCP. Lead innovation in cloud technology.
Senior Python Engineer (Cloud Platform)
Join Bonfy.AI as a Senior Python Engineer to build and maintain a cloud-based SaaS platform using Python and AWS.
Senior Machine Learning Engineer
Join as a Senior Machine Learning Engineer to design and deploy advanced ML solutions using Python, Spark, and cloud platforms. Remote work opportunity.
Senior ML Infrastructure Engineer
Join CHAI: AI Platform as a Senior ML Infrastructure Engineer to build and scale ML systems in Palo Alto.
Senior AI Engineer - Backend
Join Keboola as a Senior AI Engineer to enhance AI features, develop models, and collaborate on innovative projects in Prague.
Senior Software Engineer, Backend
Join Standard AI as a Senior Backend Engineer to design scalable microservices and APIs. Remote role with competitive salary and benefits.
Senior Data Scientist, Analytics - Performance & Quality
Join as a Senior Data Scientist focusing on analytics, performance, and quality in a remote role with competitive salary and benefits.
Senior Fullstack Software Engineer, GenAI Horizontal Task Tooling
Join Scale AI as a Senior Fullstack Software Engineer to build web-based applications for AI data annotation.
Cloud Data Engineer
Join NPO as a Cloud Data Engineer to enhance data platforms using GCP, Python, and more. Flexible hours and growth opportunities.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok. Enhance scalability and efficiency using Kubernetes, Spark, and more.
Founding Data Engineer
Join GovWell as a Founding Data Engineer to build scalable data infrastructure for modernizing government services.
Machine Learning Engineer with AI/ML Experience
Join us as a Machine Learning Engineer to develop AI/ML models and applications. Work remotely with top-tier companies.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok. Enhance scalability and efficiency using Kubernetes, Spark, and more. Relocation provided.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok, focusing on scalability, stability, and efficiency. Relocation provided.
Senior Software Engineer, Machine Learning
Join as a Senior Software Engineer in Machine Learning, working remotely to build ML-driven products for user engagement.
Senior Data Engineer
Join Stability AI as a Senior Data Engineer to build scalable data infrastructure for AI models. Remote work from Germany.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok, focusing on scalability and efficiency using Kubernetes, Spark, and cloud technologies.
Applied Scientist, Artificial General Intelligence
Join AWS as an Applied Scientist in Artificial General Intelligence, driving AI innovation in cloud computing.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok. Enhance scalability and efficiency using Kubernetes, Spark, and cloud technologies.
Data Engineer II
Join Accolade as a Data Engineer II in Prague. Design and maintain cloud-native data infrastructure using AWS and modern technologies.