About the Role
Stability AI is seeking a Senior Data Platform Engineer to enhance and expand our data platform using AWS and GCP services. This role involves understanding our needs, selecting the appropriate tech stack and tools, gaining buy-in from stakeholders, and choosing the best long-term solutions. As a technical thought leader, you will help hire and mentor other members of the team.
Responsibilities
- Architect, implement, and monitor data pipelines.
- Conduct knowledge-sharing sessions with peers to ensure effective data use.
- Work with team members and leaders to further develop our data strategy.
- Develop and maintain a data catalog for a comprehensive understanding of the data and events within our platform.
- Maintain infrastructure-as-code (AWS CDK) provisioning, spin up new and/or revised data platform resources and environments to enhance data collection and processing from our applications and data sources.
- Act as the primary technical decision-maker for the data platform and help hire and support analytical data personnel (e.g., Data Scientist, Data Analyst, BI Developer).
Technical Experience
- Containerized compute: Kubernetes (EKS, GKS), ECS, Docker
- Data ingestion: Airbyte, Mage AI, AWS Glue
- Pipeline orchestration: Airflow, Mage AI
- Data manipulation: Pandas, Numpy, Polars
- Data lake: S3, Iceberg, Delta Lake
- Data warehouse: BigQuery, Redshift
- Database: PostgreSQL
- Data visualization: Looker, Superset/Preset
- Data quality (optional): Great Expectations, Monte Carlo
- Data catalog (optional): Amundsen, AWS Glue Catalog
- Event streaming (optional): Kafka, Flink, Redis, Pub/Sub
- Infrastructure-as-code: AWS CDK, Terraform
- CI/CD: GitHub Actions
Qualifications
- Must have 7+ years of hands-on experience working with data platforms.
- Experience being involved in the design and implementation of at least one data platform.
- Experience in streaming and/or batch analytics.
- Experience with scalable distributed data processing, management, and visualization tools.
- Experience using business intelligence tools and data frameworks.
- Strong working knowledge of Python for both provisioning cloud infrastructure and working with data.
Compensation
The salary range for this role is between $130,000 and $190,000. Individual pay within the range is based on factors like job-related skills and experience. Total compensation also includes stock options and benefits.
Equal Employment Opportunity
We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or other legally protected statuses.
Benefits Extracted with AI
- Remote work flexibility
- Stock options
Similar jobs
Last update: 23 minutes ago
Staff Software Engineer
Join Aiven as a Staff Software Engineer to develop cloud operations platforms using open-source technologies. Hybrid work in Berlin.
Cloud Data Engineer
Seeking a Cloud Data Engineer with expertise in AWS, Python, and CI/CD for a hybrid role in Hannover. Join our dynamic team!
Staff Software Engineer, Data Platform
Join Personio as a Staff Software Engineer in Berlin to build scalable data platforms using Kafka, Kubernetes, and AWS. Drive innovation and excellence.
Senior Software Engineer - AWS, Python, Ruby on Rails
Join HeyJobs as a Senior Software Engineer to design scalable systems using AWS, Python, and Ruby on Rails in a dynamic team.
Senior Backend Engineer (Go)
Join Nebius as a Senior Backend Engineer (Go) to develop fault-tolerant cloud services in a hybrid work environment.
Senior DevOps Engineer with Linux, Kubernetes, and GCP
Join Redcare Pharmacy as a Senior DevOps Engineer to enhance infrastructure efficiency using Linux, Kubernetes, and GCP.
Senior Backend Engineer - Java, Rust, Go
Join Together AI as a Senior Backend Engineer in Amsterdam. Work with Java, Rust, and Go to build scalable backend systems.
Senior Python Developer with AWS Experience
Join Basetime BV as a Senior Python Developer to develop and maintain AWS cloud solutions. Hybrid work, competitive salary, and growth opportunities.
Senior GCP Data Engineer (Databricks)
Join Xebia Poland as a Senior GCP Data Engineer, focusing on Databricks, Python, and SQL for cloud-based solutions.
Senior Product Engineer [Rust & Typescript]
Join Attio as a Senior Product Engineer working with Rust & TypeScript to build innovative CRM features. Remote work available.
Senior Software Engineer (Node.js & TypeScript)
Join n8n as a Senior Software Engineer to build AI applications using Node.js and TypeScript. Remote role within Europe.
Senior Software Engineer - Python, Apache Kafka
Join Aiven as a Senior Software Engineer in Berlin, focusing on Python and Apache Kafka in a hybrid work environment.
Senior Full Stack Engineer - Climate Tech - Rust & TypeScript
Join Climatiq as a Senior Full Stack Engineer to develop climate tech solutions using Rust and TypeScript. Remote work available.
Senior Cloud Platform Engineer
Join Strava as a Senior Cloud Platform Engineer in Berlin, working on cloud infrastructure, containerization, and observability tools.
Full-Stack Engineer with Angular and React.js
Join Labelbox as a Full-Stack Engineer to develop scalable systems using Angular, React.js, and GraphQL. Work remotely in a dynamic AI-driven environment.
Software Engineer - Data Platform
Join Bot Auto as a Software Engineer to design and evolve our hybrid-Cloud data platform. Work remotely with cutting-edge technology in autonomous trucking.
Senior Systems Engineer, Managed Operations
Join AWS as a Senior Systems Engineer in Berlin to lead operations for the European Sovereign Cloud, ensuring high-availability AWS services.
Senior Software Engineer - Backend Development
Join Sysdig as a Senior Software Engineer to develop scalable backend services using Go, RESTful APIs, and microservices in a hybrid work environment.
Cloud Engineer
Join our team as a Cloud Engineer in Cologne, Germany. Work with GCP, Azure, and serverless computing in a dynamic, hybrid environment.
Expert Machine Learning Engineer
Join Dataroots as an Expert Machine Learning Engineer to design and deliver AI-powered solutions, focusing on machine learning models.
Senior Backend Engineer C++
Join DeepL as a Senior Backend Engineer C++ to design and maintain scalable backend services using C++ and AI technologies.
Senior Cloud DevOps Engineer
Join netgo as a Senior Cloud DevOps Engineer in Berlin. Work with Kubernetes, GitOps, and more in a dynamic team environment.
Software Engineer - Cloud Applications and Python
Join Topicus as a Software Engineer in Arnhem to develop cloud applications using Python, REST APIs, and ETL processes for healthcare data services.
AI Engineer
Join BCG X as an AI Engineer in Milan, Italy. Develop AI solutions, partner with clients, and drive innovation in a dynamic environment.
