Senior Software Engineer - Polaris & Data Lake Catalog

Join the Snowflake Team as a Senior Software Engineer

Snowflake is at the forefront of data innovation, enabling organizations to harness the full potential of their data with our AI Data Cloud. We are expanding our capabilities beyond traditional data warehousing to embrace the open data lake house architecture through our Polaris project. This project is an open-source implementation of the Iceberg REST catalog, and we are looking for a Senior Software Engineer to join our Polaris and Data Lake Catalog team.

Role Overview

As a Senior Software Engineer, you will be instrumental in building and evolving our open and interoperable data lake ecosystem. You will tackle complex challenges in distributed systems, contributing to Snowflake’s mission of providing a truly open data lake architecture, free from vendor lock-in.

Key Responsibilities

• Design and Implement Distributed Systems: Develop scalable systems to support Iceberg DML/DDL transactions, schema evolution, partitioning, and more.
• Architect Integration Solutions: Build systems that integrate Snowflake queries with external Iceberg catalogs and various data lake architectures, ensuring seamless interoperability across cloud providers.
• Develop High-Performance Solutions: Create low-latency solutions for catalog federation, enabling customers to manage and query their data lake assets across multiple catalogs from a single interface.
• Collaborate with Open-Source Communities: Work with Snowflake’s open-source team and the Apache Iceberg community to contribute new features and enhance the Iceberg REST specification.
• Enhance Data Access Control: Work on core data access control and governance features for Polaris, including fine-grained permissions such as row-level security and column masking.
• Contribute to Managed Services: Ensure that external query engines like Spark and Trino can read from and write to Iceberg tables through Polaris.
• Automate Data Lake Maintenance: Build tooling and services for data lake table maintenance, including compaction, clustering, and data retention.

Ideal Candidate Profile

• Experience: 8+ years in designing and building scalable, distributed systems.
• Programming Skills: Proficiency in Java, Scala, or C++ with a focus on performance and reliability.
• Knowledge: Deep understanding of distributed transaction processing, concurrency control, and high-performance query engines.
• Open-Source Engagement: Passion for open-source software and community engagement, particularly in the data ecosystem.
• Cloud Experience: Experience building cloud-native services and working with public cloud providers like AWS, Azure, or GCP.
• Data Governance: Familiarity with data governance, security, and access control models in distributed data systems.

Bonus Points

• Experience contributing to open-source projects, especially in the data infrastructure space.
• Designing or implementing REST APIs in distributed systems.
• Managing large-scale data lakes or data catalogs in production environments.
• Working on highly-performant and scalable query engines such as Spark, Flink, or Trino.

Why Join Us?

• Innovative Projects: Be part of a pioneering effort to build the most open and interoperable data lake ecosystem in the industry.
• Collaborative Environment: Work with some of the brightest minds in the data ecosystem, including core contributors to Apache Iceberg.
• Impactful Work: Contribute to a high-impact open-source project that solves real-world data challenges for enterprise customers.

Compensation and Benefits

• Salary Range: €187,000 - €276,000 per year.
• Equity and Bonus: Eligible to participate in Snowflake’s bonus and equity plan.
• Comprehensive Benefits: Medical, dental, vision, life, and disability insurance; 401(k) retirement plan; flexible spending & health savings account; paid holidays; paid time off; parental leave; employee assistance program.

Join us at Snowflake and be part of a team that is pushing the boundaries of data innovation. Apply today to make your impact!