Software Engineer II, Data Engineering

Job Overview

Join GitHub as a Software Engineer II on the Copilot Metrics team, where you'll be at the forefront of data engineering. This role is pivotal in designing, developing, and maintaining efficient and reliable data pipelines. You'll collaborate with stakeholders across the company to gather business requirements, build data models, and ensure data quality and accessibility. Your expertise in Python, SQL, Airflow, and Spark will be crucial in optimizing our data infrastructure and enabling data-driven decision-making.

Responsibilities

Data Pipeline Development

• Design, build, and maintain scalable data pipelines using Python, SQL, Airflow, and Spark.

Business Requirements Gathering

• Collaborate with stakeholders to understand and translate business requirements into technical specifications.

Data Modeling

• Develop and implement data models that support analytics and reporting needs, ensuring alignment with business goals.

Data Quality and Governance

• Ensure data accuracy, consistency, and reliability by implementing robust data validation and quality checks.

Stakeholder Collaboration

• Work with cross-functional teams, including data analysts, data scientists, and business leaders, to deliver high-quality data solutions.

Performance Optimization

• Continuously monitor and optimize data pipelines for performance, scalability, and cost-efficiency.

Monitoring and Observability

• Build and implement monitoring and observability metrics to ensure data quality and detect anomalies in data pipelines.

Documentation and Communication

• Maintain clear and comprehensive documentation of data processes and effectively communicate technical concepts to non-technical stakeholders.

Qualifications

Required

• 2+ years of experience in Software Engineering, Computer Science, or related technical discipline.
• Proven experience maintaining production software coding in languages such as C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python.
• 2+ years of experience in data engineering or analytics engineering roles.
• Strong proficiency in Python, SQL, Airflow, and Spark.
• Extensive expertise in building and maintaining robust data pipelines and ETL processes.

Preferred

• Familiarity with Go and Ruby.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
• Knowledge of data governance and data security best practices.
• Excellent verbal and written communication skills.
• Proven ability to work effectively in a collaborative, cross-functional environment.

Compensation

The base salary range for this job is USD $75,000.00 - USD $198,900.00 per year. Additional benefits include annual bonuses and stock options, with opportunities for sales incentives based on revenue or utilization.

About GitHub

GitHub is the world’s leading AI-powered developer platform with 100 million developers and counting. We’re also home to the biggest open-source community on earth. At GitHub, our goal is to create the space you need to do your best work. We’re remote-first and offer competitive pay, generous learning and growth opportunities, and excellent benefits to support you, wherever you are.

Join us, and let’s change the world, together.