Lead Data Engineer

Job Summary

Join Disney Entertainment & ESPN Technology as a Lead Data Engineer, where you will play a pivotal role in building and maintaining content metadata and performance pipelines. Collaborate closely with business, analytics, and engineering teams to create and maintain user-friendly data structures that streamline reporting and monitor key performance indicators. Your efforts will support executive reporting, analytics, data science, and personalization, crucial for achieving organizational business objectives.

Responsibilities

• Lead the design and development of Data Products and Data Warehouses focused on Content Performance and Engagement data.
• Design and develop scalable data warehousing solutions, building ETL pipelines in Big Data environments (cloud, on-prem, hybrid).
• Support data pipelines within AWS, Snowflake, Databricks, Spark, and Airflow.
• Architect data solutions/frameworks and define data models for the underlying data warehouse and data marts.
• Collaborate with Data Product Managers, Data Architects, and Data Engineers to design, implement, and deliver successful data solutions.
• Maintain detailed documentation of your work and changes to support data quality and governance.
• Ensure high operational efficiency and quality of your solutions to meet SLAs and support commitment to our customers (Data Science, Data Analytics teams).
• Be an active participant and advocate of agile/scrum practices to ensure health and process improvements for your team.
• Serve as an advanced resource to other Data Engineers on the team, mentoring and coaching junior members to improve their skills, knowledge, and productivity.
• Participate in on-call rotations.

Basic Qualifications

• 7+ years of data engineering experience modeling and developing large data pipelines.
• Strong SQL skills and ability to create queries to extract data and build performant datasets.
• Hands-on experience with distributed systems such as Spark, Hadoop (HDFS, Hive, Presto, PySpark) to query and process data.
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks).
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices.
• Problem solver with strong attention to detail and excellent analytical and communication skills.

Preferred Qualifications

• In-depth experience with Cloud technologies like AWS (S3, ECS, EMR, EC2, Lambda, etc.).
• Solid experience with data integration toolsets (i.e., Airflow) and writing and maintaining Data Pipelines.
• Good scripting skills, including Python and Bash.
• Familiar with Scrum and Agile methodologies.
• 7+ years of programming experience using Java/Scala, Python, or similar languages.

Required Education

• Bachelor’s Degree in Computer Science, Information Systems, or related field or equivalent work experience.

Why Disney?

At Disney Entertainment & ESPN Technology, we’re reimagining ways to create magical viewing experiences for the world’s most beloved stories while transforming Disney’s media business for the future. Whether evolving our streaming and digital products in new and immersive ways, powering worldwide advertising and distribution, or delivering Disney’s unmatched entertainment and sports content, every day is a moment to make a difference to partners and to hundreds of millions of people around the world.