Lead Data Engineer

DDSI Lead Data Engineer

Lake Buena Vista, United States

Department/Group Description:

The Disney Decision Science and Integration (DDSI) analytics consulting team is responsible for supporting clients across The Walt Disney Company including Direct-to-Consumer & International, Media Networks (e.g., ABC, ESPN), Studio Entertainment (e.g., The Walt Disney Studios, Disney Theatrical Group) and Parks, Experiences & Consumer Products. DDSI leverages technology, data analytics, optimization, statistical and econometric modeling to explore opportunities, shape business decisions and drive business value. 

The DDSI Data Engineering (DE) team is seeking a Lead Data Engineer to work on Media Ad Sales projects. This position will work with the Decision Science Products and Science teams to design and implement systems in the Disney Ad Sales and Viewership Forecasting space. The work will involve various data engineering activities such as data acquisition and validation, designing and implementing ETL/ELT data pipelines, and designing and implementing databases.

Responsibilities of the Role:

Work assignments may cover activities such as data requirements gathering, source-to-target mapping, data validation scripting and review, developing and monitoring ETL/ELT data pipelines, designing and implementing database schema/tables/views, and producing datasets as input to science models and visualizations.  In addition, this role will help lead on planning data engineering related projects, which includes participating in weekly project meetings, status updates, define/run sprint activities, consult and collaborate with project team members, lead design reviews, ensure standards are followed.  

Technologies generally leveraged to fulfill the work include but not limited to SQL, Python, Docker, Gitlab, Airflow, Lambda, Snowflake, and PostgreSQL.

Basic Qualifications:

• 7+ years overall experience in a software development capacity using a multi environment (Dev, QA, Prod, etc.) and DEVOP procedures for code deployment/promotion

• 2+ years using, designing and building relational databases (preferably Snowflake or PostgreSQL)

• 2+ years of experience leading and deploying code using a source control product such as GitLab/GitHub

• 2+ years of experience with job scheduling software like Apache Airflow, Amazon MWAA, GitLab Runners or UC4

• Proven experience working with large datasets and big data technologies

• Experience with defining and estimating level-of-effort data engineering activities

• Experience with project and sprint planning

• Experience translating project scope into data engineering tasks

• Experience defining solutions to sophisticated data engineering problems in support of advanced analytic processes

• Experience collaborating with multiple project teams in a fast-paced environment

• Ability to communicate technical concepts and solutions to non-technical team members

• Experience designing and building data structures to support requirements

• Multiple years of experience with ELT/ETL data pipeline development and maintenance

• Multiple years of shown experience and expertise using SQL and Python

• Experience using containerization technologies such as Docker or Kubernetes

• Knowledgeable on cloud architecture and product offerings, preferably AWS

Desired Qualifications:

• Business knowledge of Media Ad Sales and viewership data is a plus

Education:

• Bachelor's degree (Computer Science, Mathematics, Software Engineering or related field, or equivalent experience)

• Master's degree preferred (Computer Science, Mathematics, Engineering or related field preferred