The Walt Disney Company is known for creating world-class immersive digital experiences. They are seeking a Senior Data Engineer to build and operate a near-real-time data streaming platform to enhance guest experiences by delivering data products efficiently and reliably.

Responsibilities:

Contribute to the design, building, and supervision of technology architecture, solutions, and software dedicated to capturing, managing, and using both structured and unstructured data from various sources
Develop efficient processes and structures to streamline data flow, routing, and storage, adhering to both business and technical specifications, and leveraging cloud and local storage options as required
Apply technical tools and programming skills for the purposes of data cleansing, organization, transformation, and upkeep, integrating automation and advanced methodologies such as big data, artificial intelligence, and machine learning
Uphold design standards and quality assurance protocols for the development of software and systems, ensuring seamless data compatibility, functionality, and integrity across all data connections, transmissions, and storage platforms
Evaluate internal and external business and product requirements to ensure alignment of data operations and endeavors with overarching organizational objectives and priorities
Develop data pipelines and shared monitoring infrastructure that power observability, pipeline health, and data quality for the broader team
Create infrastructure data products (cost tracking, resource utilization, audit logs)
You will be expected to stay up to date with emerging technologies
May be required to provide support escalation, oversee the ongoing operations, and support the enhancement of existing tools as needed

Requirements:

Minimum 5 years of Data Engineering, Software Development or similar experience required
Proficiency in fundamentals of data pipelining, ELT/ETL processes, data architecture, and the overall data lifecycle, complemented by hands-on experience in designing and implementing efficient data workflows and processes
Demonstrating ability to contribute across multiple phases of development for complex projects, including technical design, implementation, and end-to-end testing, ensuring the successful delivery of data engineering solutions
Passionate about delivering data engineering projects collaboratively, with a strong commitment to achieving team goals
Effective troubleshooting skills, capable of identifying and resolving sophisticated issues with sound judgment and initiative
Comfortable working in an agile environment, with prior experience in Agile and Scrum methodologies, ensuring adaptability to evolving project requirements and effective collaboration with cross-functional teams
Proficiency in SQL, Python, AWS services (e.g. IAM, S3, Kinesis Data Stream, Kinesis Firehose, Lambda, DynamoDB, DMS, EventBridge), with a proactive demeanor towards continuous learning and skill development, and an interest in mastering emerging data engineering tools and methodologies
Experience working across both relational and NoSQL databases (e.g., Snowflake, DynamoDB, Redis, MongoDB), with the ability to optimize data structures and storage choices for performance and scalability
Bachelor's degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience required
Expertise coding and developing solutions that solve complex problems, while keeping in mind code space and time complexity
Proficiency in DevOps practices, e.g.: CI/CD, docker, testing, and automation
Understanding of trade-offs between different data storage systems and architectures (data warehouses, SQL vs NoSQL, partitioning, etc.)
Knowledge of DevOps concepts
Experience with API development (e.g., RESTful services, API design, authentication) for exposing and consuming data products and services resiliently, performantly, and at scale
Experience developing real-time data processing using Apache Flink, or comparable stream-processing frameworks (e.g., Spark, Kafka, or Apache Beam)
Experience with infrastructure-as-code, e.g.: Terraform, CloudFormation, AWS CDK, etc

Sr. Data Engineer

Key skills

About this role

Requirements: