The Senior Data Engineer works in a small team of multi-disciplined engineers on creating the next generation video consumption platforms. We expect you to be up to date on the happenings in the data community, passionate about what you do and connected to the open source community. You will participate in overall system design, come up with web­tailored solutions that emphasize re­use and good design patterns. Responsibilities Build and optimize performance of Hadoop/Spark batch jobs. Build and optimize performance of Spark, Kafka, Cassandra, ELK, and whatever else makes sense for realtime pipelines. Design and architect high quality data-lake, data-warehouse, and data-marts data models. Enable and implement Data Science workflows and advanced machine learning algorithms. Build and optimize performance of ElasticSearch cluster and relevance Build data pipelines orchestration. Become and stay an expert in current and emerging technologies and tools Contribute to Open Source solutions and communities wherever you can Collaborate with other software engineers and cross­functional teams Evangelize technologies, solutions, and best practices Contribute new ideas to a larger community of high­-caliber professionals Balance resources, requirements, and complexity Basic Qualifications Very passionate about coding (If you have a Github profile, that’s awesome! Would love to check it out! Otherwise we will ask you to code before or during the interview). Strong understanding of distributed systems and distributed computation. Strong working knowledge in at least 2 of: Scala, Java, Python, or Go-Lang Demonstrated working knowledge with data Apache Hadoop / Spark ecosystem like Spark, Hive, Presto, Oozie, Pig, Hue, Zeppelin Demonstrated working knowledge of data modeling Unit, Integration, and Load testing Developing REST APIs. Git Ant, Maven, SBT, and/or Gradle Unix/Linux Docker containers building and deployment. Excellent communication and collaboration skills Preferred Qualifications GraphQL knowledge Kubernetes knowledge Apache Spark MLlib Apache Spark GraphX R Amazon AWS or other cloud Services Jenkins Required Education BS in Computer Science or related field with 5+ years of experience About The Walt Disney Studios For over 90 years, The Walt Disney Studios has been the foundation on which The Walt Disney Company was built. Today the Studio brings quality movies, music and stage plays to consumers throughout the world. Feature films are released under the following banners: Disney, including Walt Disney Animation Studios and Pixar Animation Studios, Disneynature, Marvel Studios and Lucasfilm. The Disney Music Group encompasses the Walt Disney Records and Hollywood Records labels, as well as Disney Music Publishing. The Disney Theatrical Group produces and licenses live events, including Disney on Broadway, Disney On Ice and Disney Live!. Ours is a culture of innovation, inspiration and collaboration that brings together visionary artists, gifted technologists and savvy business minds to conjure up magical entertainment experiences for a global audience. About The Walt Disney Company The Walt Disney Company, together with its subsidiaries and affiliates, is a leading diversified international family entertainment and media enterprise with the following business segments: media networks, parks and resorts, studio entertainment, consumer products and interactive media. From humble beginnings as a cartoon studio in the 1920s to its preeminent name in the entertainment industry today, Disney proudly continues its legacy of creating world-class stories and experiences for every member of the family. Disney’s stories, characters and experiences reach consumers and guests from every corner of the globe. With operations in more than 40 countries, our employees and cast members work together to create entertainment experiences that are both universally and locally cherished. This position is with Movies Anywhere, which is part of a business segment we call The Walt Disney Studios. Movies Anywhere is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Disney fosters a business culture where ideas and decisions from all people help us grow, innovate, create the best stories and be relevant in a rapidly changing world.