Sr. Software Engineer, Big Data Platform

  • The Walt Disney Company
  • Bristol, CT, USA
  • May 09, 2019
Full-time AWS Big Data Computer Science Data Analysis Data Mining Hadoop Java Machine Learning Software Engineering Spark

Job Description

We have created a new Big Data Platforms group within Disney Direct-To-Consumer and International (DTCI) technology with the skills, drive, and passion to innovate, create, and succeed in enabling a Direct to Consumer Strategy for ESPN, Disney and ABC products. We are here to disrupt and start a cultural revolution that can lead a revolution in the application of data and analytics across The Walt Disney Company, focused on Content Personalization/Recommendation, Deep User Understanding, and Audience Segmentation for Linear to Digital Ad Sales, and Analytics.

The Sr Software Engineer will be working on Audience Segmentation to enable real time curation of users on TB scale data that would be used by Digital Ad Sales, Analytics, Personalization and Marketing use-cases.

The internet-scale platforms that you design and build will be a core assets in the delivering the highest quality content to over 150MM+ consumers on monthly basis. This is an opportunity to fundamentally evolve how DTCI delivers content and monetizes our audiences.

Job Summary:

  • Build cool things – Build software across our entire cutting-edge data platform, including data processing, storage, and serving large-scale web APIs, with awesome cutting-edge technologies operating in real-time with high-availability.
  • Harness curiosity – Change how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.
  • Innovate and inspire – Think of new ways to help make our data platform more scalable, resilient and reliable and then work across our team to put your ideas into action.
  • Think at scale - Lead the transformation of a peta-byte scale batch based processing platform to a near real-time streaming platform using technologies such as Apache Kafka, Cassandra, Spark and other open source frameworks.
  • Have pride – Ensure performance isn’t our weakness by implementing and refining robust data processing, REST services, RPC (in an out of HTTP), and caching technologies.
  • Grow with us – Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways which make other groups jealous.
  • Lead and coach – Mentor other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.
  • ML First – Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.
  • Build and Support – Embrace the DevOps mentality to build, deploy and support applications in cloud with minimal help from other teams.

Basic Qualifications:

  • Strong programmatic skills in Java or Go.
  • Hands on experience working on Distributed systems that handle TB scale data.
  • Practical experience on at least two projects using Spark or Hadoop M/R.
  • Strong in algorithms, design, multi-threading, caching concepts.
  • Exposure to one or more of Cassandra, Mongo DB or such columnar data stores.
  • Prior experience deploying and managing software on AWS Cloud Services. Exposure to Kubernetes, Docker and Airflow ideal.

Preferred Qualifications:

  • Prior experience building internet scale platforms – handling Peta- byte scale data, operationalizing clusters with hundreds of compute nodes in cloud environment.
  • Experience working on bitmap technologies to represent user data a big plus.
  • Experience with open source such as Spring, Hadoop, Spark, and Pilosa.
  • Proficiency with agile development methodologies shipping features every two weeks. It would be awesome if you have a robust portfolio on Github and / or open source contributions you are proud to share.

Required Education:

  • Bachelor’s degree or better in Computer Science or a related technical field or equivalent job experience.

Preferred Education:

  • Masters in Computer Science or similar is preferred.

Job ID