Sr Data Engineer

  • icrunchdata Network
  • Morrisville, NC, USA
  • May 23, 2020
Information Technology (IT)

Job Description

Recognition Inc. 500 5000 Honoree Company for 2012 , 2013, 2014 ,2015 and 2016 Atyeti Ranks No. 270 on the 2012 Inc. 500 List 2012 2016 NJ 50 Fastest Growing Companies Leading Investment Bank in Morrisville, NC area is looking for a Sr Data Engineer. (Long Term Contract) Please email me your updated resumes with rate expectations to sangitha.ganesh(at) Call - 919 583 2695 Yoursquoll work on a high impact and high visibility project that enables data availability, encompasses data analytics, machine learning, and petabyte scale datasets, and provides reliable and timely access to thousands of data sources. bull Design, architect and support new and existing data and ETL pipelines and recommend improvements and modifications. bull Create optimal data pipeline architecture and systems using Apache Airflow bull Assemble large, complex data sets that meet functional and non-functional business requirements. bull Be responsible for ingesting data into our data lake and providing frameworks and services for operating on that data including the use of Spark Databricks. bull Analyze, debug and correct issues with data pipelines bull Operate on or build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Spark and Azure technologies. You Offer bull You have 8 years of experience building high performance scalable enterprise analytics or data centric solutions bull You have at least 5 years of experience implementing complex ETL pipelines preferably in connection with Hadoop or Spark bull Experience working with Spark data pipeline and or streaming experience bull You have exceptional coding and design skills in JavaScala or C. bull You are an expert in Python, or can demonstrate your ability to readily learn new languages and your affinity for them. bull You understand the nature of distributed systems design. bull You have hands-on experience with Azure. bull You have a BS in Computer Science Software Engineering or equivalent experience. bull You have a strong understanding usage of algorithms and data structures. bull You work well in a team environment, and are a self-starter. bull You have the ability to lead, or have led teams before. bull Expertise building and managing python libraries Nice To Have Azure Data Lake experience bull Machine Learning expertise bull Experience with Apache Airflow bull Experience with C .NET Core bull Experience with traditional data martswarehouses

Job ID