Job Description :

Job Description:

  • The Senior Data Engineer will be responsible for the build of Enterprise Data platform.
  • Setting up the data pipelines that are scalable, robust and resilient and build pipelines to validate, ingest, normalize/enrich and business-specific processing of healthcare data.
  • Build Azure Data Lake leveraging Databricks technology to consolidate all data across the company and serve the data to various products and services.
  • The scope of this role will include working with engineering, product management, program management and operations teams in delivering pipeline platform, foundational and application-specific pipelines and building the Data Lake in collaboration with business and other teams.

Responsibilities

  • Design, Develop, Operate and drive scalable and resilient data platform to address the business requirements
  • Drive technology and business transformation through the creation of the Azure Data Lake
  • Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy
  • Partner with Product Management and Business leaders to drive Agile delivery of both existing and new offerings; assist with the Leadership and collaboration with engineering organizations within Change to manage and optimize the portfolio

Required Skills

  • 10+ years working experience in Data processing
  • 4+ years working experience in Databricks (essential) and Python
  • Experience with Cloud / Azure architectural components
  • Experience building data pipelines and infrastructure
  • Deep understanding of Data warehousing concepts, reporting and Analytical concepts
  • Experience with Big Data tech stack, including Spark, Scala, and SQL
  • Bachelor's degree in Mathematics, Physical Sciences, Engineering or Computer Science
             

Similar Jobs you may be interested in ..