Job Role : BigData Engineer
Location : Dallas,TX (Hybrid)
Duration : Long Term
Responsibilities:
· Design, develop, and deploy cloud-based solutions for data processing, analytics, and integration using cloud services and big data technologies.
· Collaborate with architects, data engineers, and business stakeholders to understand requirements and translate them into technical solutions.
· Implement data ingestion, transformation, and storage processes using cloud services like AWS's S3, Glue, Athena, Redshift, and EMR.
· Develop and optimize data pipelines using Snowpark, SnowSQL, Hadoop and PySpark to extract, transform, and load data efficiently.
· Implement security, data governance, and compliance measures to ensure data integrity and protection in AWS-based solutions.
· Conduct performance tuning and optimization of data processing and analytics workflows to maximize efficiency and scalability.
· Work with cross-functional teams to troubleshoot and resolve issues related to data processing, data integration, and analytics solutions.
· Collaborate with DevOps teams to implement CI/CD pipelines, automated deployments, and infrastructure as code (IaC) practices for AWS-based solutions.