Data Engineer

Job Description

Job Title: Data Engineer Experience ~ 3 years Location - Noida (Hybrid) Tech Stack - ADF, Microsoft Fabric, SQL, Pyspark, Python, Data Modelling Summary: you will play a critical role integrating source systems to cloud data lakes, Create scalable ETL pipelines, Maintain data architecture standards and quality, transform and store data using SQL/ Python in datalakes and data warehouses. Key Responsibilities: Design, build, and maintain robust, scalable data pipelines for ETL processes. Develop and maintain data architectures (data lakes, data warehouses, etc.) to ensure data is easily accessible and usable. Collaborate with Data Scientists, Analysts, and other stakeholders to understand data requirements and provide timely data Optimize and manage database performance for large-scale data systems. Implement data quality and validation checks to ensure high-quality data. Integrate data from multiple sources into unified data storage systems. Write Python, Pyspark & SQL queries for data transformation. Create and manage automated data workflows Perform data cleansing and transformation to ensure the readiness of data for analysis and reporting. Work with cloud data platforms (Microsoft Fabric) and manage infrastructure resources for optimal performance. Document data models, processes, and workflows to ensure clarity and consistency across the team. Continuously improve data pipeline efficiency, scalability, and performance. Strong understanding of ETL processes, data integration, and data warehousing. Knowledge of data modeling, data pipelines, and data governance best practices. Strong problem-solving skills and the ability to troubleshoot data-related issues. Excellent communication skills and ability to collaborate with cross-functional teams.