Sr./Lead Data Engineer- Python & SQL- A Software Start Up

Job Description

About the Role: We are seeking a talented Lead Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes. Responsibilities: Data Pipeline Development: Build and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes. Data Infrastructure: Design, implement, and manage data infrastructure components, including data warehouses, data lakes, and data marts. Data Quality: Ensure data quality by implementing data validation, cleansing, and standardization processes. Team Management: Able to handle team. Performance Optimization: Optimize data pipelines and infrastructure for performance and efficiency. Collaboration: Collaborate with data analysts, scientists, and business stakeholders to understand their data needs and translate them into technical requirements. Tool and Technology Selection: Evaluate and select appropriate data engineering tools and technologies (e.g., SQL, Python, Spark, Hadoop, cloud platforms). Documentation: Create and maintain clear and comprehensive documentation for data pipelines, infrastructure, and processes. Skills: Strong proficiency in SQL and at least one programming language (e.g., Python, Java). Experience with data warehousing and data lake technologies (e.g., Snowflake, AWS Redshift, Databricks). Knowledge of cloud platforms (e.g., AWS, GCP, Azure) and cloud-based data services. Understanding of data modeling and data architecture concepts. Experience with ETL/ELT tools and frameworks. Excellent problem-solving and analytical skills. Ability to work independently and as part of a team. Preferred Qualifications: Experience with real-time data processing and streaming technologies (e.g., Kafka, Flink). Knowledge of machine learning and artificial intelligence concepts. Experience with data visualization tools (e.g., Tableau, Power BI). Certification in cloud platforms or data engineering. Job Types: Full-time, Permanent Pay: ₹3,000,000.00 - ₹4,000,000.00 per year Ability to commute/relocate: Noida Sector 62, Noida, Uttar Pradesh: Reliably commute or planning to relocate before starting work (Required) Application Question(s): Willing to join an IT Start Up? Experience: Python: 5 years (Required) Data Engineer: 5 years (Required)