Data Engineer - Databricks

Job Description

Key Responsibilities: Work with business analysts and data architects to translate requirements into technical implementations. Design, develop, implement, and maintain pyspark code through Databricks UI that enables data and analytics use cases for the client Code, test, and document new or modified data systems to create robust and scalable applications for data analytics. Dive deep into performance, scalability, capacity, and reliability problems to resolve issues. Take on research projects and POC to improve data processing. Work and Technical Experience: Must-Have Skills: 5+ Hands on Experience with Databricks and PySpark Proficiency in SQL and data manipulation skills Good understanding of data warehousing concepts and technologies. Good-to-Have Skills: Understanding of Google Pub sub/Kafka/Mongo DB Familiarity with ETL processes and tools for data extraction, transformation, and loading. Knowledge of cloud platforms like Databricks, Snowflake, Google Cloud Familiarity with data governance and data quality best practices. Qualifications: Bachelors degree in computer science, engineering, or related field Demonstrated continued learning through one or more technical certifications or related methods 5+ years of relevant experience in Data Analytics Relevant experience in Retail domain. Qualities: Self-motivated and focused on delivering outcomes for a fast-growing team and firm Able to communicate persuasively through speaking, writing, and client presentations Able to consult, write, and present persuasively Able to work in a self-organized and cross-functional team Able to iterate based on new information, peer reviews, and feedback Able to work with teams and clients in different time zones Research focused mindset