Job Description

Key Responsibilities: Data Pipeline Development: Design and implement efficient data pipelines using Azure Databricks with PySpark to transform and process large datasets. Optimize data workflows for scalability, reliability, and performance. Application Integration: Collaborate with cross-functional teams to develop APIs using the .NET Framework for Azure Web Application integration. Ensure smooth data exchange between applications and downstream systems. Data Warehousing and Analytics: Build and manage data warehousing solutions using Synapse Analytics and Azure Data Factory (ADF). Develop and maintain reusable and scalable data models to support business intelligence needs. Automation and Orchestration: Utilize Azure Logic Apps, Function Apps, and Azure DevOps to automate workflows and streamline deployments. Implement CI/CD pipelines for efficient code deployment and testing. Infrastructure Management: Oversee Azure infrastructure management and maintenance, ensuring a secure and optimized environment. Provide support for performance tuning and capacity planning. Business Alignment: Gain a deep understanding of AMO data sources and their business implications. Work closely with stakeholders to provide customized solutions aligning with business needs. BAU Support: Monitor and support data engineering workflows and application functionality in BAU mode. Troubleshoot and resolve production issues promptly to ensure business continuity. Technical Expertise: Proficiency in Microsoft SQL for complex data queries and database management. Advanced knowledge of Azure Databricks and PySpark for data engineering and ETL processes. Experience with Azure Data Factory (ADF) for orchestrating data workflows. Expertise in Azure Synapse Analytics for data integration and analytics. Proficiency in .NET Framework for API development and integration. Cloud and DevOps Skills: Strong experience in Azure Infrastructure Management and optimization. Hands-on knowledge of Azure Logic Apps, Function Apps, and Azure DevOps for CI/CD automation.