Sr. Technical Infra Manager

Job Description

1. BACKGROUND Job Title: Senior Technical Infra Manager - Hadoop Systems and Administration IT Domain: Big Data, Data Analytics and Data Management Client Location : BKC, Mumbai, INDIA (Client Facing Role) Functional Domain: Capital Markets, Banking, BFSI Experience: Overall, 14-18 years in Systems Administration with mandatory experience for the last 6 years in Hadoop Cluster Administration in Cloudera or Horton Works Expertise: Infrastructure, Systems Administration, Hadoop Platform Administration Project : Greenfield Hadoop (CDP) based Data Warehouse (DWH) Implementation Company : Smartavya Analytica Private limited is a niche Data and AI company. Based in Pune, we are pioneers in data-driven innovation, transforming enterprise data into strategic insights. Established in 2017, our team has experience in handling large datasets up to 20 PBs in a single implementation, delivering many successful data and AI projects across major industries, including retail, finance, telecom, manufacturing, insurance, and capital markets. We are leaders in Hadoop, CDP, Big Data, Cloud and Analytics projects with super specialization in very large Data Platforms. https://smart-analytica.com Empowering Your Digital Transformation with Data Modernization and AI Job Summary: For its flagship project having more than 100 CDP nodes, we are looking for a senior and accomplished Senior / General Manager level Infrastructure leader in Hadoop, Big Data, Cloud and Data encompassing Infrastructure, CDP Platform, Applications and other Hadoop areas. The person in this role will be responsible for actively own, provide leadership, manage a team of 15 professionals, own the entire Hadoop infrastructure of 150 nodes encompassing 25+ PB’s, lead the team managing the client CDP cluster on 24*7 basis and lead a team 15 Hadoop Admins working round the clock on 365 days *24-hour basis. In this role, you will play a key part in Technical Solutioning, Infrastructure Management, Issue Resolution, Solve Performance bottlenecks, Cluster Monitoring, Cluster Administration, Linux Administration, Security Design and Management and Team Management and mentoring. The ideal candidate possesses a strong analytical mindset, a deep understanding of Hadoop Cluster Management techniques, and a knack for presenting complex analysis and resolution in a clear and concise manner. 2. KEY RESPONSIBILITIES & SKILLS a. Client and Team Leadership and Mentoring • As owner, interact with VP/Director level senior clients and provide end to end leadership • Lead a team of 15 Infra / Hadoop Administrators working on 24*7 shift basis • Design and draw long term roadmap of overall Data Strategy and implementation • Roster Management and Ensure coverage. Ensure all 3 shifts across all 365 days are manned and working efficiently • Train new team members on Hadoop Administration and get them ready on projects b. Hadoop Cluster Management and Administration: • Installation and Configuration: Install and configure Hadoop clusters using Cloudera Data Platform (CDP), ensuring optimal setup tailored to project needs. • Cluster Access Management: Utilize the Cloudera Manager Admin Console to manage cluster access and oversee administrative operations. • Cluster Maintenance and Upgrades: Plan and execute cluster upgrades, patches, and migrations with minimal downtime. • Resource Management: Ensure high availability and scalability by effectively managing cluster resources and configurations. c. Hadoop System Monitoring and Performance Optimization: • Health Monitoring: Proactively Monitor cluster health and performance using Cloudera Manager tools to ensure system reliability and ensure high availability. • Performance Tuning: Analyse system metrics to identify bottlenecks and optimize resource utilization for enhanced performance. • Troubleshooting: Promptly troubleshoot and resolve issues related to cluster operations and data processing. • Perform Linux administration tasks and manage system configurations. • Ensure data integrity and backup procedures including DR Replication. d. Security Management and Compliance • Access Control: Configure and manage user access, roles, and permissions within the Hadoop environment to maintain security protocols. Implement and manage security and data governance. • Data Security: Ensure data security by implementing encryption, authentication mechanisms, and adhering to security policies. • Regulatory Compliance: Maintain compliance with industry regulations pertinent to capital markets, ensuring data handling meets required standards. • Vulnerability Management: Collaborate with security teams to identify and address vulnerabilities, ensuring system integrity. 3. Qualifications : a. Experience: • Overall: 14-18 years in Senior Positions in Infrastructure Management, Data Center Management, Systems Administration, Big Data / Hadoop / CDP Architect and / or Senior Solutions Architect. • Hadoop Administration: Minimum of 6 years managing Hadoop clusters using Cloudera CDP or Hortonworks. b. Technical Expertise : • Proficient in managing Infrastructure, Hadoop platform administration of components and tools, including HDFS, YARN, MapReduce, Hive, Ozone, DR Replication, Kudu Management, Spark Streaming and related areas. • Strong understanding of Cloudera Data Platform (CDP) features and administrative tools. • Experience with Linux/Unix system administration and scripting languages (e.g., Bash, Python). • Knowledge of data warehouse concepts and big data best practices. • Hadoop Technologies: (Preferred) Hands-on experience with HDFS, Hive, and Spark for handling large-scale data environments. c. Domain Knowledge: Familiarity with capital markets and financial services is highly desirable. d. Soft Skills : • Excellent problem-solving and analytical abilities. • Strong communication skills, both written and verbal. • Ability to engage effectively with clients and stakeholders. • Leadership skills with experience guiding technical teams. 4. Educational Background Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. 5. What We Offer • Challenging Projects: Work on a cutting-edge greenfield project in the big data and analytics space. • Professional Growth: Opportunities for learning and career advancement within the organization. • Collaborative Environment: Join a dynamic team focused on innovation and excellence. • Competitive Compensation: Attractive salary package commensurate with experience and expertise.