Linguist

Job Description

Job Title: Linguist - AI Training & Evaluation Company: Krutrim (Ola) Location: Bengaluru, Kormangala Employment Type: Full Time Languages: Punjabi, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Tamil, Telugu. About Krutrim: Krutrim (Ola) is a leading innovator in AI and machine learning, specializing in the development of cutting-edge language models and AI-driven solutions. As we continue to grow, we are looking for linguists with expertise in Indian languages, natural language processing (NLP), and large language models (LLMs) to join our dynamic team. Help us shape the future of AI in Indian languages! Role Overview: Krutrim is seeking a Linguist - AI Training & Evaluation for LLM Evaluation to curate high-quality datasets, evaluate language models, and fine-tune performance using Reinforcement Learning from Human Feedback (RLHF). You will work on evaluating model outputs, DPO data set creation, performing comparative analysis, and benchmarking Krutrim’s AI models against industry standards. Key Responsibilities: Data Set Creation: Develop high-quality, multilingual data for training large language models (LLMs), ensuring linguistic diversity and cultural relevance in Indian languages. LLM Evaluation: Conduct rigorous evaluation of NLP models and AI language models. Benchmark model performance against internal and external systems to ensure accuracy, relevance, and coherence. Supervised Fine-Tuning (SFT) & RLHF: Apply supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) techniques to improve model performance and adapt to specific tasks. Model Benchmarking: Perform benchmarking of AI models to ensure that Krutrim’s LLMs perform competitively in real-world applications. Model Response Evaluation: Evaluate the quality of model responses across various tasks and benchmarks. Ensure alignment with desired output accuracy, contextual understanding, and linguistic precision. Collaboration with Cross-Functional Teams: Work closely with data scientists, engineers, and AI researchers to integrate linguistic insights and optimize machine learning models for Indian languages. Desired Skills & Qualifications: Linguistic Expertise: Native-level fluency in one or more Indian languages, with excellent command over English. Proficiency in multilingual linguistics is a plus. AI & NLP Knowledge: Strong understanding of Natural Language Processing (NLP), machine learning models, deep learning frameworks, and large language models (LLMs). Model Evaluation Skills: Proven experience in evaluating AI models, including benchmarking, fine-tuning, and model assessment. Attention to Detail: Strong ability to identify linguistic nuances, model errors, and biases in AI model outputs. Data Management: Skilled in managing and curating large datasets, understanding data preprocessing and data cleaning for AI model training. Analytical Mindset: Ability to perform quantitative and qualitative assessments of AI model performance, including the development of evaluation metrics. Qualifications: Education: Bachelor’s or Master’s degree in Linguistics, Language, or similar experience in the previous jobs. Experience: 2-3 years of experience in linguistics, NLP, or AI model evaluation, with a focus on large-scale data curation.