Lead Data Scientist (Gen AI)

Job Description

Blend is hiring a Lead Data Scientist (Generative AI) to spearhead the development of advanced AI-powered classification and matching systems on Databricks. You will contribute to flagship programs like the Diageo AI POC by building RAG pipelines, deploying agentic AI workflows, and scaling LLM-based solutions for high-precision entity matching and MDM modernization. Key Responsibilitie s Design and implement end-to-end AI pipelines for product classification, fuzzy matching, and deduplication using LLMs, RAG, and Databricks-native workflow s.Develop scalable, reproducible AI solutions within Databricks notebooks and job clusters, leveraging Delta Lake, MLflow, and Unity Catalo g.Engineer Retrieval-Augmented Generation (RAG) workflows using vector search and integrate with Python-based matching logi c.Build agent-based automation pipelines (rule-driven + GenAI agents) for anomaly detection, compliance validation, and harmonization logi c.Implement explainability, audit trails, and governance-first AI workflows aligned with enterprise-grade MDM need s.Collaborate with data engineers, BI teams, and product owners to integrate GenAI outputs into downstream system s.Contribute to modular system design and documentation for long-term scalability and maintainabilit y. Qualificati ons Bachelor’s/Master’s in Computer Science, Artificial Intelligence, or related fi eld.7+ years of overall Data Science experience with 2+ years in Generative AI / LLM-based applicati ons.Deep experience with Databricks ecosystem: Delta Lake, MLflow, DBFS, Databricks Jobs & Workfl ows.Strong Python and PySpark skills with ability to build scalable data pipelines and AI workflows in Databri cks.Experience with LLMs (e.g., OpenAI, LLaMA, Mistral) and frameworks like LangChain or LlamaIn dex.Working knowledge of vector databases (e.g., FAISS, Chroma) and prompt engineering for classification/retrie val.Exposure to MDM platforms (e.g., Stibo STEP) and familiarity with data harmonization challen ges.Experience with explainability frameworks (e.g., SHAP, LIME) and AI audit tool ing. Preferred S kills Knowledge of agentic AI architectures and multi-agent orchestr ation.Familiarity with Azure Data Hub and enterprise data ingestion frame works.Understanding of data governance, lineage, and regulatory compliance in AI sy stems. Thrive & Grow w ith Us: Competitiv e Salary: Your skills and contributions are highly valued here, and we make sure your salary reflects that, rewarding you fairly for the knowledge and experience you bring to th e table.Dynamic Caree r Growth: Our vibrant environment offers you the opportunity to grow rapidly, providing the right tools, mentorship, and experiences to fast-track your career.Id ea Tanks: Innovation lives here. Our "Idea Tanks" are your playground to pitch, experiment, and collaborate on ideas that can shape the future.Growt h Chats: Dive into our casual "Growth Chats" where you can learn from the best—whether it's over lunch or during a laid-back session with peers, it's the perfect space to grow your skills.Sn ack Zone: Stay fuelled and inspired! In our Snack Zone, you'll find a variety of snacks to keep your energy high and ideas flowing.Recognition & Rewards: We believe great work deserves to be recognized. Expect regular Hive-Fives, shoutouts and the chance to see your ideas come to life as part of our reward program.Fuel Your Growth Journey with Certif ications: We’re all about your growth groove! Level up your skills with our support as we cover the cost of your certifi cations. Show more Show less