Lead consultant - GenAI LLM Ops Engineer

Genpact is seeking a Lead consultant - GenAI LLM Ops Engineer to build, operate, and scale Large Language Model platforms and applications. Responsibilities include architecting secure infrastructure, designing deployment and maintenance of LLM serving infrastructure, and implementing model orchestration, CI/CD pipelines, and telemetry.

Requirements

• Architect secure, reusable, and modular infrastructure-as-code (IaC) frameworks for GenAI and LLM operations
• Design, deploy, and maintain LLM serving infrastructure (e.g., Azure OpenAI, self-hosted OSS models, vector databases).
• Implement model orchestration (routing, ensemble strategies, fallbacks, retries, cache layers).
• Build CI/CD pipelines for prompt catalogs, model configurations, guardrails, and evaluation suites.
• Define and track LLM-specific SLOs (latency, response quality, safety violations, hallucination rate).
• Implement telemetry (traces, logs, metrics, prompt/response analytics) and A/B experiments.
• Establish alerting & incident response playbooks.
• Lead the development and standardization of CI/CD pipelines for AI/ML model deployment
• Ensure security, privacy, and regulatory compliance (data residency, consent, auditability).
• Manage prompt governance (versioning, approval of workflow, change logs, rollback).
• Define and enforce best practices for model versioning, governance and lifecycle management
• Troubleshoot and resolve issues related to LLM deployment, scaling, and performance
• Stay updated with advancements in MLOps, LLMs, and GenAI technologies