Senior LLMOps / AI Platform Engineer (Onsite)
Not Disclosed•Full-TimeOn-site
location_on1729, Jasmine Circle Northwest, West Highland, Atlanta, Fulton County, Georgia, 30318, United States
This position serves as the operational backbone for "George" and the broader suite of generative AI applications. The Senior LLMOps / AI Platform Engineer is responsible for ensuring the AI platform is reliable, observable, cost-aware, secure, and release-ready across development, QA, and production environments. You will bridge the gap between experimental AI capabilities and stable, production-grade infrastructure, managing everything from Azure OpenAI model deployments to complex RAG (Retrieval-Augmented Generation) pipelines.
As the role evolves, you will expand your scope from supporting a RAG-based chatbot to enabling agentic workflows, ensuring that tool-call observability, prompt versioning, and evaluation gates are seamlessly integrated into the CI/CD lifecycle. This is a hands-on role where you will own the full lifecycle of AI infrastructure, from incident triage and root-cause analysis to the creation of executive-ready status reports and operational runbooks.
Your day involves a dynamic mix of proactive infrastructure management and reactive incident response. You will spend significant time monitoring Azure AI Search indexes and Langfuse traces to ensure retrieval quality and prompt visibility. A typical workflow includes troubleshooting rate-limit issues, managing Kubernetes pod health via Argo CD, and refining Jenkins pipelines to automate release validation.
You will collaborate closely with cross-functional teams to translate technical AI risks into business-readable updates. Whether you are coordinating model upgrades, analyzing token usage trends, or debugging failed trace ingestion, your goal is to maintain high availability and performance while controlling costs. You will also be responsible for documenting these processes, creating runbooks for common failures, and establishing repeatable deployment patterns for models, prompts, and indexes.
Candidates selected for this role will be expected to demonstrate deep expertise in the Azure AI ecosystem and Kubernetes operations. The interview process will focus on practical scenarios involving quota planning, RAG infrastructure troubleshooting, and CI/CD pipeline design.
This is an onsite role based in Atlanta, GA. Candidates must be willing to work on-site. Visa sponsorship is not available; candidates must be US Citizens (USC) or hold a Green Card (GC).
We are committed to building a diverse and inclusive team. We consider qualified applicants regardless of background, race, religion, gender, or other protected characteristics.
Work model: On-site
1729, Jasmine Circle Northwest, West Highland, Atlanta, Fulton County, Georgia, 30318, United States
Atlanta, Georgia
Experience with Langfuse, Azure AI Foundry, Azure AI Search, Dynatrace, App Insights, OpenTelemetry, LangGraph, MCP, agent frameworks, and Infrastructure as Code tools such as Bicep, Terraform, Helm, or Kustomize.
Cleo Consulting, Inc. is an Information Technology & Services firm headquartered in Buffalo, New York, specializing in IT consulting and recruitment. Founded by partners with a combined experience exceeding 40 years in the industry, the company delivers projects across multiple sectors through strategic onshore and offshore delivery models. The organization supports clients by managing complex hiring needs in IT, Finance & Accounting, Engineering, Customer Service, Admin Support, and Sales, allowing businesses to concentrate on core operations while external experts handle talent acquisition.
Unlike typical project-based firms, Cleo Consulting, Inc. maintains a selective client roster to foster strong, long-term partnerships. This approach ensures high levels of responsiveness, minimal administrative barriers, and direct partner-led involvement on all assignments. The firm operates with a focused, agile mindset that prioritizes action and tangible results over marketing claims. By remaining a smaller entity, the team maintains a driven culture dedicated to excellence and personalized service for its limited number of partners.
Browse more roles: All Cleo Consulting, Inc. jobs, it jobs on Recrutus.
Experience
Senior
Job Type
Full-Time
