​Senior AI Engineer (Evals/Observability Concentration)

Risepoint is an education technology company that provides world-class support and trusted expertise to more than 100 universities and colleges.

We primarily work with regional universities, helping them develop and grow their high-ROI, workforce-focused online degree programs in critical areas such as nursing, teaching, business, and public service.

Risepoint is dedicated to increasing access to affordable education so that more students, especially working adults, can improve their careers and meet employer and community needs.

The Impact You Will Make Risepoint is developing an AI-powered Student Journey Platform and is seeking a Senior AI Engineer with deep expertise in Retrieval-Augmented Generation (RAG), multi-agent architectures, and LLM evaluation frameworks.

This role focuses on designing, implementing, and operationalizing AI systems with a strong emphasis on structured evaluation (including LLM-as-Judge), measurable quality, and production-grade reliability.

The ideal candidate has experience integrating LLMs with enterprise data sources, building testable and observable AI workflows, and improving system performance through rigorous evaluation and iteration.

This role contributes directly to a platform that is central to the organization's long-term strategy.

How You Will Bring Our Mission to Life What You Will Do • Build and maintain evaluation frameworks (LLM-as-Judge, rubric-based scoring, regression test suites) to measure output quality, reliability, and drift with the responsibility of debugging production level issues as detected.

What Success Looks Like • RAG pipelines return grounded, source-attributed responses with minimal hallucination.

How Impact Will be Measured • AI systems demonstrate measurable improvements in quality using defined evaluation benchmarks.

What You'll Bring to the Team Experience That Matters Most • 3-5 years of full stack engineering experience with strong fundamentals in object-oriented programming, applicable design patterns, and AI-focused system design.

Langfuse, LangSmith, OpenTelemetry-based tracing, custom evaluation harnesses). • Experience implementing guardrails, policy enforcement, and safety layers in AI driven systems while leveraging LLM-as-Judge for validation and continuous improvement.

Experience That's Great to Have • Familiarity with performance optimization techniques for LLM-based systems (latency, caching, routing, batching).

Experience in Databricks (model serving endpoints, ML Flow) Risepoint is an equal-opportunity employer and supports a diverse and inclusive workforce.