Turning Moxie’s data into trusted AI answers.
In one month, Orchard engineered the context layer and evaluation system that made Moxie’s internal AI Data Agent a reliable source of answers across the business.
- AI answer accuracy
- 73%to96%
- Monthly AI sessions
- +83%
- Monthly active AI users
- +38%
Challenge
Moxie’s team was already asking its internal AI Data Agent questions about the business. But unreliable answers were eroding confidence, and engagement with Moxie’s internal data tools was suffering.
There was also no consistent way to measure the agent’s performance. Moxie needed to know which questions it could answer reliably, whether updates improved accuracy, and whether that quality held up as its data and business changed.
Orchard’s task was to prepare Moxie’s internal AI Data Agent for a company-wide launch, with reliable answers and a clear way to verify their accuracy.
Solution
Orchard engineered two connected systems: a context layer that taught the AI Data Agent how Moxie’s business works, and an evaluation system that exposed weak spots and guided improvements.
- Context engineering
- Orchard gave the agent the business knowledge it needs to answer correctly.
- Evaluation system
- Orchard used 144 business questions to find failures, guide fixes, and verify better answers.
Business context for more accurate answers
As Moxie’s data partner, Orchard already understood the definitions, calculations, and business rules behind its reporting. Orchard turned that knowledge into a shared context layer the AI Data Agent could use.
Orchard documented the data model, aligned metric definitions, and explained how company data fits together and what its numbers mean. That context guided the agent to interpret business questions, select the right information, and ask for clarification when needed.
Evals that drove the improvements
Orchard built the evaluation suite around real business questions, defined what successful answers should contain, and verified numerical expectations against Moxie’s data.
Those evals showed Orchard exactly where the agent fell short. Orchard used the findings to correct the data model, definitions, and context, then verify that answers improved. Together, the context layer and evals gave Moxie more reliable answers and the evidence to expand access with confidence.
“Orchard was instrumental in helping us optimize our semantic layer for agentic use.
They’re involved in day-to-day operations and deeply familiar with the business context needed to implement a scalable solution.
Today we have clear observability into what questions and data users are asking for, and a robust evaluation process to ensure our semantic layer stays accurate as Moxie's data needs evolve.”
Results
In one month, Orchard raised AI answer accuracy from 73% to 96%.
Orchard made AI answers more consistent across the business, narrowing the accuracy gap between the strongest and weakest areas from 67 percentage points to just 11. The lowest-scoring business area improved from one in three correct answers to nine in ten.
More consistent answers across the business.
AI answer accuracy across 14 business areas
Each line follows one business area.
| Area | Before | After Orchard |
|---|---|---|
| A | 33.3% | 88.9% |
| B | 55.6% | 100% |
| C | 57.1% | 92.9% |
| D | 62.5% | 100% |
| E | 66.7% | 95.8% |
| F | 66.7% | 100% |
| G | 70.6% | 94.4% |
| H | 72.7% | 95.5% |
| I | 76.9% | 100% |
| J | 84.6% | 96.2% |
| K | 93.8% | 93.8% |
| L | 100% | 100% |
| M | 100% | 100% |
| N | 100% | 100% |
Greater confidence. Wider adoption.
That reliability gave Moxie the confidence to bring its AI Data Agent to more people across the company. In the months following Orchard’s work, both monthly sessions and active users grew.
- Monthly AI sessions
- +83%
- Monthly active AI users
- +38%
Orchard’s evaluation system also gave Moxie a way to protect that trust as usage grew: checking new questions and capabilities against a clear quality standard before Moxie’s team relied on them.