AI Cost Governance: The Executive's Framework

AI cost governance is the operating system that turns cost visibility into cost discipline. You have a dashboard that shows cost per unit of work. Now what? Governance answers: Who makes decisions about that cost? What decisions require approval? What triggers an investigation? What gets reported to the board? How do you enforce it? Mavvrik positioned "AI bill shock" as the problem. Governance is how you prevent it. Most mid-market companies are still writing their first governance policy. This framework gives you the structure.

Seven components of AI cost governance

1. Allocation rules (who owns which cost?)

Define how costs get allocated to P&L owners. Start simple:

• Direct allocation: "If team X runs model Y, team X owns the cost."
• Shared allocation: "If multiple teams use the fine-tuned model, split cost by usage."
• Overhead allocation: "Vector DB and observability are corporate overhead, allocated by # of agents."

Write down the rules and stick to them. Example: "Support team owns 100% of support-agent cost. Product team owns 100% of lead-scoring-agent cost. Observability cost is allocated 40% to support, 30% to product, 30% to operations based on log volume." Document it in a one-pager. Update it quarterly. Communicate changes to the teams.

2. Cost targets and SLOs (what's the target?)

For each AI agent or capability, set a cost per unit of work target and an acceptable variance range. Format: "Agent X runs at $Y per unit, with an SLO of $Y ± Z%."

Examples:

• Support escalation: $0.40 per ticket, ± 15% ($0.34-$0.46)
• Claims processing: $5.00 per claim, ± 12% ($4.40-$5.60)
• Lead qualification: $12 per lead, ± 20% ($9.60-$14.40)

The variance should be realistic. ±5% is too tight for month one (you'll have false alarms). ±20% is too loose (you'll miss real problems). 10-15% is a reasonable starting point. Tighten it over time as you understand your cost drivers.

3. Anomaly thresholds (what triggers investigation?)

Define three levels of cost deviation:

• Alert (yellow): Cost drifts 15-25% above SLO for one week. Investigate. Usually it's temporary (a retry spike, a new data type, a vendor issue).
• Escalation (orange): Cost drifts 25-50% above SLO for two weeks, or exceeds the SLO for three weeks. Escalate to the team lead and CFO. Root cause required. Corrective action required.
• Pause (red): Cost exceeds SLO by 50%+ or predicted monthly spend exceeds budget by 20%+. Pause the agent immediately. No new experiments until fixed.

Automate this. Your cost aggregation tool should flag violations. You shouldn't have to manually check every agent every day.

4. Approval workflows (who decides?)

Define approval authority by spend magnitude:

• <$10k/month: Auto-approved. Any engineer or product lead can launch. Just document the hypothesis.
• $10k-$50k/month: Requires product lead approval + finance sign-off. 48-hour turnaround.
• $50k+/month: Requires CFO approval. Goes into the quarterly operating plan.

New prompt changes: Auto-approved if the agent is already running. Don't create friction for optimization.

New agents or major model changes: Follow the tiered approval.

Exception requests (e.g., "we want to run this expensive experiment that will exceed our SLO"): CFO decision. Document the request, the business case, and the decision.

5. Cost attribution logic (how do you charge back?)

This is the hardest piece. Write down your attribution logic for every shared cost:

Shared model: "If 3 agents use the same fine-tuned GPT-4, how do you split the cost?" You could split by usage (based on number of API calls), by success rate (if one agent succeeds 90% of the time and another 70%, allocate proportionally), or equally. Pick one and document it.

Observability infrastructure: "Your Datadog bill is $8,000/month. 60% of logs are from support agents, 25% from product agents, 15% from back-office. Allocate the cost accordingly."

Vector database storage: "Pinecone stores 1M vectors. 600k are from claims embedding, 400k from product embeddings. Allocate 60% of cost to claims, 40% to product."

You don't need to be perfect. You need to be consistent and defensible.

6. Exception handling (what if something breaks?)

Define the process for exceptions:

• Temporary exception: "Our support agent hit $0.65 per ticket (SLO is $0.40 ± 15%) because we had a retry storm on Tuesday. This is temporary. Approved for two weeks while we investigate."
• SLO revision: "Our original $5.00/claim target was too tight. Claims are naturally more complex this quarter (higher document count, more edge cases). New target: $6.00 ± 15%. Finance and ops sign off."
• Pause and redesign: "The lead-qualification agent is costing $18 per lead (SLO is $12 ± 20%). We can't hit the target with this approach. Pause it for 2 weeks. Engineering will redesign the prompt and model selection."

The key: exceptions are documented and time-bound. You don't have "indefinite exceeds SLO" agents running.

7. Reporting cadence and content (what does the board see?)

Monthly report to the board:

• Total AI spend (vs. budget, vs. prior month, vs. plan)
• Cost per unit for each agent (vs. SLO, vs. target)
• Net margin benefit (AI cost - salary savings)
• Any cost anomalies (agents breaching SLO, new vendors, large experiments)
• 1-2 paragraph narrative (what's working, what's not, what's the plan next month?)

This should be a one-pager. If it's longer, it's too detailed for the board. Keep it high-level and business-focused.

Three levels of governance maturity

Level 1 (Basic): You have cost visibility and allocation rules. You report monthly spend by team. You have basic approval workflows (requests go to CFO). You don't have SLOs or anomaly detection yet. You're at stage 3 of the Maturity Curve.

Level 2 (Intermediate): You have allocation rules, cost per unit targets for top agents, SLO monitoring, and anomaly detection. You report monthly to the board. You have tiered approval workflows (Tier 1 auto-approved, Tier 2 requires approval, Tier 3 CFO-only). You're at stage 4.

Level 3 (Advanced): You have all of level 2 plus: cost attribution logic for shared infrastructure, exception handling procedures, quarterly budget reforecasting, board-grade dashboard, automated alerts, and hardened access controls (who can approve new agents, who can change SLOs). You're at stage 5. Only 10-15% of mid-market companies are here yet.

Red flags: Governance breaks down when...

• You don't measure cost per unit. You measure total spend or spend by team, but not by outcome. You can't tell if an agent is expensive or cheap relative to the work it's doing.
• You have no SLOs. An agent is running at $0.60 per ticket and nobody flagged it because there was no SLO. SLOs create accountability.
• Approvals are slow or political. A $25k/month experiment waits 3 weeks for CFO approval. Teams bypass the process and launch experiments off-book.
• You have no anomaly detection. An agent breaks on Tuesday and costs 5x normal for a week. You discover it when the bill arrives on Friday. Automated alerts would have caught it Wednesday.
• Cost governance is disconnected from product decisions. Engineering launches a cool feature that costs $100k/month. Finance says "we have no budget for that." This could have been prevented with approval workflows.
• You don't track margin benefit. You see AI cost but don't track the headcount savings or the revenue upside. Without margin context, spend looks one-dimensional.

How to start: First 90 days

Week 1-2: Document your allocation rules. Who owns what cost? Write it down.

Week 3-4: Set SLOs for your top 5 agents. Pick conservative variance bands (±15-20%). You can tighten later.

Week 5-6: Implement automated anomaly detection. Connect your cost platform to Slack. Alerts show up when agents breach SLOs.

Week 7-8: Draft board report template. What 5-7 metrics does the board care about? Build the first version.

Week 9-10: Define approval workflows. Tier 1 / Tier 2 / Tier 3. Document it. Communicate it to the team.

Week 11-12: Go live. Run the first month with the new governance policy. Measure adherence. Adjust as needed.

By end of 90 days, you'll have moved from stage 2-3 (Tracked / Allocated) to stage 4 (Optimized). You'll have cost discipline without killing innovation. Ready to see how Runrate automates this governance layer? Book a demo.