NFL Analyst Agent
An LLM agent that answers NFL betting questions through intent-routed structured retrieval. Seventeen recipes, no vector search.
This is the companion writeup to Beating the Closing Line, which covers the data pipeline and the XGBoost model that sat behind it. This one covers the agent that turned all of that structured data into natural-language answers: an LLM agent that takes free-text NFL questions, classifies their intent, executes pre-defined analytical recipes against a parquet store, and synthesizes the structured result into a Markdown response. Seventeen recipes, no vector search, no embeddings.
People often call any LLM-over-data system a “RAG bot.” This one isn’t, and the difference matters. The architecture I’ll describe is closer to a function-calling agent with a hand-built dispatcher than to retrieval-augmented generation. Whether that’s the right choice depends on the shape of your data; I’ll get to why I think it was the right choice for this one.
why not pure RAG
The default path for “I want to ask questions of my data” is well-trodden: chunk everything into a vector store, embed the chunks, embed the question, do a similarity search, stuff the top-k chunks into the context window, let the LLM answer. It’s a fine pattern when your data is prose (documentation, support transcripts, internal wikis) and your questions are exploratory.
This domain is the opposite shape:
- The data is highly structured. Parquet files with strongly-typed columns. There’s no prose to chunk in the first place.
- Questions are entity-keyed. “Joe Mixon’s rushing yards line of 65.5” needs to dereference the exact player, the exact stat type, and the exact line value. Similarity search is the wrong primitive: you don’t want “approximately Joe Mixon,” you want Joe Mixon.
- Intents are bounded. Real NFL betting questions fall into a small number of patterns: prop analysis, matchup preview, trend analysis, term definition, a few others. The space isn’t open-ended like “ask me anything about my company’s wiki.”
- Hallucination risk is asymmetric. Wrong stats in a betting context aren’t just embarrassing, they actively damage trust. An answer of the form “Joe Mixon averaged 68 rushing yards per game” had better be exactly right.
For shapes like this, the right architecture is structured retrieval: classify the question’s intent, look up the entities, pull exactly the right columns from the right tables, hand the LLM a JSON of pre-computed facts, and ask it to narrate. Vector search adds latency, cost, and a layer of fuzziness that the problem doesn’t need.
the recipe architecture
Three phases. Two LLM calls. One pandas-heavy middle layer.
user question
│
▼
┌──────────────────────┐
│ LLM deconstructor │ → {recipe_id, entities}
└──────────────────────┘
│
▼
┌──────────────────────┐
│ recipe executor │ pandas lookups → context object
│ ├─ helpers │ (player profile, team context,
│ ├─ over/under calc │ matchup, game log, injuries)
│ └─ injury report │
└──────────────────────┘
│
▼
┌──────────────────────┐
│ LLM synthesizer │ ← base prompt + recipe-specific prompt
└──────────────────────┘
│
▼
Markdown answerThe deconstructor and synthesizer are both gpt-5-mini, picked for cost and speed. The deconstructor runs in JSON mode with low temperature; its only job is to emit {"recipe_id": "...", "entities": {...}}. The synthesizer is freeform text generation that follows a hand-written Markdown template.
Splitting the work this way isolates the two failure modes of an LLM stack. The deconstructor can mis-route (pick the wrong recipe), but it can’t hallucinate stats because it never sees any. The synthesizer can phrase things badly, but it can’t invent stats because the JSON it gets is the only source of truth in the context window and the base prompt explicitly tells it: “Narrate, don’t calculate.” A bad answer is almost always traceable to one of the two layers, which is half of debugging.
There are seventeen recipes in total: player_prop_analysis, anytime_touchdown_scorer, team_matchup_analysis, player_comparison_analysis, spread_analysis, total_analysis, weekly_sharp_report, post_game_recap, qb_deep_dive, historical_head_to_head, player_vs_team_historical, trend_analysis, team_evolution_analysis, term_definition, lookup_player, favorite_player_props, start_sit_analysis. Each has a dedicated execution path in the recipe layer and a dedicated synthesis prompt.
the semantic map
The middle layer is configuration-driven. There’s a directory of JSON files that I called the “semantic map,” assembled at startup from a _manifest.json. It declares two things: which metrics exist (organized by entity into nine player categories and seven team categories), and which recipes need which metrics.
For player_prop_analysis, the config is keyed by stat type. Here’s a trimmed slice for rushing yards:
{
"rush_yards": {
"player_profile_metrics": [
{ "label": "Edge Score",
"metric_bundle": {
"score": "player_grade_season_avg",
"tier": "player_grade_season_category",
"rank": "qualified_player_grade_season_rank" } },
{ "metric_id": "player_rush_yards_season_avg",
"label": "Rushing Yards / Game (Avg)" },
{ "metric_id": "player_yards_per_rush_attempt_season",
"label": "Yards Per Carry (Efficiency)" },
{ "metric_id": "player_rush_yards_season_std",
"label": "Performance Volatility (StDev)" }
],
"team_context_profile_metrics": [ ... ],
"matchup_profile_metrics": [ ... ],
"game_log_metrics": ["rush_yards", "rush_attempts", "rushing_epa"]
}
}The recipe declares what it needs. A separate HelperMethods class (about 700 lines) knows how to fetch each metric_id from the in-memory parquet dataframes. Adding a new stat type is a config edit plus, if the metric is new, a helper method. Adding a new recipe is a new JSON file plus a synthesis prompt plus a dispatch entry. The skeleton of the system, the deconstructor, the executor’s data-fetch routines, the synthesizer’s invocation, never moves.
This is the part of the architecture I’d carry forward into any future agent. The pattern of “recipes declare, helpers retrieve, prompts narrate” lets a single developer maintain a domain agent without it turning into spaghetti. Most of the work is in the JSON, not in the Python.
a recipe end-to-end
Walk through player_prop_analysis with a real example.
The user asks: “What do you think about Joe Mixon’s rushing yards line of 65.5 this week?”
The deconstructor returns:
{
"recipe_id": "player_prop_analysis",
"entities": {
"player_name": "Joe Mixon",
"stat_type": "rush_yards",
"line": 65.5
}
}The executor:
- Looks up Joe Mixon’s team and the current week’s game context, finds the opponent
- Loads the recipe config slice for
rush_yards(the snippet above) - For every metric in
player_profile_metrics, calls a helper to retrieve the value from the in-memoryplayer_season_masterdataframe. Same for team context (player’s own team), matchup profile (opponent’s defense), and the game log (last five games) - Computes how many of the last five games went over the 65.5 line
- Looks up the impactful injuries for both teams in this game
The context object handed to the synthesizer is a single JSON blob with all of the above, plus the narrative_template (“Here is the analytical breakdown for Joe Mixon’s rush yards line of 65.5 for the upcoming game against the Steelers.”).
The synthesizer loads two prompts: the universal base prompt (Markdown formatting rules, “narrate don’t calculate,” injury table format) and the recipe-specific prompt (the player-prop-analysis-specific instructions: how to combine score+tier+rank into a single string, how to interpret game script for rushing props, how to identify “funnel” defenses, how to structure the Final Verdict).
The output looks roughly like:
Here is the analytical breakdown for **Joe Mixon**'s **rush yards** line of
**65.5** for the upcoming game against the **Steelers**.
#### The Player Profile
| Metric | Value |
| ------------------------------- | ----------------- |
| Edge Score | 71.2 (Good, #14) |
| Rushing Yards / Game (Avg) | 68.4 |
| Yards Per Carry (Efficiency) | 4.1 |
| Performance Volatility (StDev) | 22.7 |
| ... | ... |
His season-long Edge Score of 71.2 ranks him #14 out of 38 qualified
running backs, placing him in the 'Good' tier. His rushing volatility
is moderate at a 22.7 yard standard deviation, which means he can post
85+ yard games and 40-yard duds in the same month...
#### The Matchup Analysis
[...table...]
This is a **Tough** matchup, as the Steelers' Run Defense Edge ranks
#7 in the league. Furthermore, this is a classic **pass-funnel**
defense: they boast the #7 ranked Rush D but are vulnerable through the
air at #21, which should encourage Cincinnati to attack via the pass...
#### Final Verdict
The case for the over rests on volume: Mixon has cleared this number in
3 of his last 5, and Cincinnati's implied team total of 22.5 suggests
positive game script. The case for the under is the matchup: a top-10
run defense pairing with a pass-funnel tendency means Cincinnati's
ground game is unlikely to find consistent traction. **Lean: Under**.That entire response is structurally guaranteed by the prompt. No format drift between questions of the same type. No invented numbers. No “I don’t have data on that” because the executor either has all the metrics or returns an error before the synthesizer ever sees the question.
player grades
One source in the pipeline doesn’t come from a public NFL data feed: weekly hand-graded player grades, fetched from a Google Sheet. The pipeline ingests them via a _calculate_player_grades routine and turns them into three derived features:
player_grade— the raw grade as entered in the sheetplayer_grade_weekly_rank— rank within (week, position), where 1 is bestplayer_grade_category— percentile bucket: Poor (0-20), Below Average (20-40), Average (40-60), Good (60-80), Great (80-95), Elite (95-100)
The agent surfaces the season-rollup of these three numbers as the “Edge Score” string format: [Score] ([Tier], #[Rank]). So an answer might include a line like Edge Score: 71.2 (Good, #14) for a running back. The synthesizer’s prompt explicitly enforces this format so the rendering is identical across every recipe that uses it.
Hand-grading every week feels primitive next to a fully automated pipeline, but it served a purpose: a stat-based grade can tell you a player averaged 70 yards on 15 touches, but it can’t tell you their offensive line was hemorrhaging pressure all game and they were running into eight-man fronts. A grader watching the film can. The downside is that hand-grading doesn’t scale: it’s a few hours of someone’s time every week, and it caps the cadence at which the system can update. If I revisited this, the right move is probably to train a small classifier on existing hand-graded data to extend the grader’s eyeball test into more games at less cost.
what worked
The recipe-and-semantic-map pattern is the part I’d build the same way again. It scales to new domains cheaply: the architecture is the work, the recipes are the product. Most additions are JSON, not Python.
The two-pass LLM split (deconstruct, then narrate against pre-computed facts) is the most under-talked-about pattern in LLM application design. The default mode of single-shot RAG bakes in the failure modes (entity confusion, fabricated statistics, format drift) that the two-pass design straightforwardly eliminates. Whenever someone shows me an “LLM over structured data” app that’s hallucinating, I look for whether the LLM is doing both the retrieval and the narration; if it is, that’s the bug.
The semantic map’s separation of “what to fetch” (recipes) from “how to fetch” (helpers) made the codebase navigable a year after the initial build. I could open a recipe JSON, see exactly which metrics it needed, and know exactly which helper method was responsible for each. Nothing dynamic, nothing introspection-based, nothing that required holding the whole system in your head.
what didn’t, and what I’d change
Three honest concessions.
The deconstructor’s prompt grew into a monster. It’s a 300-line system prompt with 22 worked examples spanning all 17 recipes. Every new recipe required adding 1-2 more examples to disambiguate. By the end, the prompt was approaching the size of a small fine-tuning dataset; at that point, fine-tuning a smaller model on the same examples would have been cheaper, faster, and more accurate than running the giant prompt through gpt-5-mini on every request. I never made the switch, but I should have.
Recipes don’t compose. A user who asks “compare Joe Mixon’s rushing yards line to Bijan Robinson’s” gets routed to player_comparison_analysis, which is its own recipe with its own prompt and its own data shape. That recipe re-derives a lot of what player_prop_analysis already knows, and the two prompts have drifted apart over time. A better design would have made the prop analysis a primitive that comparison and matchup recipes could compose, rather than parallel siblings. I knew this when I shipped it; I shipped anyway because it was faster, and predictably the cost showed up later in maintenance.
No conversation state. Every question is independent. “What about against the Steelers?” as a follow-up to a Joe Mixon question would require the user to repeat the player and the line. Adding session state to the agent would have been a few hours of work and would have meaningfully improved the feel of the product. I left it out because the initial release was “answer one question well” and I never came back to it.
why I shut it down
The architecture was the artifact I wanted to build. Once it was working, the question stopped being “can I make this?” and started being “do I want to operate this?” Those are very different questions, and I was much more interested in the first.
Operating an NFL data product is a real weekly time commitment. Sources change format, scrapers break, model retraining needs supervision, hand-graded inputs need someone doing the grading. None of that is the work I find interesting at the cadence required to keep a live product healthy. The pipeline was good enough to run for months at a time without intervention, but “months” isn’t “indefinitely,” and the gap between those two regimes is most of what running a real product is.
The agent itself I’m still proud of. The recipe-and-semantic-map design holds up. The two-pass LLM pattern (deconstruct first, narrate later) is the one I’d reach for again immediately for any structured-data agent I might build next. If you’re considering similar architecture for a different domain, the message worth taking from this writeup is the one in the section above titled “why not pure RAG”: don’t reach for vector search by default. Look at the shape of your data and the shape of your questions first, and pick the retrieval strategy that fits. For a lot of domains, structured retrieval is the right answer, and almost nobody is writing about it.