Authorship note: this rewrite was authored with Grok 4.5 in the loop and final edited/deployed by Milo H.
model.default: grok-4.5 on xai-oauth).
I like what xAI is doing. I like the willingness to ship, the weird company energy, the ambition, the willingness to compete directly with the expensive default models. But admiration is not a routing policy. A model gets to touch my agent stack only after it survives real work: tools, files, stale notes, hidden tests, retries, and the little operational traps that vendor launch tables never show.
This post is the full story of how Grok 4.5 moved from “interesting launch” to “route I will actually use.” The short version is above. The details below are the receipts.
xAI launched Grok 4.5 with exactly the claim that gets my attention: frontier agentic capability at a price that might be cheap enough to use heavily. The public materials put it near GPT-5.5/Fable territory on Terminal-Bench style work. The model page lists a 500k context window, text+image input, reasoning, function calling, structured outputs, and pricing at $2/M input and $6/M output, with cached input at $0.50/M.
That is the kind of model that can change a Hermes deployment. Not because it wins a logo fight. Because a cheap, good-enough frontier route can become a background worker, verifier, batch summarizer, council member, or coding canary without making every long agent run feel financially stupid.
The catch: agent systems do not care about benchmark vibes. They care about boring survival. Does the model follow tool instructions? Does it recover after interruption? Does it avoid stale notes? Does it pass hidden tests after the public tests are green? Does the OAuth/subscription route work, or are we accidentally paying API-key rates? That is what I tested.
The integration path is now real:
xAI OAuth provider: xai-oauth
Model slug: grok-4.5
Hermes switch command: /model grok-4.5 --provider xai-oauth
OpenClaw alias: grok45 -> xai/grok-4.5
Default Milo route during testing: GPT-5.5I did not make it the default immediately. That would have been the wrong move. A main agent route is the nervous system, not a trophy shelf. The default has to be predictable when the work is messy, stateful, and possibly destructive. Grok 4.5 first had to earn a job.
The first harness was intentionally small: ten direct prompts, no tools, same inputs. It checked reasoning, code tracing, SQL, strict JSON, ops triage, and constrained writing.
| Model | Score | Median latency | Average latency | API errors | Estimated cost |
|---|---|---|---|---|---|
| GPT-5.5 | 10 / 10 | 2.70s | 2.73s | 0 | $0.0735 |
| Grok 4.5 | 10 / 10 | 4.13s | 4.71s | 0 | $0.0471 |
Useful, not decisive. Grok 4.5 did not faceplant. It was cheaper in this tiny run and slower than GPT-5.5. That was enough to continue, not enough to promote.
The hard-reasoning probe used 18 GPQA Diamond questions, balanced across biology, chemistry, and physics, sampled with seed 20260708. No tools, no web, strict final-answer parsing, sanitized artifact.
| Model | Score | Format failures | API errors | Median latency | Reasoning tokens | Cost |
|---|---|---|---|---|---|---|
| GPT-5.5 | 15 / 18 (83.3%) | 3 | 0 | 16.91s | 22,128 | $0.7684 |
| Grok 4.5 | 17 / 18 (94.4%) | 1 | 1 | 31.91s | 49,111 | $0.3373 |
| Grok 4.3 | 17 / 18 (94.4%) | 0 | 0 | 16.27s | 20,921 | $0.0711 |
This made Grok impossible to dismiss. It also made the routing question more interesting: Grok 4.3 tied Grok 4.5 on this mini cut while being cheaper and faster. So the answer was not “always use the newest Grok.” It was “test the route by workload.”
The Terminal-Bench subset was closer to agent reality: shell, files, code, tests, and recovery. I used Harbor with terminal-bench/terminal-bench-2, agent terminus-2, one attempt per task, and three selected hard tasks. This is not comparable to xAI's public aggregate. It is an operational smell test.
| Task | GPT-5.5 | Grok 4.5 | Grok 4.3 |
|---|---|---|---|
cancel-async-tasks | pass · 75.20s | fail · 122.0s | pass · 64.60s |
write-compressor | pass · 127.4s | pass · 436.8s | AgentTimeoutError · 939.4s |
feal-differential-cryptanalysis | pass · 206.4s | APIError · 117.8s | interrupted after 30m53s |
This was the caution sign. GPT-5.5 was cleanest at 3/3. Grok 4.5 showed real ability, but also tool-loop and API-path friction. If I had stopped here, the answer would have been: wire Grok, but keep it far from default.
The next question was practical: can we compare these models through subscription/OAuth routes instead of direct API-key billing? Yes. GPT-5.5 ran through openai-codex. Grok 4.3 and Grok 4.5 ran through xai-oauth. Direct API-key environment variables were stripped from the child processes.
| Regime | GPT-5.5 | Grok 4.3 | Grok 4.5 | Read |
|---|---|---|---|---|
| Subscription gauntlet | 14/15 · 181.64s | 14/15 · 193.16s | 14/15 · 163.13s | All tied on score; Grok 4.5 fastest. |
| Hermes Agentic Gauntlet v2 | 12/12, pass² 6/6 | 11/12, pass² 5/6 | 12/12, pass² 6/6 | Grok 4.5 matched GPT-5.5 and beat Grok 4.3. |
| Hermes Bench v3 seeded evolutionary pass³ | 90/90, family pass³ 10/10 | 81/90, family pass³ 7/10 | 90/90, family pass³ 10/10 | Grok 4.5 tied GPT-5.5 and was faster. |
| Hermes Bench v4 hidden-test repo repair | 12/12, pass² 6/6 | 11/12, pass² 5/6 | 12/12, pass² 6/6 | Grok 4.5 stayed perfect and was much faster. |
V2, V3, and V4 changed my posture. They showed that Grok 4.5 was not just good at clean prompt probes. It could run as a Hermes child agent through the subscription route, touch files, patch repos, survive hidden tests, and match GPT-5.5 on deterministic scoring. Grok 4.3 remained useful but cracked on edge cases.
Still, V4 left an annoying question unanswered. GPT-5.5 and Grok 4.5 were tied on correctness. Grok was faster, but speed alone is not enough to swap trust. So I made the test uglier.
V5 and V6 were designed to feel more like real agent work. Each task had a diagnosis phase, an intentional interruption, stale or misleading notes injected by the harness, then a fresh resume phase. The model had to preserve the right work, ignore the stale trap, pass public tests, and pass hidden tests. No LLM judge. Just artifacts.
| Model | All-pass | Phase 1 | Public | Hidden | Stale resisted | Total wall | Median pass |
|---|---|---|---|---|---|---|---|
| GPT-5.5 | 1/3 | 3/3 | 3/3 | 1/3 | 3/3 | 581.91s | 229.32s |
| Grok 4.5 | 1/3 | 3/3 | 3/3 | 2/3 | 2/3 | 284.02s | 73.56s |
V5 was the first real shape split. Grok 4.5 passed more hidden tests and finished faster. GPT-5.5 was more conservative: it resisted every stale-note trap. Neither model dominated. That was useful, but it was only one pass per task.
V6 repeated the same stateful resume tasks three times per model. A task only counted as pass³ if all three independent runs cleared phase 1, public tests, hidden tests, and stale-note resistance.
| Model | Attempts | Phase 1 | Public | Hidden | Stale resisted | Task pass³ | Timeouts | Total wall | Median pass |
|---|---|---|---|---|---|---|---|---|---|
| GPT-5.5 | 6/9 | 9/9 | 9/9 | 6/9 | 9/9 | 1/3 | 0 | 1698.41s | 197.84s |
| Grok 4.5 | 7/9 | 9/9 | 9/9 | 9/9 | 7/9 | 2/3 | 0 | 786.94s | 82.71s |
| Task | GPT-5.5 pass³ | Grok 4.5 pass³ | What separated |
|---|---|---|---|
| Router/auth/fallback | 1/3; hidden failed twice | 1/3; hidden passed 3/3 but stale retry pattern failed twice | GPT missed hidden semantics; Grok solved behavior but sometimes copied the stale retry smell. |
| Dirty scheduler locks | 2/3; hidden failed once | 3/3 | Grok was more reliable on dirty-state/idempotency repair. |
| Migration planner | 3/3 | 3/3 | Both cleared pass³; GPT stayed much slower. |
| Hermes/OpenClaw job | Route | Why |
|---|---|---|
| V6-era table (historical). Current applied call: Grok 4.5 default · Grok 4.3 fast · GPT-5.5 fallback · GPT-5.6 Sol not promoted. | ||
| Default interactive Milo session (V6) | GPT-5.5 for now | V6 read: safer stale-context/destructive-change lane. Superseded: default is now Grok 4.5 after v7. |
| Background coding canary (V6) | Grok 4.5 | V6 showed stronger hidden-test reliability and much better wall-clock on stateful repo repair. |
| Batch summarization/classification | Grok 4.5 first; test 4.3 for cheap/simple buckets | Grok 4.5 survived the Hermes gauntlets; Grok 4.3 is cheaper/faster in v7 and is the fast background lane. |
| Hard reasoning second opinion | Grok 4.5, with 4.3 as a cheap comparison route | Both Grok routes were strong on the GPQA mini cut. |
| Destructive migrations, credentials, deploys | GPT-5.5 or higher-stakes verifier | Still true: GPT-5.5 remains the conservative fallback/verifier in live Hermes config. |
| MoA/council diversity | Yes, include Grok | Different lab, different failure shape, cheap enough to be useful. |
| Expectation | What happened here | My read |
|---|---|---|
| Grok 4.5 would be available as a real route | Met xai-oauth / grok-4.5 smokes passed. | It is wired and usable. |
| Cheap frontier tokens would matter | Met Cost was attractive on early probes; wall-clock was even more important later. | It can reduce spend if routed deliberately. |
| Hard reasoning would be credible | Met 17/18 on GPQA mini. | Legit second-opinion model. |
| Agentic coding would match GPT-5.5 | Mostly met Tied V3/V4, then beat GPT on V6 hidden/pass³ speed shape. | Good enough for real canary work. |
| Grok 4.5 would be the obvious default | Not yet at V6 Stale-note resistance lagged GPT-5.5 in V5/V6. | V6: promote canary, keep GPT keys. V7 later made Grok 4.5 the default with GPT-5.5 as fallback. |
V6 gave the canary map; v7 then promoted Grok 4.5 to default and that route is now applied live. The next proof is operational endurance: real Hermes work over several days with interruptions, rollbacks, review gates, and live maintenance — especially scheduler-sensitive tasks where v7 still looked weak.
~/.hermes/work/benchmarks/, including the v2, v3, v4, v5, and v6 harness directories.The GPQA artifact is sanitized and intentionally excludes full question text. Terminal-Bench artifacts exclude raw trajectories and task text. OAuth gauntlet artifacts publish route proof and scored summaries, not tokens, account IDs, or API keys. All headline routing claims above come from live API/OAuth smokes, Hermes child-agent runs, deterministic file/test artifacts, and Harbor artifacts from July 8–9, 2026.
Filed under: Hermes, OpenClaw, Grok, GPT-5.5, model routing, subscription models, Terminal-Bench, GPQA. Written from measured artifacts, not vendor vibes.