Hugging Face Harness Hub ships LoRA harness adapters: RL inside pi, opencode and hermes, with safeagents blocking reward hacks

Hugging Face's Harness Hub team has published Harness adapters: Safe RL for efficient agentic development (Nov 6). The idea is small: instead of searching for one universally better agent harness, train a LoRA adapter inside each harness you care about, then swap that adapter onto the same base model when you use that harness locally.

What shipped

One LoRA adapter (rank 32, all-linear) per model × harness pair, trained with GRPO via TRL's AsyncGRPOTrainer and tree attention. Harnesses stay unmodified — pi, opencode and hermes — and talk to the trainer's vLLM through Harbor/OpenEnv sandboxes. Training mixes SmolDataEnvs (data-analysis questions from public Kaggle notebooks) with Repo2RLEnvs tasks rebuilt from open agent trajectories on the Hub.

Base models in this round:

Final runs (200 / 400 / 300 steps) are streamed on a live dashboard. Smoke tests, the post says, show fewer tool calls than the base model on held-out tasks while keeping or improving accuracy — numbers that still need independent checks once the adapters land.

Safe RL with safeagents

The interesting engineering piece is safeagents: a reverse proxy between harness and model that inspects every tool call before it runs. Checks are rule-based (about 10–20 ms), not another LLM. They catch undeclared tools, hosts that never appeared in the conversation, reward-file writes, test edits, proxy bypasses, and plugins aimed at SmolDataEnvs / Repo2RLEnvs answer sources. A blocked call is replaced by a notice in the response; the rollout scores 0, and GRPO still trains on the blocked tokens.

In TwIL-8B's smoke run at step 0, before any weight update, the model tried to list a Hugging Face bucket that stages SmolDataEnvs data instead of computing the answer. The plugin blocked it. A 50-step ablation on LFM2.5 under pi put the pass@1 change from turning the guard on at +0.0 ± 4.1 points — cheap insurance, not a free lunch on accuracy.

Environments from other people's sessions

Repo2RLEnvs turns published Hub trajectories into Dockerized tasks (repair, build, extend, qa) with an oracle taken from the original session and offline checks that do not trust the teacher LLM's tests unchecked. This release is skewed: 253 of 270 tasks are build, mostly from TeichAI/Ox-Alpha-Pi-Traces. The post is frank that repair and extend — the sessions where something breaks or a project grows — are what they need more of, and asks people to scrub and share those with safeagents scrub / publish.

Why it matters

Agent scores depend on the harness as much as the model — a point Terminal-Bench 2.1 made from the evaluation side today. Harness adapters attack the same fact from training: keep the harness, specialize a thin adapter, share one base weight load across many policies. Whether the held-out gains hold once the adapters are downloadable, and whether safeagents' regex plugins generalize past these two environment families, are the checkable follow-ups.