Rubric-Conditioned Self-Distillation Turns Rubrics Into Training Signal, Not Just Judge Paperwork

Rubrics are where model evaluation pretends to be mature. A human or LLM judge writes down the criteria, assigns weights, and then, too often, the whole thing gets crushed back into a single reward number. That is like writing a thoughtful code review and letting CI see only “approved” or “failed.” Rubric-Conditioned Self-Distillation is interesting because it refuses to throw away the review comments.

The paper’s argument is straightforward: if structured rubrics are useful enough to judge model outputs, they are useful enough to shape training before the answer is finished. RCSD trains a rubric generator, then uses rubric-conditioned teacher guidance to provide token-level supervision on the student model’s own rollouts. It is a middle path between reward-only reinforcement learning and reference-answer imitation. The model still learns from its own trajectories, but the teacher is conditioned on what a good answer must satisfy.

That matters because scalar rewards are blunt instruments. A final answer can be mostly right with one missing constraint, or mostly wrong with one excellent intermediate step. A reward-only method sees a compressed outcome. A reference-answer distillation method may overfit to one blessed reasoning path. Rubrics can express “this criterion is essential,” “this one is important,” “this is optional,” and “this is a pitfall.” RCSD tries to keep that structure alive during training.

The method has two stages. Stage I learns to generate instance-specific rubrics from questions. Stage II trains a reasoner using rubric-conditioned teacher guidance on on-policy student trajectories. The rubric items include a title, description, and weight bucket such as Essential, Important, Optional, or Pitfall. The training data includes about 10,000 samples for rubric generation and 30,000 samples for reasoning generation, built from RaR-Science, RubricHub, and filtered Natural Reasoning data.

The useful signal is inside the criteria

The main experiments use Qwen3-8B with LoRA rank 64 and alpha 128, AdamW at 5e-6, batch size 32, FlashAttention 3, and a maximum completion length of 4096 for the reasoner. Evaluation covers GPQA-Diamond, SciBench, PIQA, ResearchQA, RaR-Science, and RubricHub, with transfer checks on MedMCQA and PubMedQA. RCSD reaches a 70.6 average, beating the Qwen3-8B base at 65.9, GRPO at 69.2, GRPO-Rubrics at 69.6, and OPSD at 69.7.

The per-benchmark numbers make the story clearer: 64.5 on GPQA-Diamond, 70.8 on SciBench, 90.8 on PIQA, 68.6 on ResearchQA, 73.1 on RaR-Science, and 55.7 on RubricHub. The strongest gains land where the task itself is rubric-heavy: +8.2 points on ResearchQA and +4.9 points on RubricHub versus the base model. Medical transfer is modest but not catastrophic: RCSD improves from 64.5 to 65.8 on MedMCQA and from 74.2 to 75.1 on PubMedQA, while GRPO and OPSD lead individual medical benchmarks.

The ablation that deserves attention is rubric source. Generated rubrics are close to ground-truth rubrics on GPQA-Diamond, SciBench, and PIQA: 64.5 versus 65.2, 70.6 versus 71.0, and 90.8 versus 91.0. Generated rubrics average 8.4 criteria versus 7.5 for reference rubrics, with similar token length. That does not prove generated rubrics are safe for every domain, but it does suggest the model can produce structured supervision good enough to matter at scale.

For engineering teams, the first takeaway is not “fine-tune RCSD tomorrow.” It is: stop logging only final judge scores. If you use LLM-as-judge for agents, support replies, code review, research assistance, policy checks, or document drafting, store the criterion-level judgments. Which requirement failed? Was the answer ungrounded, incomplete, unsafe, overlong, wrong format, missing citation, bad at constraint handling, or reasoning from the wrong evidence? Scalar reward logs feel tidy until you want to improve the system. Then you discover you erased the debugging signal.

This is especially relevant for agent systems. Agent quality is rarely one-dimensional. A coding agent might produce a correct patch while ignoring project style, skipping tests, or editing unrelated files. A research agent might cite sources but miss the user’s constraint. A customer-support agent might be factually right but violate tone, privacy, or escalation policy. Rubric-level supervision lets teams separate those failures instead of smearing them into one “bad answer” bucket.

The caveats are real. RCSD’s average gain over OPSD is 0.9 points, useful but not a revolution. Open-ended evaluation still leans on LLM judges, so judge calibration and rubric quality matter. And generated rubrics can be confidently incomplete, especially in regulated or domain-specific work. A medical, legal, finance, or security rubric should not be treated as correct just because a model produced it with nice bullet points. The production pattern should be human-reviewed rubrics for high-stakes domains, generated rubrics for scale, and audit trails for both.

The broader signal is that post-training is moving from answer verification toward structured quality interfaces. That is good. Unit tests work for code. Verifiers work for math. But much of professional work lives in the messy middle where quality is made of criteria, tradeoffs, and pitfalls. RCSD is not the final answer, but it points at the right abstraction: teach models the shape of quality, not just whether the final artifact got a green check.

Sources: arXiv, RCSD code, DeepSeekMath / GRPO, GPQA, LiveCodeBench