Samples

One task, four kinds of data.

An illustrative RL task with its verifier, demonstration, preference and rubric. Full sample sets on request.

legacy-batch-reconcileILLUSTRATIVE

Fix the nightly reconciliation report.

A GnuCOBOL batch job reads ledger.dat and writes report.txt with daily totals. The totals are wrong whenever a record has a negative (refund) amount. Fix it so report.txt matches the ledger, without changing ledger.dat or the report layout.

Software engineering · GnuCOBOL · RECON.cbl, ledger.dat, report.txt

Illustrative outcomes. Not a recorded model run.

Attempts and rewards
Attempt	Result	Why	Reward
Empty solution	FAIL	Totals still drop the refund sign	0.0
Hack: delete refund rows	BLOCKED	Ledger checksum changed	0.0
Hack: hand-write the report	BLOCKED	Report rebuilt from code	0.0
Reference solution	PASS	Report matches ledger	1.0

ledger_unchanged: Checksum of ledger.dat must match the original. Deleting or editing rows fails here.
report_rebuilt: report.txt is discarded and regenerated by running the submitted RECON.cbl. A hand-written report never survives.
totals_match: Daily totals in the rebuilt report must equal totals computed from ledger.dat, including refunds.
layout_unchanged: The report layout must match the original format.

1Read RECON.cbl.
2The amount field is PIC 9(7)V99: unsigned.
3The unsigned field drops the refund's minus sign.
4Change it to PIC S9(7)V99.
5Rebuild and rerun.
6report.txt now matches the ledger.

CHOSEN

Fixes the sign, keeps the layout

REJECTED

Rewrites the report layout

The chosen response follows every instruction. The rejected one changes the report layout, which the task forbids.

Correct totals for refunds2/2
Ledger left unchanged1/1
Root cause explained0/1

Score 3/4

Illustrative. Written to show format and checks; no model was run and no results were measured.

Request

Request full sample sets.

Name the domains and formats you want to review.