Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →In a reported sweep of 170 change-planning goals across 40 domains, Debashish Ghosal found three recurring structural failure types: plans relied on unverified dependencies, put steps in an unsafe order, or proposed weak rollbacks. The result is a useful warning about AI-generated plans—not proof that every AI planner makes the same mistakes. Ghosal’s counts come from his own PlannerCritic experiment, not an independent evaluation.
What the 170-goal experiment tested
Ghosal describes PlannerCritic as a planning-and-review system: one large language model (LLM) drafts a structured plan, deterministic checks enforce hard rules, and a second LLM critiques plans that pass those checks. A bounded revision loop either continues or escalates the plan to a human. The goals spanned 40 domains, including identity management, multi-agent operations, site reliability engineering, supply-chain policy, and FinOps. Ghosal’s account of the experiment is the source for its results and interpretation; the project repository provides system context and project-maintained field-test summaries, not independent replication.
As an Amazon Associate I earn from qualifying purchases.
In the v0.2.1 sweep, Ghosal reports 170 goals at a total cost of $0.49. He also reports 2.58 mean blockers per goal, 58 escalation decisions per 100 goals, 1.4 mean LLM calls per goal, and a median of 1.0 revisions to resolution. Approved plans had a reported median latency of 13.86 seconds; escalated plans, 27.82 seconds. These are measurements reported by the author for this experiment, not independently verified benchmarks or general performance rates.
Free tools Windows power users keep installed
One-click scans. No signup required.
The three recurring blocker types
Ghosal says the sweep produced 132 concrete blockers. Three families accounted for 121 of them. The counts describe blockers in this test, not the share of AI plans that fail across the industry.
Unverified dependencies: 57 blockers
A task assumes a condition that no earlier task establishes or checks. For example, a plan might cut traffic over completely without first confirming that earlier traffic stages were stable. The plan may read smoothly, but it skips the evidence that would justify taking the next step.
Unsafe sequencing: 46 blockers
A task appears before a prerequisite. Ghosal’s example is backfilling vectors before verifying index quality. Even if every task is relevant, the order can make the plan unsafe or undermine the result.
Weak rollback: 18 blockers
A rollback does not adequately address the state a high-impact change could leave behind. In Ghosal’s example, reverting dual-write mode does not correct possible inconsistencies created while dual writes were active. A reversal is not necessarily a recovery plan.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsWhat the proposed safeguards would—and would not—do
Check preconditions before approving a plan
Ghosal proposes a deterministic “precondition closer” that checks whether each task’s preconditions are established by an earlier task. He estimates that this check would eliminate 64 of the 132 blockers, or 48%. He explicitly presents that figure as a projection, not a measured result after applying a fix. It should be read as a hypothesis about the potential value of the check, not as proof that it works.
Enforce order, detect loops, and escalate unresolved plans
The article also describes topological auto-repair for task ordering and oscillation detection for plans that repeat cycles. These checks address different failure shapes: repair can reorder tasks when dependencies are known, while loop detection can flag revisions that fail to make progress. If a plan still cannot meet its constraints, escalation to a human provides a stop rather than treating a plausible-looking draft as safe.
The underlying design principle is to make non-negotiable invariants executable. As Ghosal puts it, “Deterministic gates catch what must be caught. The LLM critic is allowed to be unstable because it can only add findings, never suppress a gate blocker.” That division gives the model room to identify concerns while preventing its critique from overriding a hard-rule failure.
What the results say about model size and critique
Ghosal reports that using a larger model did not change the defect pattern in his tests. He also describes a critic trial where label and evidence drift did not result in zero-blocker approvals for seeded defects. Those are observations within this experiment; they do not establish that model size never matters or that critics are generally unreliable.
The distinction matters because a second model can add another review perspective without independently proving a plan correct. The project README says PlannerCritic does not execute approved plans and does not guarantee their correctness. Passing encoded checks is evidence only that those particular checks passed—not that every hazard has been found.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to apply the lesson to change planning
For teams using an AI planner, the practical takeaway is to separate rules that must always hold from judgments that can benefit from a model’s review.
- Represent prerequisites explicitly, then check that every task’s preconditions are established by earlier work.
- Encode known dependencies so the system can reject or safely reorder steps that violate them.
- Require rollback plans to address the state a change may create, not merely name a reversal action.
- Keep hard-rule failures blocking; do not let a model-generated critique suppress them.
- Escalate plans that cannot satisfy constraints or that keep cycling through revisions.
- Treat approval as permission to proceed to the next human-controlled process, not as a guarantee of correctness or a command to execute.
These safeguards can only enforce conditions that a team has identified and encoded. A deterministic gate is dependable about its stated rule; it cannot catch an omitted rule or a failure mode outside its design.
How far to generalize the findings
The experiment is a concrete account of recurring planning defects across a varied set of goals, but its evidence has limits. The article and project repository are author- and project-maintained sources; the reported sweep was not independently reproduced on the same corpus with the same blocker definitions. The 57, 46, and 18 counts therefore belong to Ghosal’s test, not to AI planning as a whole.
Ghosal also cautions that the three families may not cover domains his test did not include. He writes, “I also don’t have proof the fix is complete. The 64-of-132 projection is a projection, not a post-fix measurement.” The strongest defensible conclusion is narrower: in this reported sweep, missing or unverified prerequisites, unsafe ordering, and weak rollback dominated the blockers he identified. The experiment makes a case for encoding critical invariants and routing unresolved plans to human review; it does not establish that those steps eliminate every planning error.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
