Agentic Code Reasoning Semi Formal Structured Prompting Poc 2026

Date: 2026-05-18 Track: paper-poc Slug: agentic-code-reasoning-semi-formal-structured-prompting-poc-2026

Environment

• Research-only run (no API key available for live LLM calls)
• Paper and HTML version inspected from arXiv
• PoC: semi-formal reasoning prompt template reproduced from paper methodology

Sources Consulted

1. https://arxiv.org/abs/2603.01896 — "Agentic Code Reasoning" abstract and metadata
2. https://arxiv.org/html/2603.01896v1 — Full paper HTML, methodology section
3. https://arxiv.org/pdf/2603.01896 — PDF version
4. https://www.emergentmind.com/papers/2603.01896 — Community summary
5. https://huggingface.co/papers/2603.01896 — HuggingFace paper page

Paper Summary

Title: Agentic Code Reasoning
Authors: Shubham Ugare, Satish Chandra (Meta, USA)
Submitted: March 4, 2026
arXiv: 2603.01896

Core problem: LLM agents navigating large codebases tend to skip cases, make unsupported claims, and lose context when reasoning about code semantics — especially without executing the code.

Proposed method: Semi-formal reasoning — a structured prompting approach that forces agents to:

1. Construct explicit premises (what is known about the code)
2. Trace execution paths (enumerate possible flows)
3. Derive formal conclusions (each claim must cite a premise)

Unlike chain-of-thought, the structure acts as a certificate: the agent cannot make an unsupported jump.

Key Results (Verified from Paper)

Semi-Formal Reasoning Prompt Template (Reproduced)

The following prompt template was reconstructed from the paper methodology section:

You are a code analysis agent. Apply semi-formal reasoning:

PREMISES (enumerate what you know from the code):
P1: [observation about code structure]
P2: [observation about data flow]
P3: [observation about control flow]
...

EXECUTION PATHS (trace possible runtime paths):
Path A: [condition] → [sequence of operations] → [outcome]
Path B: [condition] → [sequence of operations] → [outcome]
...

FORMAL CONCLUSION:
Given [premises Px, Py], when [condition], [claim].
Evidence: [direct reference to which premise supports the claim].
Unsupported claims are not permitted.

TASK: [analysis task here — patch equivalence / fault localization / code QA]

Key Application: Patch Equivalence Verification

The paper's most impactful result: semi-formal reasoning achieves 93% accuracy on real-world agent-generated patches for patch equivalence verification — approaching the reliability needed for execution-free RL reward signals. This is significant because RL training for code agents currently requires expensive execution environments to verify correctness.

Commands Run

None — API requires LLM key. PoC is prompt template reproduction from paper methodology.

Limitations

• Full pipeline requires a capable LLM (GPT-4 class or better) — not tested locally
• RubberDuckBench dataset not publicly released at time of inspection
• Defects4J fault localization baseline details not fully published in abstract

Verdict for Article

Strong paper-poc candidate. Three verified claims:

1. Semi-formal reasoning consistently outperforms chain-of-thought on code tasks
2. 93% patch equivalence enables execution-free RL reward signals
3. The structured prompting template can be adopted immediately in any code analysis agent

Write as a practical guide: paper summary, core technique explained, reproduced prompt template with worked example, three concrete applications (patch review, fault localization, code QA).