Deep Review: turn a flawed proposal into a checked decision brief

1. Set up a small, inspectable test

Create a folder named cityparcel-review. Save the fixture below as proposal-v1.txt. Keep that file unchanged throughout the exercise. Later, add review-raw.txt, findings.csv and proposal-v2.txt. These are our filenames; the service does not create this folder structure for you.

Allow one free review for the first attempt. The live page offers a fixed Medium setting on the free path. A sampled tutorial suggests choosing other effort levels for free reviews; that advice conflicts with the interface checked today. Use the current interface as the authority.

Use the supplied fictional text for this first test. Google says documents can remain stored for up to a year and are not used for training without permission. An organization’s rules may restrict real client material. The optional paid API route is unnecessary for this exercise.

CityParcel dispatch pilot — fictional review exercise, draft 1

Purpose
We want to decide whether to expand a dispatch assistant to more routes.
The numbers below are invented for this exercise. There are no customer records.

Recorded observations
Baseline week: 120 deliveries; 96 on time; total dispatch cost $240.
Pilot week: 150 deliveries; 135 on time; total dispatch cost $450.
Baseline routes were suburban. Pilot routes were downtown.
Route assignment was chosen by the operations manager, without randomization.

Draft conclusions
1. The pilot achieved 95% on-time delivery.
2. On-time delivery improved by 5 percentage points from the baseline.
3. These observations prove that the assistant caused the improvement.
4. Our failure analysis reviewed all 150 pilot deliveries. Its analysis dataset
   contains the 135 on-time deliveries and excludes the 15 late deliveries.
5. Dispatch cost per delivery decreased during the pilot.

Additional observation
The late-delivery share fell from 20% to 10%, a 50% relative reduction.

Decision request
Identify which conclusions can be retained, corrected, or need more evidence.
Suggest the next measurement before expanding the pilot.
Do not invent traffic, customer satisfaction, or delivery-distance data.

2. Submit the fixture and keep its context separate

Open the official Deep Review link in the sources below. Select Sign in with Google and complete your own sign-in. Under Upload new document, select Paste text. Paste the complete contents of proposal-v1.txt into Paste document text.

Paste the following instructions into Add custom context (optional). Keep the instructions outside the proposal so you can distinguish the evidence being reviewed from the reviewer’s task. Check that both blocks are complete before choosing Upload Document for Review.

The FAQ directs you to the Your documents table for the result. Open the completed entry when it appears. Copy the complete review into review-raw.txt, preserving its wording and any cited sources. Record the date and the model/effort setting shown by your account. The signed-in submission and results screens were not tested during this research; stop and record any changed controls instead of assuming this procedure still matches.

Review this fictional operational proposal for arithmetic, internal consistency,
and the strength of its causal claims. Use the supplied observations as the
only numerical evidence. For each finding, identify the relevant sentence,
explain the issue, show any calculation, and suggest a precise correction.
Separate errors from unanswered questions. Retain statements that the numbers
support. If a conclusion needs new evidence, name the missing measurement.
Do not invent observations or external sources for this fictional company.

3. Check the review against the five seeded problems

Read the review before looking at the answer key. For each finding, identify its supporting sentence and redo the calculation or reasoning yourself. A correction only counts when you can explain why it follows from the supplied evidence. Style suggestions are optional and should remain separate from factual findings.

The table below is the expected answer key. It is not a captured Deep Review response. Different wording can pass if it reaches the same supported conclusion.

The additional observation about late deliveries is deliberately correct: a drop from 20% to 10% is a 50% relative reduction. If the review calls that arithmetic wrong, mark a false positive. This distinction also checks whether the reviewer can separate a percentage-point difference from a relative change.

For this fixture, pass the first review only if all five issues are correctly identified and it introduces no unsupported numerical claim. Record misses individually. A failure is useful information about this attempt; a pass on one small proposal cannot establish reliability across documents.

4. Reproduce the arithmetic independently

A calculator is sufficient. If you have Python 3, save this original helper as check_numbers.py in the exercise folder and run python3 check_numbers.py from that folder. It uses only Python’s standard features, reads no files and makes no network requests. Python is optional and is not a Deep Review dependency.

baseline_total, baseline_on_time, baseline_cost = 120, 96, 240
pilot_total, pilot_on_time, pilot_cost = 150, 135, 450
baseline_rate = baseline_on_time / baseline_total
pilot_rate = pilot_on_time / pilot_total
late_before = 1 - baseline_rate
late_after = 1 - pilot_rate
print(f"Baseline on time: {baseline_rate:.0%}")
print(f"Pilot on time: {pilot_rate:.0%}")
print(f"Change: {(pilot_rate - baseline_rate) * 100:.0f} percentage points")
print(f"Relative late-share reduction: {(late_before - late_after) / late_before:.0%}")
print(f"Dispatch cost per delivery: ${baseline_cost / baseline_total:.2f} -> ${pilot_cost / pilot_total:.2f}")

Baseline on time: 80%
Pilot on time: 90%
Change: 10 percentage points
Relative late-share reduction: 50%
Dispatch cost per delivery: $2.00 -> $3.00

This helper was executed locally on October 16 and produced the output shown. It checks the invented arithmetic only. The causal and dataset-coverage findings still require reading the proposal.

5. Write a correction log and a usable second draft

Create findings.csv from the template below. Replace pending with pass or fail after reading the actual review. Fill observed_evidence with the review’s relevant sentence or a short description of the omission. Keep this log separate from the original response so the audit trail stays intact.

In proposal-v2.txt, state the measured rates and costs, explain the route mismatch and remove the causal conclusion. Specify how the next pilot would compare similar routes and include late deliveries in the analysis. Do not invent the next pilot’s outcome. A reader should be able to distinguish an observed result from a proposed measurement.

id,result,observed_evidence,correction_verified
F1,pending,,pending
F2,pending,,pending
F3,pending,,pending
F4,pending,,pending
F5,pending,,pending

Ask a colleague to read the revised brief against the original data without seeing your score. They should obtain 80% and 90%, a ten-point change, and costs of $2 and $3. They should also identify the two unresolved design questions: comparable route selection and missing failure cases. If they cannot, revise the explanation before sharing it further.

Your finished handoff contains four files: unchanged source text, unchanged AI review, your evidence log and your corrected brief. Include the date, actual model label and any failed acceptance checks in the top of the evidence log or a short accompanying note.

When the result or the interface gets in the way

Additional reviews through an API key incur model usage charges, which depend on the selected options and document size. This guide makes no fixed-price estimate. A paid retry should answer a specific question, such as whether a changed instruction catches a known omission.

For a real proposal, replace the seeded answer key with independently checked claims, calculations and source links. Some issues will require a domain expert or fresh data. Keep those items visibly unresolved until that evidence exists.

Guide history

Revisions preserve the original instructions. A source recheck alone does not create a new revision.

Revision 1 · New original proposal fixture, exact public entry controls, five-issue answer key, locally checked arithmetic helper and correction handoff. Deep Review itself was not executed.