Public Reports Method

How the six agent-assisted analytical briefs under site/reports/public-*.html were generated for the Dorset HealthCare (RDY) demonstration site.

Agentic AI was used to accelerate discovery, preparation, checking and drafting. The analysis remains bounded by the public aggregate data available, and any operational interpretation would require local subject matter review, data owner confirmation and accountable sign-off.

Overview

1. Public-data pipeline (site/public-data/01–04_*.R) downloads official NHS aggregate publications, filters RDY rows, and creates demo CSV extracts in site/public-data/processed/.
2. Report render script (site/R/03_render_public_reports.R) reads those demo CSVs (and selected full RDY extracts where demo files are column-limited), computes descriptive statistics, and writes static HTML using the agent-assisted analytical brief template.
3. Styling (site/assets/nhs-report.css) applies NHS-inspired colours without NHS logo or implied endorsement.

See also: AGENT_ANALYTICAL_BRIEF_REFRAME_PLAN.md for the reframe specification.

Agent-assisted analytical brief template

Each public report follows this review-first structure (rendered by agent_brief_sections()):

1. Title and short subtitle (report header)
2. Public-data caveat box
3. What the agent was asked to do (business question, dataset, first-draft note)
4. Data used (sources, period, RDY filter, historic trend availability)
5. What this report can and cannot tell us (scope and limits)
6. Headline reading (3–5 plain-English takeaways)
7. Provider / RDY scope badge (MHSDS, CSDS, Talking Therapies only)
8. Priority callout (top 1–3 review flags where applicable)
9. Key findings explained — enriched table with period caption (see column model below)
10. Trend summary (unique headings per source where historic data exists)
11. Human validation checklist (4–6 plain-English bullets)
12. Bottom line — one paragraph for non-technical readers
13. Why this is useful — short box on agent triangulation value
14. Audit trail and source checks (collapsible: prompt excerpt, grouped findings, draft interpretation, supporting tables)

Grouped findings and draft interpretation are collapsed into the audit trail to reduce repetition above the fold.

Key findings explained — column model

Trend direction labels: Improving, Worsening, Broadly stable, Mixed / unclear, Not available from current extract, Rising, Falling, Stable, Volatile. Validation statuses use a separate badge column — never mixed into Trend.

Comparator priority: official standard (if in source) → published peer median/rank (NOF only) → previous comparable period → none stated honestly.

Source metadata for curated standards

Manually curated standards and enriched public context include:

Applied to NOF_METRIC_SPEC, Talking Therapies 75%/95% standards, CSDS CHS waiting-list signpost and CQC context. Rendered as inline <cite class="nhs-standard-cite"> in the standard column.

Period label convention

Every main table and trend chart includes an explicit reporting period or trend window via period_caption_html(). Where headline window differs from supporting extract window, both are stated (e.g. six-month display vs eight-month stacked extract).

Reports are indexed on site/draft-reports.html as worked examples of agent-assisted analytical briefs.

Demo CSVs used

If a demo CSV is missing, the render script writes a data-availability page rather than inventing figures.

Statistical methods applied

All analysis is descriptive and bounded by published aggregates:

• Latest-period values from the demo extract reporting period columns
• Peer comparison (NOF): rank, median, lower/upper quartile where present — not significance testing
• Counts and sums where numeric; suppressed values (*) excluded from numeric calculations
• Top-N tables and bar charts (base R HTML) for largest numeric measures
• Plain-English metric/theme commentary cards with agent flags (Potential strength, Review locally, Watch / clarify, Definition check required, Source validation only, Trend not available)
• Key figures explained table with standardized trend direction labels: Improving, Worsening, Broadly stable, Mixed / unclear, Not available from current extract, Source validation only, Definition check required
• Comparator priority: official standard (if in source) → published peer median/rank (NOF only) → previous comparable period → none stated honestly
• Trend tables and charts where downloaded time-series extracts contain ≥2 comparable periods for the same measure (latest vs previous month, absolute and percentage change; rolling mean only if ≥3 periods)
• Source presence checks (urgent/diagnostics report) for RDY row existence
• Missingness / suppression counts flagged explicitly

Not used: p-values, causal inference, forecasting, or blending with synthetic data.

Why causal claims are avoided

Public aggregate data:

• May be provisional or revised (MHSDS, CSDS, Talking Therapies)
• Uses suppression and rounding
• Spans different breakdowns (ICB resident vs provider, referral source splits)
• Cannot explain local operational context (pathways, workforce, definitions)

The briefs describe what a governed agent can produce from the public file and what a human must still check — not why performance changed.

Public aggregate vs local operational data

Business partners bridge these layers — public data informs questions; local data answers them.

How the Cursor agent assisted

• Discovered and downloaded official publication files
• Documented sources in DATA_SOURCE_REGISTER.csv
• Filtered RDY rows with documented matching rules
• Created demo extracts and the agent brief render script
• Drafted per-report questions, prompt excerpts, process steps, first-draft narrative and reviewer questions

Human reviewers must still:

• Confirm metric definitions against NHS England / NHS Digital guidance
• Verify publication status (provisional vs final)
• Align periods with local reporting cycles
• Obtain named owner sign-off before operational or Board use

Rerun instructions

# 1. Refresh public data (requires network)
cd site/public-data
Rscript 01_download_public_data.R
Rscript 02_inspect_public_data.R
Rscript 03_filter_dorset_healthcare.R
Rscript 04_create_demo_extracts.R

# 1b. Historic trend stack (optional — requires network; does not overwrite existing raw files)
Rscript 05_download_historic_public_data.R

Historic pipeline outputs (when stacking succeeds):

Register and run summary: HISTORIC_SOURCE_REGISTER.csv, HISTORIC_PUBLIC_DATA_RUN_SUMMARY.md.

# 2. Render public HTML reports (offline OK if CSVs exist)
Rscript site/R/03_render_public_reports.R

# 2b. Standalone post-render validation (also runs automatically at end of step 2)
Rscript site/R/04_validate_public_reports.R

Post-render validation (04_validate_public_reports.R)

After each render, the pipeline checks all six public-*.html files for:

• Duplicate <h2> section headings within a file
• Validation wording misplaced in the Trend column (Definition check, Finance, Source validation only)
• Talking Therapies M019–M021 and M019–M022 totals vs demo_talking_therapies.csv
• Chronological order of month labels in time-series bar charts
• Missing period captions on Key findings explained sections
• Missing bottom-line and why-this-is-useful sections
• KH03 quarterly snapshot wording on the urgent-care report

Validation fails the render with a numbered issue list if any check fails. Add new check_*() functions as failure modes are discovered.

The Report Analysis and Improvement Agent provides a human-judgement layer above this script — standards interpretation, readability, revised wording and publication refusal — with a worked example on the operating model page.

Known limitations in this run

• FFT: org-level RDY rows not found in downloaded summary XLSX — see metadata/fft_manual_download_needed.md
• KH03: latest snapshot in public file may lag NHS England “latest quarter” — verify on source site; trend uses recent snapshots only (not full 2007–2024 history)
• A&E: RDY row reflects other emergency admissions, not ED attendances (no ED at RDY); historic stack is source validation only
• DM01: audiology may dominate activity; historic trend requires full-extract monthly ZIPs (not aggregate REVISED bundles)
• MHS23: stacked from MHSDS main_data monthly files — not the Provider time-series bundle
• CSDS / MHSDS / A&E trends: provisional monthly data; descriptive period-on-period only — not causal performance claims

Recommended next improvement

• Obtain FFT org-level or setting-level rows to fill the assurance gap (manual download steps in metadata)
• Align KH03 to the latest NHS England publication quarter when the overnight page updates
• Add more DM01 full-extract months if FY scrape misses older months
• Optional Talking Therapies outcome measures (M192, M186) for a separate outcome brief

See NEXT_COMMENTARY_IMPROVEMENTS.md, HISTORIC_PUBLIC_DATA_EXPANSION_PLAN.md, and FINAL_REPORT_QA_SUMMARY.md for the latest report QA pass.

Traceability and verification

Every public brief figure should be traceable back through the pipeline:

1. Report HTML table or KPI → demo CSV in site/public-data/processed/demo_*.csv
2. Demo CSV → full RDY processed extract (site/public-data/processed/rdy_*.csv)
3. Processed extract → raw public file in site/public-data/raw/ (filtered to RDY only)
4. Raw file → NHS England / NHS Digital source URL in DATA_SOURCE_REGISTER.csv

All reports include a How to verify these figures section at the bottom of the narrative (before the final human-review warning in the template flow, with human review as the closing gate).

NOF performance overview (full audit)

The NOF brief includes plain-English agent commentary for each displayed metric (see “Agent commentary on selected metrics”), a How to read this table guide, and machine-readable audit outputs:

• site/public-data/metadata/public_report_audit_nof_overview.csv — one row per displayed metric
• site/public-data/metadata/public_report_audit_nof_overview.md — plain-English verification guide

Metric polarity and definitions should be confirmed against the official NHS England NOF technical metric specification — this long-read was not bulk-downloaded in the public-data pipeline.

Audit summary and per-metric detail appear in collapsible blocks under How to verify these figures.

Important: Value, Median_value, and Rank in the NOF brief are NHS England published fields from the official NOF data CSV. This demo does not recalculate medians, ranks, or league tables.

To verify a figure manually:

1. Open the audit CSV or HTML verification section for the metric.
2. Use RDY_row_identifier to find the row in the processed and raw NOF files.
3. Confirm Value, Median_value, and Rank match exactly.
4. Check Comparator_rows_used against the count of numeric trust rows in the raw file for the same Quarter + Metric_ID + Reporting_date.
5. Confirm rank direction and metric definition against NHS England metadata before operational use.

Regenerating reports also regenerates the audit CSV and MD:

Rscript site/R/03_render_public_reports.R

Other public briefs (verification appendix)

Briefs for MHSDS, CSDS, Talking Therapies, assurance, and urgent/diagnostics include How to verify these figures linking to demo CSVs, filter notes, and governance documentation. Technical detail (full A&E columns, assurance index with all columns, CQC context note) is in collapsible verification blocks where appropriate.

Human reviewer checklist

Before relying on any public report figure operationally:

• Confirm publication quarter and whether a newer release supersedes the extract
• Check provisional vs final status and suppression rules
• Verify RDY matching against local ODS register
• Do not treat published rank or median as a significance test
• Confirm metric polarity (higher/lower better) against NHS England definitions
• Obtain accountable sign-off from the relevant data owner