> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trybrein.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 02 DURABLE INVESTIGATION CORE

# Phase 2 — Durable Investigation Core

## Goal

Wrap the validated discovery engine in a resumable, idempotent resource that runs outside an HTTP
request and survives deploys without duplicating candidates.

## Domain contract

Create one explicit `Investigation` kind for this slice: `company_expansion`.

States:

`draft -> scoping -> planned -> queued -> running -> completed | partial | failed | cancelled`

The platform owns a closed plan:

1. scope;
2. discover (one to three bounded rounds);
3. resolve identity and dedupe;
4. verify;
5. enrich from semantic warehouse operations;
6. prioritize;
7. synthesize.

The model may supply content inside typed steps. It may not add steps, choose budgets, decide to
loop indefinitely, mint evidence ids or set terminal status.

## Persistence

Add exactly three tables in one forward-only migration:

### `investigations`

Parent lifecycle, workspace, creator, optional source thread, objective, status, typed plan JSON,
closed clarification answers, counters, heartbeat, cancellation request, failure reason and
timestamps.

### `investigation_events`

Append-only curated replay with monotonic sequence per investigation. Store a closed `kind`, a
`label_key`, and numeric/enum detail only. Never store model reasoning, queries, tool/provider names
or raw result payloads.

### `investigation_candidates`

Persist the existing `WebMarketCandidate` projection: deterministic candidate id, organization,
signal, geography, published date, evidence ids, independent-source count, status, score, ranking
reasons, warnings and optional resolved entity id.

Unique key: `(investigation_id, candidate_id)`.

Do not create `investigation_steps`; step state lives in the versioned, runtime-validated plan JSON.
Do not reuse the orphaned `WorkSession*` or `gtm_research_runs` schema.

## Execution

* Add a dedicated pg-boss queue following `StewardQueueService`.
* Reallocate the obsolete `workSessionWorkflow` connection-budget slot; do not increase the total
  connection budget without evidence.
* Use compare-and-set for state transitions.
* Persist a step outcome before marking the step `done`.
* On retry, skip `done`/`skipped` steps and UPSERT candidates from any re-run step.
* Add a reaper for stale queued/running rows following existing cron/state-machine conventions.
* Use `retryLimit: 2` only while every step remains idempotent. Pin that invariant in tests/comments.
* Reuse `AgentRunsService` for each model attempt and `ModelUsageService` with surface
  `investigation`. Do not write either ledger directly.

## Cancellation and partial results

* `cancel_requested_at` is checked between steps and aborts an in-process provider call when
  possible.
* Cancelled and partial runs keep every candidate already persisted.
* A web failure ends web discovery; it must not delete warehouse enrichment or prior rows.
* `partial` is an explicit terminal state with incomplete steps and warnings, not a disguised error.

## Tests

* Legal and illegal state transitions.
* Two concurrent starts enqueue once.
* Re-entry from every step.
* Process death after candidate UPSERT but before step completion.
* Replayed step produces no duplicate rows.
* Tenant isolation.
* Stop preserves rows.
* Reaper handles stale heartbeats and leaves fresh jobs alone.
* Entity and migration registration guardrails.
* Architecture guard: `apps/api/src/chat/**` and `apps/api/src/investigations/**` cannot import each
  other.

## Acceptance

Kill the worker while discovery is running, restart it, and observe one terminal investigation with
no duplicate or lost persisted candidates. The ordinary Chat test suite must pass without modifying
Chat behavior.

## Rollback

Disable queue startup, remove `InvestigationsModule` registration, and revert the phase commit.
Tables remain inert until a separate forward-only removal migration is deliberately approved.