Form Sessions and Abandonment Recovery

Purpose

When a visitor opens a Prima-embedded lead form, a form_sessions row is minted before they type anything, carrying UTM/click-id attribution and an entry URL (FormSessionService.create_session, session_service.py 45 to 121), so that if they never finish, the workspace still has something: who arrived, from where, and (if they typed it before leaving) an email. The invariant the model docstring implies is that partial field data merges cumulatively into that row as the visitor types (update_partial_data, models/forms.py 680 to 684) and survives to be read back for a recovery email - but that invariant does not hold in this code: the merge is a plain in-place dict.update() on a JSONB column that is not wrapped in SQLAlchemy's MutableDict, so the write never reaches the database (Gotchas, test_advances_page_and_documents_partial_data_merge_bug), and the endpoint that is supposed to persist it 500s on the same request regardless (Gotchas). What this recipe reliably delivers, as measured, is attribution capture on session start and a full submission at completion; the "save-as-you-go" half of the name is present in the code and wired into the real frontend, but does not work end to end at this commit.

When to use

Read it before touching FormSessionService.create_session/save_partial/capture_email/complete_session/mark_abandoned/get_recovery_candidates/get_session_stats, before adding a field to FormSession, or before wiring a background job to call mark_abandoned. Trigger phrases: "form session", "session_token", "partial_data", "auto-save form", "abandoned form recovery", "recovery candidates", "why is completion_percentage crashing", "why doesn't a form ever get marked abandoned", "UTM/gclid/fbclid on a form".

Not this recipe: the actual public submission pipeline, POST /f/{embed_code}/submit and /submit-ai in public_forms_router.py (validation, bot-protection scoring, lead creation via services/forms/lead_integration.py, the secure-medical-email mode, Quentn sync, appointment invites) - a neighbouring recipe; those endpoints read a FormSession only for the bot-timing check (_check_timing, public_forms_router.py 107 to 117) and to fold its get_attribution_data() into the submission's tracking_data (line 668), then mark it completed by hand (session.mark_completed(), lines 708 and 1158) - they never call FormSessionService.complete_session, so that method's own Divination "Lead" conversion send (_track_form_completed, session_service.py 596 to 631) is unreachable from the real public submit path (Gotchas); the form-builder/template CRUD (FormTemplate, FormField) and the AI-drafting endpoints, forms-leads/ui/form-builder-page's own backend; the lead-magnet upload/deliver flow (forms_router.py, "Lead Magnet Endpoints", from line 1316); Divination itself (services/divination_client.py), this recipe only calls track_event/identify/send_conversion on it.

Stack assumptions

At 62ba4fc: fastapi==0.118.2, synchronous handlers on sqlalchemy.orm.Session (SQLAlchemy==2.0.43), pydantic==2.12.0 for every request/response schema, alembic==1.16.5 for the migration chain, psycopg2-binary==2.9.10, httpx==0.28.1 inside services/divination_client.py. Foundations assumed: S4 (fastapi-project-layout) for get_db and the two routers' mount (app_fastapi.py 928, 949, both under prefix="/api", so the two live paths are /api/forms/... and /api/f/{embed_code}/...); S8 (alembic-migrations) for the chained revision and the add_column_if_missing/create_foreign_key_if_missing/create_index_if_missing helpers from migrations/_phantom_guard.py the migration uses; S5 (auth-dependency-pattern) for get_current_user, gating only the four admin session endpoints in forms_router.py; S6 (workspace-scoping) for the WorkspaceID dependency those same four endpoints take and use to filter template_id.in_(workspace_template_ids); S7 (error-envelope) because every failure here is a plain HTTPException(status_code=..., detail="<sentence>"), which renders as Shape B (the Labs envelope) through http_exception_handler, never Shape A (neither router is mounted under the gateway prefix); S9 (test-layout) for the db_session fixture and the create_all()-built schema the test file runs against (Gotchas: this means the test schema and the migration-built schema are not the same schema, see Data model).

Environment, all in core/config.py: DIVINATION_URL (line 1012, default "https://divination.example.com"), DIVINATION_API_KEY (1013, Optional[SecretStr], unset by default), DIVINATION_SERVICE_NAME (1014, default "labs"). Fail behaviour: DivinationClient._is_enabled() (divination_client.py ~48) is bool(self.api_key); with no key every identify/track_event/send_conversion call returns {"success": False, "error": "tracking_disabled"} without raising, so an unconfigured Divination integration never blocks a session write. Every Divination call this recipe makes is additionally wrapped in its own try/except Exception: logger.warning(...) (session_service.py 564 to 565, 593 to 594, 630 to 631, 656 to 657) - except that mark_abandoned's per-row analytics-event dict is built before the guarded _log_analytics_event call and outside any try/except of its own (322 to 332), which is exactly where the crash in Gotchas originates.

Data model

form_sessions, one row per visitor session (models/forms.py 561 to 701; created by migrations/versions/20251205_add_form_sessions.py 31 to 69, the live shape below is the ORM model, which is not identical to what the migration creates - see the note after the constraint):

form_sessions (
  id                      uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  template_id             uuid NOT NULL REFERENCES form_templates(id) ON DELETE CASCADE,
  session_token           varchar(64) UNIQUE NOT NULL,           -- secrets.token_urlsafe(48)[:64]
  user_id                 uuid REFERENCES users(id) ON DELETE SET NULL,
  anonymous_id            varchar(255),                          -- tracking script's sessionStorage id
  utm_source              varchar(255), utm_medium varchar(255), utm_campaign varchar(255),
  utm_content             varchar(255), utm_term varchar(255),
  fbclid                  varchar(255), gclid varchar(255),
  landing_page            text, referrer text, entry_url text,
  ip_address              inet, user_agent text,
  partial_data            jsonb NOT NULL DEFAULT '{}',           -- field values saved so far
  current_page            integer NOT NULL DEFAULT 1,
  status                  varchar(50) NOT NULL DEFAULT 'in_progress',
  email_captured          varchar(255),
  started_at              timestamptz NOT NULL DEFAULT now(),
  last_activity_at        timestamptz NOT NULL DEFAULT now(),
  completed_at            timestamptz,
  recovery_email_sent     boolean NOT NULL DEFAULT false,
  recovery_email_sent_at  timestamptz,
  created_at              timestamptz DEFAULT now(),             -- see note below
  updated_at              timestamptz DEFAULT now()               -- see note below
);
CHECK form_sessions_status_check: status IN ('in_progress','completed','abandoned')

created_at/updated_at are declared on the ORM model (models/forms.py 611 to 612, onupdate=func.now() on the second) but 20251205_add_form_sessions.py's create_table (31 to 69) does not include them. A separate, loose file, backend/migrations/add_form_sessions_timestamps.sql (own comment: "The model expects these but the original migration didn't include them"), ALTER TABLE ... ADD COLUMN IF NOT EXISTS for both - but it lives at backend/migrations/ root, not under migrations/versions/, carries no Alembic revision id, and is not referenced by scripts/migrate.sh (which runs only alembic upgrade head, per its own header comment) or by any other migration in the chain (grep -rln add_form_sessions_timestamps across the repo, outside .git, finds only the file itself). Whether this column patch has actually been applied to any environment that runs off the alembic chain alone is not determinable from the repository - that is a genuine gap, not an assumption; a rebuild either folds these two columns into the create-table migration or ships this file as a proper chained revision.

A second, smaller drift, also measured: the index set the model declares in __table_args__ (618 to 630: idx_form_sessions_email, idx_form_sessions_last_activity, idx_form_sessions_anonymous_id, plus template_id/status/utm_source/started_at) is not the index set the migration creates (72 to 79: idx_form_sessions_email_captured, idx_form_sessions_last_activity_at, idx_form_sessions_user_id, idx_form_sessions_utm_campaign, plus the same four). A database built by alembic upgrade head alone has the migration's eight names, none on anonymous_id; a database built by the test suite's Base.metadata.create_all() (S9) has the model's seven, none on user_id or utm_campaign. Neither environment has both.

What is NOT stored: a raw card number or payment detail (this recipe has none); the visitor's password or auth token; anything beyond what partial_data and the tracking columns above hold, all of which is form-field content and attribution, entered by the visitor themselves.

form_submissions.session_id (nullable FK, ON DELETE SET NULL, models/forms.py 450) and .tracking_data (JSONB, added by the same migration, 92 to 97) are this recipe's only footprint on the neighbouring submission table; FormSubmissionService/the submission pipeline recipe owns the rest of that table.

API surface

The session lifecycle is implemented twice, once behind no auth at all (forms_router.py, mounted at /api/forms) and once behind an embed code (public_forms_router.py, mounted at /api/f). Neither the "no auth" router nor the embed-scoped one re-verifies that a session_token belongs to the embed_code/workspace in the path on any call after start - a session token is a 64-char secrets.token_urlsafe value and is treated as the only credential needed for save/email/complete/get.

Route	Auth	Tenancy	Status codes	Notes
POST /api/forms/sessions/start (forms_router.py 903 to 947)	none (docstring: "No authentication required")	none	201 StartSessionResponse; 404 ValueError (bad template_id); 500 any other exception	Body StartSessionRequest (schemas/forms.py 329 to 333: template_id, optional tracking: TrackingData, optional entry_url)
GET /api/forms/sessions/{session_token} (950 to 993)	none	none	200 FormSessionResponse; 404 not found; 500 TypeError on every call that reaches the response	line 992 calls session.get_completion_percentage() with zero arguments against a model method whose signature is get_completion_percentage(self, total_fields) (models/forms.py 696) - always raises
POST /api/forms/sessions/{session_token}/save (996 to 1040)	none	none	200 FormSessionResponse; 404 not found; 500 TypeError (line 1039, same bug)	the JSONB merge itself is also silently dropped server-side regardless (Application logic, Gotchas)
POST /api/forms/sessions/{session_token}/email (1043 to 1088)	none	none	200 FormSessionResponse; 404 not found; 500 TypeError (line 1087, same bug)	email_captured scalar column DOES commit before the response-building crash (Application logic)
POST /api/forms/sessions/{session_token}/complete (1091 to 1113)	none	none	200 FormSubmissionResponse; 404 not found	returns FormSubmissionResponse, never touches get_completion_percentage, does not crash
GET /api/forms/sessions (1120 to 1185, admin)	get_current_user	WorkspaceID -> workspace_template_ids	200 List[SessionListResponse]; 500 TypeError (line 1182, same bug)
GET /api/forms/templates/{template_id}/sessions/stats (1188 to 1226, admin)	get_current_user	WorkspaceID, template ownership checked (1201 to 1211)	200 SessionStatsResponse; 404 template not in workspace; 500 sqlalchemy.exc.DataError whenever the template has ANY abandoned session	get_session_stats's avg_completion_percentage subquery (Application logic)
GET /api/forms/sessions/recovery-candidates (1229 to 1274, admin)	get_current_user	WorkspaceID -> workspace_template_ids	200 list of dicts; 500 TypeError (line 1268, same bug)
POST /api/forms/sessions/{session_token}/mark-recovery-sent (1277 to 1313, admin)	get_current_user	WorkspaceID, joined against FormSession.template_id == FormTemplate.id (1291 to 1296)	200 {"status": "marked", ...}; 404 not found or not in workspace	does not touch get_completion_percentage, does not crash
POST /api/f/{embed_code}/session/start (public_forms_router.py 1393 to 1425)	none	embed_code resolves a FormEmbed -> template_id	200 StartSessionResponse	no explicit status_code, so 200 not 201, unlike the admin-router twin
POST /api/f/{embed_code}/session/{token}/save (1428 to 1445)	none	none (token only)	200 {"success": true, "last_saved": ...}; 404 not found	does not build a FormSessionResponse, so it does not hit the get_completion_percentage bug
POST /api/f/{embed_code}/session/{token}/email (1448 to 1465)	none	none (token only)	200 {"success": true, "email_captured": true}; 404 not found	email is a query parameter (email: str = Query(...), line 1452) here, a request body on the admin-router twin (CaptureEmailRequest) - the two "same" endpoints have different contracts
GET /api/f/{embed_code}/session/{token} (1467 to 1511)	none	none (token only)	200 FormSessionResponse	the one place in the whole surface that calls get_completion_percentage(total_fields) correctly (line 1487: total_fields = len(template.fields) if template else 0), so this specific route does not crash

Application logic

create_session (session_service.py 45 to 121), commit order: verify FormTemplate exists or raise ValueError (65 to 70); build a TrackingData from the request or an empty one (73); construct FormSession with partial_data={}, current_page=1, status='in_progress', and every UTM/click-id/context field copied verbatim from tracking/request/the caller's ip_address/user_agent (76 to 98); add, commit, refresh (100 to 102); fire-and-forget _track_form_started (Divination track_event, own try/except) and _log_analytics_event(event_type='form_started') (104 to 117, the latter added to the session but not separately committed, "let the caller handle the transaction", _log_analytics_event's own comment, 677).

save_partial (137 to 173): load by token, None if missing (153 to 157); no-op and return the session unchanged if status == 'completed' (159 to 161); otherwise session.update_partial_data(request.partial_data) then session.current_page = request.current_page, last_activity_at = now(), commit, refresh (163 to 169). update_partial_data (models/forms.py 680 to 684) is self.partial_data.update(new_data) - an in-place mutation of a plain JSONB-typed column, not MutableDict-wrapped, so SQLAlchemy's dirty-tracking never sees it and commit() writes nothing back for partial_data specifically; current_page, a scalar column reassignment, persists correctly. Measured directly: test_advances_page_and_documents_partial_data_merge_bug seeds a row with partial_data={"name": "Ada"}, calls save_partial with {"company": "Analytical"}, and asserts the refreshed row is still exactly {"name": "Ada"}.

capture_email (175 to 212): load by token; lowercase+strip the email onto the scalar email_captured column; ALSO call update_partial_data({'email': email}) (202, same dead-write bug as above); commit, refresh; then _identify_in_divination (own try/except: divination.identify plus a form_email_captured track_event, neither reaches the local FormAnalytics table). test_lowercases_strips_and_stores confirms the scalar column persists and is normalized while partial_data.get("email") stays None.

complete_session (214 to 288): load by token; if already status == 'completed', return the existing FormSubmission for that session (idempotent, no duplicate row, 235 to 241, test_idempotent_returns_existing_submission); else session.mark_completed() (sets status='completed', completed_at=now()), build tracking_data = session.get_attribution_data(), construct and commit a FormSubmission carrying request.data verbatim plus that tracking snapshot (250 to 263), then fire _track_form_completed (Divination track_event + send_conversion(event_name='Lead'), only if session.email_captured is set) and _log_analytics_event(event_type='form_submitted').

mark_abandoned(minutes_inactive=30) (290 to 340): query every status == 'in_progress' row with last_activity_at < now() - minutes_inactive (304 to 312); per row, session.mark_abandoned() (status only, no email/partial_data change), then if email_captured is set, _track_form_abandoned (own try/except), then unguarded _log_analytics_event(..., event_data={'completion_percentage': session.get_completion_percentage(), ...}) (322 to 332). Because get_completion_percentage is called with zero arguments against a method that requires total_fields, this raises TypeError on the first stale row and the whole sweep aborts, uncaught, before count > 0 is ever reached and before the loop's db.commit() (336 to 338) runs. No code path in this repository calls mark_abandoned (grep -rn mark_abandoned outside models/forms.py, session_service.py and the test file finds nothing: no scheduler job, no admin route, no cron entry), so this is a documented, load-bearing, unreachable-in-production bug: abandonment is never detected automatically at this commit, whether or not the TypeError is fixed.

get_recovery_candidates/mark_recovery_sent (342 to 400) are correct and tested (test_selects_abandoned_with_email_not_yet_sent_in_window, test_workspace_scoping_excludes_other_tenant) but, because nothing ever sets status='abandoned' in production, get_recovery_candidates has nothing to return there regardless of correctness.

get_session_stats (402 to 497): four func.count queries for total/in_progress/completed/abandoned, a top-UTM-sources GROUP BY, a 30-day sessions_by_day GROUP BY - all correct and tested (test_counts_completion_rate_and_top_sources). The fifth query, avg_completion_percentage (443 to 452), runs func.avg(func.jsonb_array_length(FormSession.partial_data)) filtered to status == 'abandoned' for the template - partial_data is a JSON object, not an array, so Postgres raises cannot get array length of a non-array (sqlalchemy.exc.DataError) the moment the template has even one abandoned session with any partial_data, and the whole stats call fails, not just that one field (test_abandoned_session_triggers_jsonb_array_length_bug).

Invariants

• A session is addressed by session_token alone; no route re-derives or re-checks the workspace a token's template belongs to on save/email/complete/get (measured by reading every route in API surface; there is no workspace/embed-code comparison in any of save_partial, capture_email, complete_session, get_form_session, save_partial_public, capture_email_public, get_public_session).
• complete_session never creates a second FormSubmission for the same session (test_idempotent_returns_existing_submission).
• A completed session's partial_data/current_page are frozen against save_partial (test_completed_session_is_not_mutated).
• email_captured is always lowercased and stripped before it is stored or sent to Divination (test_lowercases_strips_and_stores).
• NOT actually held, despite the code's intent: "partial field data merges into the session as the visitor types" - update_partial_data's in-place dict.update() on a non-MutableDict JSONB column drops every merge past the session's creation-time value (test_advances_page_and_documents_partial_data_merge_bug, same mechanism in test_lowercases_strips_and_stores).
• NOT actually held: "stale sessions are marked abandoned" - mark_abandoned is never invoked in this codebase and would TypeError on its first candidate row if it were (test_stale_session_triggers_get_completion_percentage_bug).

Frontend

By id, the module that renders this: forms-leads/ui/form-builder-page (the recipe author, authoring in parallel; not yet written at this commit). The real caller measured in this repo, which a rebuild should treat as the working reference for how the contract is actually consumed: alchemy_labs/frontend/src/components/forms/FormSessionManager.tsx (384 lines) via alchemy_labs/frontend/src/services/forms/formsApi.ts (536 lines), both calling the no-auth admin-router paths (/api/forms/sessions/...), never the embed-scoped /api/f/... ones.

FormSessionManager on mount (74 to 131): if enableResume and a session_token is in localStorage under form_session_<templateId>, calls formsApi.getSession() (→ GET /api/forms/sessions/{token}, the always-500 route); the surrounding try/catch (91 to 105) treats ANY failure identically to "session not found" - clears the stored token and starts a brand-new session. So a returning visitor's in-progress session is never actually resumed in the UI, regardless of what is or isn't on the server row. On field blur, savePartialData() (177 to 187) calls formsApi.savePartial() (→ the always-500 /save route) inside a try/catch whose comment reads, verbatim, // Don't show error to user - auto-save failures are silent (185) - so this specific, always-firing 500 has no user-visible or logged symptom in the running app. On an email-looking field's blur (217 to 226), formsApi.captureEmail() (→ the always-500 /email route) is called inside an empty catch (err) {} (224) - same silent swallow. completeSession() (→ /complete, which does not crash) is the one call in this component whose failure is surfaced to the user (onSubmitError, 258).

Net effect measured end to end: a visitor's UTM/attribution is captured correctly at session start (no bug on that path); a visitor who types into the form and leaves has their email_captured scalar written (the service call commits before the response-building crash) but their in-progress field values are not durably recoverable (double failure: the server-side merge silently drops them, and the client-side save request that would have surfaced a symptom is swallowed); nothing marks the session abandoned so it never appears as a recovery candidate; a visitor who fully submits gets a working, uncorrupted submission via /complete, independent of all of the above.

Minimum admin/debug surface actually present (no dedicated debug route beyond the buggy list/stats endpoints above): SELECT id, template_id, status, email_captured, partial_data, last_activity_at FROM form_sessions WHERE status = 'in_progress' AND last_activity_at < now() - interval '30 minutes' is the queue mark_abandoned would process if anything ever called it; SELECT id, session_token, email_captured, last_activity_at FROM form_sessions WHERE status = 'abandoned' AND email_captured IS NOT NULL AND recovery_email_sent = false is what get_recovery_candidates selects (guaranteed empty in production per the note above).

Security & tenancy

Auth dependency: none on session creation/save/email/complete on either router (surface_exempt-equivalent by simple omission of Depends(get_current_user); the public router is additionally mounted surface_exempt("public", "form embeds on client sites - no auth by design"), app_fastapi.py 949); get_current_user + WorkspaceID gate only the four admin routes in forms_router.py (list, stats, recovery-candidates, mark-recovery-sent). Workspace derivation on the admin routes: workspace_template_ids = [t.id for t in db.query(FormTemplate.id).filter(FormTemplate.workspace_id == str(workspace_id))], computed fresh per request (forms_router.py 1140 to 1144, 1248 to 1252) - a template not in that list is refused with 404, never leaked as "exists elsewhere".

Fail direction: the two 500s in Application logic (the TypeError and the DataError) are fail-open on data (nothing is written incorrectly, the write already committed before either crash) but fail-closed on the response (the caller gets a 500, not a wrong answer) - the practical harm is availability of the feature, not data integrity, measured across every crash site above. capture_email's email is taken as any string the caller sends (CaptureEmailRequest.email: str, no EmailStr validator, no format check at the schema layer) and is only .lower().strip()'d, never validated as an email shape, before being handed to divination.identify.

Secrets and PII at rest: partial_data is unencrypted JSONB and can legitimately hold anything a form field asks for (name, phone, free text); ip_address/user_agent are stored per session with no retention policy visible in this recipe's files. Rate limits: none on any session endpoint in either router (the bot-protection rate limiter in public_forms_router.py, RATE_LIMIT_MAX_SUBMISSIONS/RATE_LIMIT_WINDOW_SECONDS, 55 to 56, guards /submit and /submit-ai only, not /session/start//session/{token}/save, so session creation itself can be hammered without limit). Attacker-controlled inputs: session_token (a bearer credential in all but name - anyone who has it can read, save into, or complete that session; it is a 64-char secrets.token_urlsafe(48) value, cryptographically unguessable, but nothing revokes or expires it); every TrackingData field (utm_*, fbclid, gclid, fbp, fbc, li_fat_id, ga_client_id, page_url) is caller-supplied and stored verbatim, never used in a query or rendered as markup in this recipe's own files, so no injection surface here specifically; partial_data is caller-supplied Dict[str, Any] with no shape validation against the template's actual fields (a save can write arbitrary keys unrelated to any real form field).

Tests

Run 2026-09-12 against the deap-postgres-test container (docker-compose.test.yml, port 5434) from alchemy_labs/backend, using .venv:

python -m pytest tests/v2/coverage/test_forms_g7_formsessionservice_lifecycle_create_save_capture_complete_abandon_stats_.py \
    -q --no-cov -o addopts="" -p no:cacheprovider

prints 25 passed (0 failed; 313 warnings, all PydanticDeprecatedSince20/DeprecationWarning noise unrelated to this recipe). grep -c "^ def test_" ... prints 25; grep -c "@pytest.mark.parametrize" ... prints 0 (no parametrized cases, 25 collected tests map to 25 defs exactly).

Hermetic vs live: real Postgres via the transactional db_session fixture (real FormTemplate/FormSession/FormSubmission/FormAnalytics rows), divination mocked at the module boundary with AsyncMock (mock_divination fixture, unittest.mock.patch("services.forms.session_service.divination")) so no network call happens; no PSP, no HMAC seam, no external HTTP in this suite at all. Schema comes from Base.metadata.create_all() (S9, _register_full_model_graph imports app_fastapi first to register the whole model graph) - not from alembic upgrade head - which is exactly why this suite cannot see the created_at/updated_at migration gap or the index-set drift under Data model: create_all() builds straight from the model's own column and index declarations.

Known-answer/documented-bug vectors, each a test whose name states the bug and whose body characterizes (not hides) the current behaviour: test_advances_page_and_documents_partial_data_merge_bug, test_stale_session_triggers_get_completion_percentage_bug (asserts pytest.raises(TypeError) and that "get_completion_percentage" is in the message), test_abandoned_session_triggers_jsonb_array_length_bug (asserts pytest.raises(sqlalchemy.exc.DataError)). Workspace-scoping tests: test_workspace_scoping_excludes_other_tenant (recovery candidates), test_workspace_scoping_when_no_template_filter (list_sessions), both build two real Workspace/WorkspaceUser/FormTemplate fixtures and assert cross-tenant exclusion.

What this suite does not cover, stated plainly: the HTTP layer. Every test above drives FormSessionService methods directly with asyncio.run; none of them go through forms_router.py or public_forms_router.py, so the suite is green while five of the nine routes it backs 500 on every real call (API surface) - the router-level bug is invisible from this file alone and was found by reading the router source, not by a failing test. No dedicated Playwright/E2E spec exists for the session lifecycle at this commit (grep -rl "sessions/start\|session/start" tests/ outside this file finds nothing under tests/e2e or similar).

Deploy & ops

Config keys per app: DIVINATION_URL, DIVINATION_API_KEY, DIVINATION_SERVICE_NAME (all above; none specific to form sessions beyond the Divination client itself). Migration: add_form_sessions (migrations/versions/20251205_add_form_sessions.py), down_revision = 'add_meeting_phase_id', chained forward by 20251205_add_multipage_fields.py (down_revision = 'add_form_sessions'); alembic heads at this commit prints one head, prima_device_codes_001, confirming this revision sits on the single live chain, not a stray branch. The loose add_form_sessions_timestamps.sql patch is not part of that chain (Data model) and is not run by scripts/migrate.sh, which executes only alembic upgrade head under a Postgres advisory lock (MIGRATION_LOCK_ID = 8110731) before every deploy (Fly.io staging, Render, the Infomaniak k8s job, docker-compose's one-shot migrate service, per that script's own header comment).

Scheduler: none. mark_abandoned's own docstring says "Should be called by a background job" (session_service.py 296), but no job exists in services/scheduled_jobs.py or anywhere else in this repo that calls it (Application logic) - a rebuild that wants the abandonment half of this feature to work must both fix the TypeError and register a job on _shared/scheduler-and-background-work's APScheduler singleton; neither exists yet.

Argus-style checks: POST /api/forms/sessions/start with a valid template_id returns 201 with a session_token, proves the seam is reachable; SELECT count(*) FROM form_sessions WHERE status = 'abandoned' is expected to be exactly 0 in steady state at this commit, because nothing ever sets that value (a nonzero count would mean either a manual UPDATE or that some other, unmeasured code path calls mark_abandoned outside this recipe's own files). Rollback: no destructive migration in the chain; a form_sessions row is never deleted by this code.

Gotchas

• 2026-09-12 (measured at 62ba4fc, models/forms.py 696 vs eight call sites: session_service.py 329, 590, 651 and forms_router.py 992, 1039, 1087, 1182, 1268): FormSession.get_completion_percentage requires a total_fields argument; all eight call it with none. Six propagate as an uncaught TypeError that reaches the HTTP client: the five forms_router.py routes (none of those lines sits inside a try/except) and mark_abandoned's own call (session_service.py 329, outside that method's own guards). The other two - inside _identify_in_divination (590) and _track_form_abandoned (651) - are each caught by that method's own try/except Exception: logger.warning(...), so capture_email's and mark_abandoned's Divination side-effects silently fail there too, without crashing the caller. Only public_forms_router.py's get_public_session (line 1487) calls it correctly, with total_fields = len(template.fields).
• 2026-09-12 (measured, models/forms.py 680 to 684, a bare dict.update()): FormSession.partial_data is a JSONB column, not wrapped in SQLAlchemy's MutableDict, so any in-place mutation (update_partial_data, called from both save_partial and capture_email) is invisible to the ORM's dirty-tracking and is silently dropped on commit(). Only a full column reassignment (session.partial_data = {...}) would persist; nothing in this recipe does that.
• 2026-09-12 (measured, grep -rn mark_abandoned across the repo): mark_abandoned has no caller anywhere outside models/forms.py, session_service.py itself, and its own tests. Abandonment detection does not run in this codebase, independent of the TypeError bug above.
• 2026-09-12 (measured, session_service.py 443 to 452): get_session_stats's avg_completion_percentage subquery calls jsonb_array_length on a JSON object column; it raises DataError for any template with an abandoned session that has any partial_data, taking down the whole stats response, not just that one number.
• 2026-09-12 (measured, FormSessionManager.tsx 185, 224): the real frontend caller swallows both of the always-crashing save and email-capture requests silently (an empty catch and a comment stating the swallow is deliberate for auto-save), so the router-level bug above produces no visible symptom in the running app; it was found by reading the router, not by an error report.
• 2026-09-12 (measured, add_form_sessions_timestamps.sql vs scripts/migrate.sh): created_at/updated_at are declared on the ORM model but not created by the chained migration; the loose SQL file that adds them is not wired into the deploy pipeline. Whether it has been applied out-of-band to any live database is not determinable from this repository (Data model).
• 2026-09-12 (measured, public_forms_router.py 1452 vs forms_router.py 1046, schemas/forms.py 353 to 355): the "same" capture-email endpoint takes email as a query parameter on the public router and as a JSON body field (CaptureEmailRequest) on the admin router; a client written against one will not work against the other unchanged.

Reference implementation

Paths relative to the repo root; all at main 62ba4fc41009b454e1c033e5036c7e8577db2061 unless noted.

• alchemy_labs/backend/models/forms.py (958 lines): FormSubmission (443 to 519); FormAnalytics (522 to 558, event_type CHECK 542 to 548); FormSession (561 to 701: table 566 to 630, to_dict 635 to 664, get_attribution_data 666 to 678, update_partial_data 680 to 684, mark_completed 686 to 690, mark_abandoned 692 to 694, get_completion_percentage 696 to 701).
• alchemy_labs/backend/schemas/forms.py (709 lines): TrackingData (298 to 326); StartSessionRequest/Response (329 to 344); SavePartialRequest (347 to 350); CaptureEmailRequest (353 to 355); CompleteSessionRequest (358 to 361); FormSessionResponse (364 to 395); SessionListResponse (398 to 414); SessionStatsResponse (417 to 427).
• alchemy_labs/backend/services/forms/session_service.py (680 lines, whole file in scope): FormSessionService and every method named under Application logic; the Divination integration block, 543 to 679.
• alchemy_labs/backend/api/routers/forms_router.py (1571 lines): the "Session Endpoints" and "Admin Session Endpoints" blocks, 899 to 1313 (router mounted prefix="/forms", app_fastapi.py line 928, /api prefix).
• alchemy_labs/backend/api/routers/public_forms_router.py (1815 lines): bot-protection helpers 50 to 217; the four session routes, 1393 to 1511 (router mounted prefix="/f", app_fastapi.py line 949); the inline session.mark_completed() calls inside the real submit paths, lines 708 and 1158.
• alchemy_labs/backend/migrations/versions/20251205_add_form_sessions.py (135 lines, whole file); alchemy_labs/backend/migrations/add_form_sessions_timestamps.sql (6 lines, not chained, see Gotchas).
• alchemy_labs/backend/tests/v2/coverage/test_forms_g7_formsessionservice_lifecycle_create_save_capture_complete_abandon_stats_.py (582 lines, whole file in scope) and tests/conftest.py for the db_session/test_engine fixtures this recipe's Tests section depends on.
• alchemy_labs/frontend/src/components/forms/FormSessionManager.tsx (384 lines) and alchemy_labs/frontend/src/services/forms/formsApi.ts (536 lines): the real, measured caller (Frontend).

Deviations of the reference from an ideal implementation, all dated 2026-09-12: the get_completion_percentage signature mismatch across eight call sites; the non-MutableDict JSONB merge that silently drops every partial-data write; the unwired mark_abandoned; the jsonb_array_length misuse over a JSON object; the two capture-email contracts (query param vs body) that disagree between routers; the untracked created_at/updated_at migration patch; no re-verification of workspace/embed ownership on a bare session_token past session start.

Acceptance checks

Run from alchemy_labs/backend at 62ba4fc with the deap-postgres-test container up on port 5434 (or any Postgres reachable as TEST_DATABASE_URL). Expected observations are exact, as measured 2026-09-12.

1. python -m pytest tests/v2/coverage/test_forms_g7_formsessionservice_lifecycle_create_save_capture_complete_abandon_stats_.py -q --no-cov -o addopts="" -p no:cacheprovider prints 25 passed.
2. grep -c "^ def test_" tests/v2/coverage/test_forms_g7_formsessionservice_lifecycle_create_save_capture_complete_abandon_stats_.py prints 25; grep -c "@pytest.mark.parametrize" ... prints 0.
3. grep -n "^ def get_completion_percentage" models/forms.py shows total_fields in the signature on line 696; grep -n "get_completion_percentage()" services/forms/session_service.py api/routers/forms_router.py (the empty parens select the zero-argument form) prints exactly the eight lines named in Gotchas: three in session_service.py (329, 590, 651), five in forms_router.py (992, 1039, 1087, 1182, 1268). The same grep against api/routers/public_forms_router.py prints nothing, because its one call (line 1487) carries total_fields.
4. grep -n "self.partial_data.update" models/forms.py shows line 684 inside update_partial_data, and grep -rn "MutableDict" models/forms.py returns no match, confirming the column is plain JSONB.
5. grep -rln mark_abandoned . from alchemy_labs/backend (excluding .venv) lists only models/forms.py, services/forms/session_service.py, and the G7 test file - no scheduler, no router.
6. git merge-base --is-ancestor 62ba4fc41009b454e1c033e5036c7e8577db2061 main; echo $? (run from a clone with main available) prints 0, confirming this commit is on main (at the time measured, it was main's own HEAD).
7. python -m alembic heads from backend/ prints exactly one head (prima_device_codes_001 at measured_on), confirming the add_form_sessions revision sits on the single live chain.