Email Bot Documentation

Overview

The Repo Bot is an email-first interface to the YieldExchange platform. A trader sends a plain-English email like“Borrow 500M CAD at 4.85% from June 15 to June 22, Govt Bonds”and the bot interprets it, asks for anything missing, summarises the quote for confirmation, and on CONFIRM submits the RFQ to V2 — exactly as if the user had filled the web form themselves.

It exists because:

• Traders live in Outlook / Gmail. Forcing a web UI for fast-moving repo flow loses adoption.
• Email already carries enough structure for an LLM to extract every field.
• The bot reuses the V2 schema 1:1 — no parallel data model, no drift.

Two flows, one inbox

Architecture

The bot is a Python (FastAPI) service that runs alongside the V2 stack in the same Docker Compose network. There are no extra databases for application data — V2 owns all RFQs, offers, orgs and users. The bot only stores conversational state.

┌──────────────┐        IMAP IDLE         ┌──────────────────────────┐
│  Trader's    │  ───────────────────────▶│  Repo Bot (FastAPI)      │
│  inbox       │                          │  Python 3.12             │
│  (Gmail/IMAP)│  ◀───────────────────────│                          │
└──────────────┘        SMTP reply        │  • IMAP poller (IDLE)    │
                                          │  • PII tokenizer (5-layer│
                                          │  • LLM extractor (OpenAI │
                                          │    gpt-4o-mini, JSON)    │
                                          │  • State machine (FSM)   │
                                          │  • V2 client (httpx, JWT)│
                                          │  • Bot admin dash (Next) │
                                          │  • SQLite/MySQL (state)  │
                                          └──────────┬───────────────┘
                                                     │ HTTPS + JWT
                                                     │ (lookup, recipients,
                                                     │  RFQ create, offers)
                                                     ▼
┌──────────────┐        Session JWT     ┌────────────────────────────────┐
│ V2 dashboard │  ─────────────────────▶│  V2 API Gateway (Express)      │
│  (Next.js)   │  (Org admin uses       ├────────────────────────────────┤
│  Org admin   │   Integrations tab to  │  • reference-data-service      │
│  controls    │   toggle botAccess +   │     ↳ /organizations/lookup    │
│  Integrations│   bot-members +        │     ↳ /organizations/:id/bot-* │
│  tab here    │   allowed-domains)     │     ↳ permission cache (Redis) │
└──────────────┘                        │  • negotiation-service         │
                                        │  • notification-service        │
                                        │  • audit-service               │
                                        └──────────────┬─────────────────┘
                                                       │
                                       ┌───────────────┼───────────────┐
                                       ▼               ▼               ▼
                                  ┌─────────┐    ┌──────────┐    ┌─────────┐
                                  │  MySQL  │    │  Redis   │    │ audit   │
                                  │  (V2 DB)│    │ (sess +  │    │  log    │
                                  │         │    │  notif.) │    │  (DB)   │
                                  └─────────┘    └──────────┘    └─────────┘

Repos & container names

Quick start

Bring the whole stack up locally and send your first email to the bot.

1. Start everything

# V2 stack (api-gateway, services, MySQL, Redis)
cd V2-APP-DASHBOARD
docker compose up -d

# Bot + dashboard
cd ../ye-repo-bot
docker compose up -d

# Verify
docker ps --format 'table {{.Names}}\t{{.Status}}' \
  | grep -E 'ye-repo-bot|api-gateway|reference-data|negotiation'

2. Sign in to the admin dashboard

Open http://localhost:8000, sign in with the admin credentials from the bot’s .env (ADMIN_EMAIL / ADMIN_PASSWORD), then enter the OTP that arrives at ADMIN_OTP_EMAIL.

3. Send a test email

From a registered V2 user (e.g. [email protected]), email the bot inbox (defaults to [email protected]) with:

Subject: New repo posting

Borrow 500M CAD at 4.85% from 2026-06-15 to 2026-06-22, Govt Bonds.

You should receive a confirmation summary within ~30 seconds. Reply CONFIRM and the posting goes live in V2 — counterparties on bot-enabled orgs receive a notification email.

Org onboarding

How a firm goes from “not on the email bot” to “trader sends an email and gets a confirmation in 30 seconds” — and exactly what V2 and the bot do under the hood.

Two-layer access control

Every inbound email must pass both gates before the bot will engage:

The admin journey in V2 (three clicks)

1. 1

   Open Organization → Integrations

   An ORG_ADMIN navigates to Settings → Organization → Integrations. Non-admins see the same page in read-only mode.
2. 2

   Flip the master switch

   Toggle Email Bot Integration on. This calls PATCH /api/organizations/:id/bot-access, requires permission ORG_SETTINGS_MANAGE, and writes an audit log entry (BOT_ACCESS_TOGGLE).
3. 3

   Enable specific members

   The Members table appears with a toggle next to every active member. Flip the ones who should be able to email the bot. Each toggle calls PATCH /api/organizations/:id/bot-members/:userOrgId and writes BOT_MEMBER_TOGGLE to the audit log. Enable all / Disable all buttons batch-update via POST /api/organizations/:id/bot-members/bulk.
4. 4

   (Optional) Whitelist extra sender domains

   If traders sometimes email from a sibling domain (e.g. td-securities.com when they're registered under td.com), add it to the allowed-domains list. Saved via PATCH /api/organizations/:id/allowed-domains. Up to 10 domains; format-validated server-side.

What gets persisted

-- Migration: 20260514000000-add-bot-per-user-toggle-and-domains
ALTER TABLE user_organizations
  ADD COLUMN botEnabled BOOLEAN NOT NULL DEFAULT false;

ALTER TABLE organizations
  ADD COLUMN allowedDomains JSON NULL DEFAULT NULL;
-- shape: ["td.com", "td-securities.com"]   (max 10 entries)

Defaults are deliberately opt-in: when orgbotAccess is flipped on, no member is automatically enabled. The admin must explicitly grant each one (or clickEnable all). New invitees default to disabled. This is the compliance-safe default — no surprise activations.

Permission model

All four mutation endpoints are guarded by V2’s shared requirePermission(Permissions.ORG_SETTINGS_MANAGE) middleware. The permission cache lives in Redis (ye:perms:{userOrgId}) and is populated at login. By default ORG_SETTINGS_MANAGE is granted toORG_ADMIN only — Senior Traders, Traders, and Read-Only users can see the page but can’t make changes.

// services-v2/reference-data-service/src/controllers/bot-config.controller.ts
botConfigRouter.patch(
  '/:orgId/bot-members/:userOrgId',
  requirePermission(Permissions.ORG_SETTINGS_MANAGE),
  async (req, res) => { /* ... */ },
);

Email → org → user, with the new gates

Every inbound email walks the same five-step chain. New steps in bold:

GET /api/organizations/[email protected]

1. users.email = '[email protected]'                    →  404 if not found
2. user_organizations: active membership          →  404 if not found
3. organizations: row exists                      →  404 if not found
4. org.botAccess = true                           →  403 "org_not_enabled"
5. userOrganization.botEnabled = true   ← NEW    →  403 "user_not_enabled"
                                                   200 OK → bot engages

The 403 sub-code in the response body tells the bot which gate rejected the sender, so it can pick the appropriate reply template.

Allowed domains (what they do today, what they will do)

The allowed-domains list is saved per org but isn’t consulted by the bot lookup yet. The current behaviour is “exact email match against the users table.” The plumbing is in place so a future v1.1 can fall back to domain-matching for guest senders, with a per-org policy decision. We didn’t ship that in v1 because attributing emails to an org without a specific user opens audit/compliance questions worth a separate review.

Audit trail

Every action emits an audit-service entry via the shared emitAuditLog helper. Compliance can reconstruct the full history of who enabled whom, when, from what IP.

V2 Integrations UI

Every control on the V2 dashboard’s Settings → Organization → Integrations tab — what it does, what backend call it makes, what gets persisted, and what audit entry it writes.

Where to find it

On the V2 dashboard, navigate to the user profile menu → Settings → Organization. The page has a tab strip; pick Integrations. Anyone in the org can open the page; only members with ORG_SETTINGS_MANAGE can change anything (typically ORG_ADMIN).

Anatomy of the page

┌──────────────────────────────────────────────────────────────────────┐
│  ✉  Email Bot Integration                                  [ ⬤ ]    │  ← (1)
│     Master switch for your organisation. Members must also          │
│     be individually enabled below.                                  │
│     [Active — Members listed below who are enabled can email…]      │
├──────────────────────────────────────────────────────────────────────┤
│  Members allowed to use the bot         [Enable all] [Disable all]  │  ← (2)
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │ NAME           EMAIL                ROLE          BOT ACCESS  │ │
│  │ Ravi Sumal     [email protected]         ORG_ADMIN     On  [⬤]      │ │
│  │ Ernest Kiromo  [email protected]       TRADER        Off [○]      │ │
│  │ Sarah Chen     [email protected]   SENIOR_TRADER On  [⬤]      │ │
│  └────────────────────────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────────────────┤
│  Additional allowed sender domains                                  │  ← (3)
│  ⌗ td.com  ⌗ td-securities.com  ⌗ tdam.com                          │
│  [Pick from detected ▾]  [td-asset-mgmt.com........  ] [+ Add]      │
│  3/10 domains                                  [Cancel] [Save]      │
└──────────────────────────────────────────────────────────────────────┘

(1) Master switch — Email Bot Integration

(2) Members table — per-member bot access

What each control on the Members table does

(3) Additional allowed sender domains

Read-only mode (non-admins)

Members who lack ORG_SETTINGS_MANAGE still see the whole page — useful for traders sanity-checking whether their own row is enabled — but every control is disabled and an info banner explains why:

┌──────────────────────────────────────────────────────────────────────┐
│ ⓘ  Read-only view                                                    │
│    You can see the current configuration but only an Org Admin       │
│    can change it.                                                    │
└──────────────────────────────────────────────────────────────────────┘

• Master switch: visible, disabled, shows current state.
• Member rows: each toggle shows the current state but won’t respond to clicks.
• Domain chips: remove-X icons are hidden; the input + Add button are absent.
• If a non-admin still tries via the API directly, the server returns 403 Insufficient permissions from requirePermission.

What happens behind a single click

Sequence diagram for “Org admin flips Ernest’s row from Off to On”:

Admin clicks ⬤  in the row for [email protected]
   │
   ▼
[React] IntegrationsTab.handleMemberToggle(member)
   • setMemberState({ ...prev, [member.userOrgId]: true })   ← optimistic
   • updateBotMember.mutate({ orgId, userOrgId, enabled:true })
   │
   ▼
[RTK Query] PATCH /api/organizations/:orgId/bot-members/:userOrgId
   • Authorization: Bearer <jwt>
   • body: { enabled: true }
   │
   ▼
[api-gateway → reference-data-service]
   • authMiddleware decodes JWT → req.user.userOrgId
   • requirePermission(ORG_SETTINGS_MANAGE)
        ├─ reads ye:perms:{userOrgId} from Redis
        ├─ if missing → 401 PERMISSION_CACHE_EXPIRED → client refreshes
        └─ if permission missing → 403 Insufficient permissions
   • controller:
        ├─ UserOrganization.findOne({ id: userOrgId, orgId })  → 404 if not found
        ├─ membership.update({ botEnabled: true })
        └─ emitAuditLog({ entityType:'USER_ORGANIZATION',
                          action:'UPDATE',
                          details:{ subType:'BOT_MEMBER_TOGGLE',
                                    from:false, to:true,
                                    userId: membership.userId } })
   • returns 200 { userOrgId, botEnabled:true, unchanged:false }
   │
   ▼
[RTK Query] invalidatesTags: [{ type:'BotConfig', id:orgId }]
   • the GET /bot-config query is auto-refetched in the background
   • server returns the fresh members list — UI reconciles
   │
   ▼
[Toast]  "Enabled the email bot for Ernest Kiromo"
   │
   ▼
Next email from [email protected] lookup → bot proceeds.
On failure anywhere above, the optimistic local state is rolled back
and a "Failed to update Ernest — please try again" toast appears.

User journeys

End-to-end stories — what an org admin / trader / auditor / compliance officer actually does, what they see in the UI, what the bot replies, and what gets recorded. Six representative scenarios covering setup, day-2 ops, and incident response.

1 · First-time firm onboarding

Persona: Ravi, Org Admin at TD. Goal: get TD’s desk on the email bot.

1. 1

   Sign in to V2

   Ravi logs into app.yieldexchange.ca. The session-context cache (ye:perms:{userOrgId}) is populated in Redis on completion of the OTP step.
2. 2

   Open Integrations tab

   Settings → Organization → Integrations. Sees the master switch (off), an empty Members table (no rows enabled yet), and no allowed domains.
3. 3

   Flip the master

   Toggles Email Bot Integration on. Toast: “Email bot enabled for TD Bank”. Members table un-greys.
4. 4

   Enable the desk

   Decides which traders should be in the v1 pilot. Toggles three rows: himself, Ernest, Sarah. Each toggle fires a separate audit entry.
5. 5

   (Optional) Add a sibling domain

   Sarah’s registered email is [email protected] but she emails from [email protected]. Ravi adds td-securities.com to allowed domains (saved, not consumed by lookup until v1.1).
6. 6

   Trader sends test email

   Sarah forwards a quote to [email protected]. Within ~30s she gets a confirmation summary from the bot. After replying CONFIRM the RFQ is live on V2; counterparties on other bot-enabled orgs (with at least one opted-in member) receive notifications.

2 · A new trader joins TD

Persona: Maya, recently promoted to a junior repo trader. Pre-condition: TD already has the email bot enabled and Ravi as Org Admin.

1. 1

   Ravi invites Maya in Members tab

   Standard V2 invite flow. Maya appears in the org with role TRADER and botEnabled=false by default.
2. 2

   Maya signs in for the first time

   She completes OTP, sees the trading dashboard. Nothing in V2 hints at her bot-access state — that’s by design (the toggle is the admin’s call, not a self-service feature for v1).
3. 3

   Maya emails the bot too early

   She tries the bot from her work address. The bot calls lookup; V2 returns 403 with code: user_not_enabled. The bot replies with the Personal Bot Access Required template — points her at her Org Admin, names her firm, and explains the exact toggle her admin needs to flip.
4. 4

   Maya pings Ravi

   The reply email is the entire script — Ravi knows exactly what to do without a back-and-forth.
5. 5

   Ravi enables her

   Opens Integrations, flips Maya’s row to On. Audit entry recorded with subType BOT_MEMBER_TOGGLE, from=false, to=true.
6. 6

   Maya retries

   She resends the email. Lookup passes both gates this time → bot extracts fields → confirms → submits. Total elapsed: minutes, no support ticket.

3 · A trader leaves the firm

Persona: Ernest leaves TD on Friday. Goal: make sure no email he sends Monday morning can reach the V2 platform.

1. 1

   HR offboarding pings the platform admin

   Ravi gets the same ticket he'd get for any V2 access removal. Two separate actions: revoke V2 access AND revoke email-bot access.
2. 2

   Flip bot toggle off first

   Integrations → Ernest’s row → toggle Off. This takes effect immediately for the next inbound email — even before Ernest’s V2 account is fully closed.
3. 3

   Then close the V2 account (Members tab)

   Standard V2 deactivation. Membership status flips to SUSPENDED or the user is removed.
4. 4

   Verify on Monday

   If Ernest tries to email from his still-active corporate inbox, lookup returns 404 (user removed) and the bot replies “Account Not Found”. Audit log shows the exact off-toggle that preceded the deactivation.

4 · “The bot stopped replying!” — a trader debugs herself

Persona: Sarah, senior trader, used the bot yesterday and today it’s silent.

1. 1

   Sarah sends another posting

   No silence — within seconds she gets an automated reply from the bot.
2. 2

   Read the reply, not the headline

   The bot’s reply is one of three diagnostic templates depending on the failure mode:

   • Account Not Found — her email isn’t registered (typo / wrong inbox).
   • Email Bot Access Not Enabled — TD turned the master off (probably an admin emergency).
   • Personal Bot Access Required — TD is still on but her specific toggle was flipped off (probably a leaver-script ran wrong).
3. 3

   Forward the reply to Ravi

   The reply is self-contained — Ravi sees which gate failed without opening V2 or the audit log.
4. 4

   Ravi fixes

   Either flips her toggle back on (per-member case) or flips the master back on (org case). The fix is one click and there is an audit row for both the original off and the restore.

5 · Compliance kill switch

Persona: a Compliance Officer needs to disable inbound trading for the whole firm now (e.g. a regulatory incident, suspected account compromise).

1. 1

   Open Integrations as Org Admin

   Compliance contacts the Org Admin (or has the role themselves). Settings → Organization → Integrations.
2. 2

   Flip the master switch off

   One toggle. Takes effect on the next inbound email — typically within seconds. No member rows change, so re-enabling later restores the exact same per-member configuration.
3. 3

   Audit row is the legal record

   entityType=ORGANIZATION, action=UPDATE, subType=BOT_ACCESS_TOGGLE, with timestamp, actor userOrgId, IP, user-agent, and the before/after values. Joinable with the regulatory case ID via the audit-service.
4. 4

   Bot replies during the freeze

   Any email sent during the freeze gets the Email Bot Access Not Enabled reply — clear, non-alarming, pointing at the admin. No deals slip through.
5. 5

   Restore

   When cleared, the admin flips the master back on. All previously-enabled members are still individually enabled. Two audit entries total: off, on.

6 · Auditor reconstructs a year of access changes

Persona: a compliance auditor preparing for SOC 2 / regulatory review.

1. 1

   Query the audit-service

   Filters on entityType IN (ORGANIZATION, USER_ORGANIZATION) AND details.subType LIKE ‘BOT_%’. Returns every relevant change.
2. 2

   Reconstruct master-switch history per org

   From BOT_ACCESS_TOGGLE entries: orgId, timestamp, actor, before, after. Builds a per-org timeline of when the firm had bot access on.
3. 3

   Reconstruct per-member history

   From BOT_MEMBER_TOGGLE entries: which user was enabled when, by whom. BOT_MEMBER_BULK_TOGGLE entries give the count + acting principal but not per-row breakdown (intentional — bulk is an admin convenience, not a fine-grained event).
4. 4

   Cross-check with email events

   For any inbound email the bot rejected, the bot’s logs show the exact 403 sub-code (org_not_enabled / user_not_enabled) and the lookup timestamp — matchable to the audit timeline.
5. 5

   Output report

   Every access grant and revoke for the period, with actor + IP + before/after, exportable as CSV/JSON via the audit-service's standard endpoints.

Side A — Posting flow

What happens when a trader emails a new repo posting.

1. 1

   Inbound email arrives

   The IMAP poller (idling on the bot inbox) wakes up, fetches the message, deduplicates against the processed_messages table.
2. 2

   Sender is resolved

   The bot calls GET /api/organizations/lookup?email=… on V2. The endpoint checks two gates: (1) org botAccess=true and (2) user botEnabled=true. If either fails, the bot sends the appropriate reply (org-level vs user-level) and closes the thread. See Org onboarding for the full chain.
3. 3

   PII is tokenised

   The body is run through the 5-layer tokeniser — names, emails, phones, and any unknown ORG are replaced with placeholders before the text reaches the LLM. See PII tokenisation.
4. 4

   Intent + fields extracted

   An OpenAI call (gpt-4o-mini, JSON schema response) classifies the intent (POSTING, OFFER, BROWSE, CONFIRM, CANCEL) and extracts every field present.
5. 5

   Validate locally

   The bot’s validator (bot/v2/validator.py) catches problems V2 would otherwise reject — bad currency, past start date, amount < $100k, missing collateral — and turns them into a clear reply.
6. 6

   Collect anything missing

   If any required field is absent the bot replies with an adaptive ask: full helper card on the first turn, concise “still need rate, dates” on follow-ups. The LLM message and the static template share a single opener (no duplicate greetings).
7. 7

   Summarise &amp; confirm

   Once everything is present, the bot sends a styled summary email. The thread state is PENDING_CONFIRMATION until the user replies CONFIRM or CANCEL.
8. 8

   Submit to V2

   On CONFIRM, the bot maps fields to the V2 shape (collateral normalised, direction translated, dates serialised), calls POST /api/requests, and stores the returned request_id on the thread.
9. 9

   Notify counterparties

   The bot calls GET /api/organizations/bot-recipients?excludeOrgId=… which returns active, individually-opted-in users in other orgs whose botAccess is on. Each gets a counterparty-alert email. Users in bot-enabled orgs who haven’t themselves been toggled on are not notified.
10. 10

    Thread closes

    The thread transitions to COMPLETE. Further replies on the same thread won’t accidentally resubmit.

Side B — Offer flow

When a counterparty wants to respond to an active RFQ, they reply to the bot’s counterparty-alert email (or start a fresh thread quoting the request ID).

1. 1

   Identify the RFQ

   The bot resolves which RFQ the reply refers to from the email’s In-Reply-To header, then fetches the latest RFQ details via GET /api/requests/:id.
2. 2

   Extract offer terms

   The LLM extractor uses a different schema for offers — rate, max_amount, proposed_collateral, proposed_settlement_date, notes.
3. 3

   Collect missing fields

   If only rate is present, the bot asks for the optional fields with hints (max amount, settlement date, collateral, free-text notes).
4. 4

   Summarise the quote

   The offer-summary email shows both sides — the RFQ’s indicative rate and the counterparty’s quoted rate.
5. 5

   Submit the offer

   On CONFIRM, the bot calls POST /api/requests/:id/offers, stores the offer_id, and replies with a submission confirmation including a dashboard link.

Bulk uploads (text + CSV)

A user can post several repos in one email — either by listing them in the body or by attaching a CSV. Both paths converge on the same internal pipeline.

Text bulk

Please post these 3 repos:

1) Borrow 500M CAD at 4.85% from 2026-06-15 to 2026-06-22, Govt Bonds
2) Lend 250M USD at 5.10% from 2026-06-15 to 2026-06-22, T-Bills
3) Borrow 100M EUR at 3.95% from 2026-06-15 to 2026-06-22, Corp Bonds

extract_bulk_postings() runs against the tokenised body (LLM with structured JSON schema), returning one object per posting. A regex backfill then runs over each row’s source slice to recover any collateral the LLM may have dropped — this is a safety net for known multi-row LLM flakiness.

CSV attachment

direction,amount,currency,rate,start_date,end_date,collateral_type
BORROW,500000000,CAD,4.85,2026-07-01,2026-07-08,Govt Bonds
LEND,250000000,USD,5.10,2026-07-01,2026-07-08,T-Bills
BORROW,1B,EUR,3.95,2026-07-01,2026-07-08,Corporate Bonds

The CSV parser is forgiving: 1B / 500M expand, collateral synonyms normalise (Govt Bonds →Government Bonds), date formats accept YYYY-MM-DD, DD/MM/YYYY, etc. Up to one CSV attachment is processed per email; additional attachments are logged and ignored.

Thread state machine

Every email thread has exactly one state. The state controls which inbound replies are valid and which template responds. States are stored on the email_threads table.

PII tokenisation

The bot is never allowed to send raw personal data to OpenAI. A deterministic 5-layer pipeline replaces every PII span with an opaque token before any LLM call, then re-hydrates the token in outbound text.

Token format

[TOKEN:TYPE:NNN]

TYPE    NAME | EMAIL | PHONE | ORG | REF
NNN     3-digit per-thread counter

The full real → token map is persisted as JSON in email_threads.pii_registry so the same real value always maps to the same token across messages in a thread, and survives process restarts.

Domain vocabulary

One source of truth — _FIN_VOCAB_PATTERNS in bot/core/pii_tokenizer.py — protects domain terms from tokenisation. The same terms are normalised to V2’s enum by _normalise_collateral() in bot/v2/v2_client.py.

Other protected terms

LLM extraction

All LLM calls go to OpenAI gpt-4o-mini with response_format={ type: "json_schema" } and a strict per-task schema. The model never returns free text we can’t parse.

Anti-hallucination guards

• Dates: if the user’s text contains no date-like cue (regex sweep), all date fields are forced to null regardless of LLM output.
• Bulk collateral: after the LLM returns, each row’s source slice is regex-scanned for collateral keywords and used to backfill anything missed.
• Mandatory opener: the bot-message prompt assigns a random word from a 20-word pool to start the reply, so two consecutive emails never sound identical.
• Issues summary: when there are validation issues, the LLM is told explicitly — “NEVER claim work was done that wasn’t” — so it can’t say “correction applied” when no correction occurred.

V2 API integration

The bot acts as a special V2 user (a service account configured via V2_BOT_EMAIL / V2_BOT_PASSWORD). All calls flow through the _v2_request() helper which provides:

• Automatic JWT acquisition + caching with refresh on 401
• Exponential-backoff retry on 5xx (502/503/504) and transport errors
• Per-request timeout (default 15s, 30s for token fetch since V2’s bcrypt is slow)
• X-Org-Id header set so V2 attributes the action to the correct org
• X-Act-As-User / X-Act-As-Org headers on create/offer calls so V2 attributes the row to the real user’s org, not the bot’s service-account org — see V2 bot impersonation

Endpoints used

The Caller column shows who initiates each request. Bot-callers go via the bot’s service-account JWT; admin-UI-callers go via the user’s session JWT and pass permission checks (requirePermission).

Lookup response codes

The bot branches its reply on the code field that the lookup endpoint adds to non-200 responses. Templates are referenced by name; see bot/email/template.py.

Field mapping (bot → V2)

{
  "direction":  "BORROW"            →  COLLATERAL_PROVIDER     // borrowing cash means providing collateral
                "LEND"              →  CASH_PROVIDER
  "amount":     500000000           →  amount: 500000000
  "currency":   "CAD"               →  currency: "CAD"
  "rate":       4.85                →  repoTerms.fixedRate: 4.85
  "start_date": "2026-06-15"        →  startDate: "2026-06-15"
  "end_date":   "2026-06-22"        →  endDate:   "2026-06-22"
                                       termType: "TERM" (derived from end_date - start_date)
  "collateral_type": "Corp Bonds"   →  repoTerms.preferredCollateralTypes: ["Corporate Bonds"]
}

V2 bot impersonation

The bot logs into V2 as a single service account (configured via V2_BOT_EMAIL, e.g. [email protected]). Without help, V2’s authMiddleware reads orgId straight from that account’s JWT, so every RFQ the bot creates lands under the bot’s own org (in our dev data: AIMCo) — never under TD, RBC, or whoever the email actually came from. That breaks the entire counterparty model: an org can’t quote on its own RFQs, notifications fan out to the wrong list, dashboards show garbage attribution, etc.

The impersonation header lets the bot tell V2 “I’m calling on behalf of this user at this org for this single request”. The header is honoured only for JWTs whose email appears in V2’s BOT_SERVICE_ACCOUNTS allowlist — a regular user’s JWT can never impersonate anyone.

The two headers

Request lifecycle (with impersonation)

1. 1

   1. authMiddleware

   Decodes the bot’s JWT. Puts req.user.email = [email protected] and req.user.orgId = AIMCo on the request.
2. 2

   2. botImpersonation

   Sees that req.user.email is in BOT_SERVICE_ACCOUNTS. Reads X-Act-As-User = [email protected]. Calls /internal/organizations/[email protected] over HMAC.
3. 3

   3. Lookup validates

   Returns { userId, orgId, orgName } ONLY if: user is ACTIVE, has an ACTIVE membership, botEnabled=true on that membership, and the org has botAccess=true. Any failure is forwarded as 403.
4. 4

   4. Rewrite

   The middleware rewrites req.user.id, req.user.email, req.user.orgId, req.orgId to the resolved values. The original bot identity is preserved on req.botActor for downstream audit/logging.
5. 5

   5. Audit

   Emits a fire-and-forget BOT_IMPERSONATION entry to audit-service with both the bot identity (performedBy) and the impersonated identity (entityId), plus path and method.
6. 6

   6. Controller runs

   The downstream controller (POST /api/requests, POST /api/requests/:id/offers, …) sees the rewritten identity and writes the row attributed to the real user’s org.

Security gates

Every one of these must pass — otherwise the headers are silently dropped (no-op) or the request is rejected (403). The middleware itself lives in services-v2/platform/shared/middleware/bot-impersonation.ts.

What the bot sends

Bot-side wiring lives in bot/v2/v2_client.py. The two headers are threaded through _headers() → _v2_request() → create_request(), create_request_batch(), and submit_offer(). The poller (bot/email/poller.py) fills them with the values resolved from the inbound email:

# Side A — posting (bot/email/poller.py::_submit_posting)
await v2_client.create_request(
    org_id,                                  # from thread.org_id
    fields,
    act_as_email=sender_email,               # the email author
    act_as_org=org_id,                       # their resolved V2 org
)

# Side B — offer (bot/email/poller.py::_submit_offer)
# sender_org is cached by _detect_offer_block before this call.
await v2_client.submit_offer(
    request_id,
    payload,
    act_as_email=sender_email,
    act_as_org=sender_org or thread.org_id,
)

# Service calls (lookup, recipients) DO NOT impersonate — they run as
# the bot's own identity so the impersonation middleware stays out of
# the way of the very endpoint it depends on.
await v2_client.lookup_org_by_email(sender_email)            # no headers

Configuration

How to roll it out / roll it back

Auditability

Every successful impersonation writes a BOT_IMPERSONATION entry via emitAuditLog:

{
  "entityType":  "AUTH",
  "action":      "BOT_IMPERSONATION",
  "performedBy": "<bot user id>",        // who really made the call
  "orgId":       "<impersonated org id>",
  "entityId":    "<impersonated user id>",
  "ipAddress":   "<caller ip>",
  "userAgent":   "<caller ua>",
  "details": {
    "botEmail":          "[email protected]",
    "botOrgId":          "demo_org_aimco_000008",
    "actingAsEmail":     "[email protected]",
    "actingAsUserId":    "usr_ernest_kiroma_01",
    "actingAsOrgId":     "demo_org_td_000000002",
    "actingAsOrgName":   "TD Bank",
    "path":              "/requests",
    "method":            "POST"
  }
}

Both the bot identity and the impersonated identity are in the same row, so compliance can answer “who really did this” and “who was it attributed to” without joining tables.

Known good / known bad responses

Files that implement this

Admin dashboard

The same Next.js app you’re reading right now. Reachable at http://localhost:8000. Routes:

Login flow

1. 1

   Step 1 — Password

   Sign in with ADMIN_EMAIL / ADMIN_PASSWORD (or ADMIN_PASSWORD_HASH bcrypt for production). Rate-limited to 10 attempts / minute / IP.
2. 2

   Step 2 — OTP

   A 6-digit one-time code is generated, persisted in admin_otps (so it survives restarts), and sent to ADMIN_OTP_EMAIL. Valid for 10 minutes. Rate-limited to 8 verifications / minute / IP.
3. 3

   Step 3 — JWT

   On success, a signed JWT is returned and stored in localStorage as bot_token. The token expires in 24h and is sent on every API request.

Configuration

All settings come from environment variables (see ye-repo-bot/.env):

Bot identity

V2 integration

V2 stack (set in V2-APP-DASHBOARD/.env)

Admin / LLM / other

Operations & health

Health checks

# Bot service
curl -s http://localhost:8001/health
# {"status":"ok","imap":"connected","db":"ok","version":"1.0.0"}

# Admin dashboard
curl -s http://localhost:8000/api/bot/health

# V2 stack
docker ps --format 'table {{.Names}}\t{{.Status}}' \
  | grep -E 'api-gateway|reference-data|negotiation'

Useful commands

# View latest threads
docker exec ye-repo-bot-bot-1 python -c "
from bot.db import SessionLocal
from bot.db.models import EmailThread
with SessionLocal() as db:
  for t in db.query(EmailThread).order_by(EmailThread.updated_at.desc()).limit(5):
    print(f'{t.state:24s} {t.request_id} {t.updated_at}')
"

# Tail bot logs
docker logs -f ye-repo-bot-bot-1

# Run the e2e regression
docker exec ye-repo-bot-bot-1 python /app/scripts/e2e_qa.py

# Verify the V2 stack has the bot's recent RFQ
docker exec mysql_v2app_bak mysql -uroot -ppassword ye_negotiation -e "
  SELECT id, direction, amount, currency, status, createdAt
  FROM requests ORDER BY createdAt DESC LIMIT 5;"

Logs & their meaning

Security

Troubleshooting

Bot isn’t replying to emails

1. Check the bot is online: docker logs ye-repo-bot-bot-1 --tail 20. Look for IMAP connected.
2. Confirm both the sender’s org has botAccess=true AND the sender personally has botEnabled=true in V2 (Org Settings → Integrations → toggle the member row).
3. Check EMAIL_ENABLED=true in .env.
4. Look for Reply sent to … in logs after the email lands — if absent, the bot didn’t process the message (likely deduped or filtered).

Sender received a “not registered” or “not enabled” reply

The reply itself is your diagnostic — three distinct templates, each pointing at a specific fix:

To trace it server-side, fetch the lookup response the bot saw:

# Inside the bot container, replay the lookup the bot did:
docker exec ye-repo-bot-bot-1 python -c "
import asyncio, json
from bot.v2 import v2_client
async def main():
    r = await v2_client._v2_request('GET', '/api/organizations/lookup',
                                    params={'email': '[email protected]'})
    print(r.status_code, json.dumps(r.json(), indent=2))
asyncio.run(main())
"

“Collateral type is required” on a row that mentions collateral

1. Open the thread in the dashboard, switch to Bot view to see the tokenised text. If a financial term shows up as [TOKEN:ORG:001], the term needs to be added to _FIN_VOCAB_PATTERNS.
2. Otherwise the LLM dropped the field — the regex backfill should recover it; check logs for Backfilled collateral_type for row ….

V2 returned 401 / 502

1. 401: token expired or service account password rotated — bot auto-refreshes once; if still 401, check V2_BOT_EMAIL / V2_BOT_PASSWORD.
2. 502/503/504: V2 service is down or hot-reloading — the retry layer should handle this; if it persists check the corresponding container.

OTP email never arrives

1. Look for OTP sent to … in bot logs — if absent, SMTP isn’t configured. Check SMTP_* env vars.
2. Check the spam folder of ADMIN_OTP_EMAIL.
3. OTPs are valid for 10 minutes. After that, request a new one.

FAQ

Does the bot store the original (raw) email body?

Yes — in thread_messages.body_raw, accessible only to admins via this dashboard. The PII registry is per-thread so the dashboard can rehydrate tokens for display, but tokens are what get sent to the LLM.

What happens if the bot is offline when a user sends an email?

The IMAP poller picks up missed messages on reconnect — the lookback window is max(today − 7 days, last_processed − 1 day). Idempotency keys (Message-ID) prevent double-processing.

How do I add a new collateral type?

Add one line each to (1) _COLLATERAL_NORM in bot/v2/v2_client.py mapping the new spelling to the V2 enum value, (2) _FIN_VOCAB_PATTERNS in bot/core/pii_tokenizer.py so it survives tokenisation, and (3) _COLLATERAL_KEYWORDS in bot/llm/field_extractor.py for the regex backfill. The e2e suite will catch any drift.

Can the bot speak languages other than English?

Not yet — the LLM prompts and all email templates are English-only. Adding multilingual support would require localised templates plus a per-thread locale field.

How do I enable the bot for a new trader?

Sign in as an Org Admin → Settings → Organization → Integrations → find their row in Members allowed to use the bot → toggle Bot access on. Takes effect on the next email they send. The action is audited; no V2 redeploy or admin script required.

How do I urgently disable the bot for the whole firm?

Same screen — flip the top-level Email Bot Integration switch off. All inbound emails from your firm start being rejected immediately with a polite “Email Bot Access Not Enabled” reply. Per-member enabled-states are preserved, so flipping the master back on restores the prior configuration exactly.

Why are new invitees not automatically enabled?

Compliance-safe defaults. botEnabled defaults to false for every new membership row, including newly invited users. The Org Admin must explicitly opt each user in (or click Enable all). This guarantees that adding a user to V2 is never accidentally an authorisation to trade by email.

Can a trader see whether their own bot access is on?

Yes — the Integrations tab is readable by any member, not just admins. Non-admins see the same page in read-only mode (toggles disabled, an info banner explains why). The trader can confirm their own row and ask their admin to flip it if it’s off.

What does adding a domain to the allowed-domains list do today?

Today: it’s saved per org and surfaced in the UI, but the bot’s lookup is still strict-email-match. A future v1.1 will let mail from those domains be attributed to the org even when the specific address isn’t a registered user (after a separate compliance review on guest-sender attribution). Curate the list now; the lookup will start consuming it without a UI change.

Where do I file bugs / feature requests?

File an issue against the ye-repo-bot repo and tag it appropriately. For data corrections on existing threads, contact the on-call engineer with the thread ID — never edit the DB by hand without an audit trail.