Wiki · Data Tagger Path 3 · bid RFP → pipeline · R598

Data Tagger Path 3 — bid RFP to pipeline

A bid or RFP arrives, gets auto-extracted into the Bid Center's external pipeline log, and triggers the existing bid_pipeline_review workflow. This path is the formal bridge from the Data Tagger pillar (5) into the Bid Center pillar (4).

strategies REAL (mig 142) ★ pillar bridge to Bid Center
What this is

The bridge between Data Tagger and Bid Center

RFPs and bid invitations arrive at bids@ai-globalfoodsolutions.co. The Data Tagger applies the bid_rfp template, extracts the canonical bid metadata (bid_id, customer / district, items[], decision_date, bid_value), and writes to bid_external_pipeline — a table owned by the Bid Center pillar (migration 136, Agent M). This is the crossing-the-streams moment: the Data Tagger's output becomes the Bid Center's input.

After insert, the bid_pipeline_review workflow fires; the Bid Center dispatcher picks one of 11 skills (typically LOG_EXTERNAL_BID on first arrival, then BID_PIPELINE for status rollup). From there, the bid lives in the Bid Center pillar: pricing work happens there, the .docx quote regenerates there, and if we win the bid the customer joins the 69-customer cohort and eventually rolls up into the Jul 1 named-list flip.

Diagram: ns-data-tagger-path-3-bid-rfp-to-pipeline.html. Downstream: ns-bid-center-master.html.

When to use it

Trigger conditions

Pillar bridge

This is the most explicit cross-pillar handoff in the platform. Step 16 of the diagram closes the Data Tagger's involvement. From then on, the bid lives in the Bid Center.

Worked example

NYC DOE B5875 Food Distribution

Scenario

NYC Department of Education sends bid B5875 (Food Distribution, due Jun 3 2026, ~246 spec rows, ~$4.2M est) to bids@ai-globalfoodsolutions.co. Pipeline runs: log + R2 + parse. Sender domain resolves to customer_id = NYC DOE. Classifier returns doc_type = bid_rfp. Template lookup finds tpl_nycdoe_bid_v2 (district default).

5 strategies run:

  • regex_after_label Solicitation # -> bid_id = "B5875" (0.95)
  • literal_constant customer_id = "NYC DOE" (1.00)
  • table_with_headers Item / Description / Qty / Unit -> 246 spec rows (0.89)
  • regex_after_label Decision Date -> "2026-06-03" (0.94)
  • formula sum(qty * est_unit) -> bid_value = $4.2M est (0.78)

Weighted overall = 0.91. HITL stages "NYC DOE B5875 - approve pipeline insert?" Mike approves. INSERT bid_external_pipeline with status=proposed. bid_pipeline_review workflow fires, dispatches to LOG_EXTERNAL_BID. events.bid.rfp_logged fires; gfs-pricing.pages.dev bid_pipeline tile updates.

From step 16 on, the bid lives in Bid Center pillar. Mike decides whether to bid; if yes, FILL_REGIONAL_PRICES + ADD_UPDATE_CUSTOMER_PRICING work begins.

Step-by-step

18 steps from RFP to Bid Center handoff

  1. 01-02

    Intake

    bids@ -> log + R2 + parse

  2. 03-05

    Identify

    buyer + doc_type + template (with district inheritance)

  3. 06-10

    Apply 5 strategies

    bid_id / customer / items[] / decision_date / bid_value

  4. 11

    Confidence

    weighted (bid_value lower weight - estimates fuzzy)

  5. 12

    Stage proposed_action

    data_tagger_bid_extraction

  6. 13

    INSERT bid_external_pipeline

    ★ cross-pillar write (Bid Center mig 136)

  7. 14

    Trigger bid_pipeline_review workflow

    Bid Center dispatcher fires

  8. 15

    events.bid.rfp_logged

    dashboard refreshes

  9. 16

    ★ BRIDGE to Bid Center pillar

    Data Tagger's job is done

  10. 17-18

    Reflexion + subscribers

    template metrics + dashboards + notifications

Outcomes

What's different after the cycle

Pipeline row
+1
bid_external_pipeline
Workflow
fired
bid_pipeline_review
Dashboard
refreshed
gfs-pricing.pages.dev
Pillar
handed off
5 -> 4
Failure modes

What can go wrong

Unknown buyer

Buyer not in customers table. System flags customer_id=NEW; Mike resolves in HITL by picking the right customer (or creating a new bid_customers row).

Free-form bid (no tables)

Some RFPs ship as DOCX prose with no item table. table_with_headers fails; fallback to llm_with_schema which is expensive. Per-district template specialization helps.

bid_external_pipeline mig 136 not yet landed

Bid Center migration 136 (8 bid_* tables) is in flight (Agent M). Until landed, Path 3 stages but doesn't complete the insert; rows sit in data_tagger_extractions.

Related

Adjacent flows + diagrams

DiagramPath 3 diagram MasterData Tagger pillar wiki ★ Pillar 4Bid Center master (downstream) Pillar 4 wikiBid Center master wiki SiblingBid Center Path 1 (similar intake, less structured) EventualJul 1 rollover (if we win) Live bidNYC DOE B5875 lifecycle
For developers

Code paths + invariants

ConcernWhere
Mailboxbids@ai-globalfoodsolutions.co
Email pipelinesrc/email.ts
Parsersrc/document_converter.ts
Templatetpl_nycdoe_bid_v2 in data_tagger_templates
Destinationbid_external_pipeline (Bid Center mig 136)
Workflowbid_pipeline_review (Bid Center)
Workflow runnerPOST /api/workflow/run
Eventevents.bid.rfp_logged
Dashboardgfs-pricing.pages.dev
// On approve, write to Bid Center owned table, then trigger Bid Center workflow await db.prepare("INSERT INTO bid_external_pipeline (bid_id, customer_id, status, items_json, decision_date, bid_value, source_extraction_id) VALUES (?, ?, 'proposed', ?, ?, ?, ?)").bind(...).run(); await fetch('/api/workflow/run', { method: 'POST', body: JSON.stringify({ workflow_type: 'bid_pipeline_review', bid_id: 'B5875' }) }); // Bid Center pillar takes over from here
Changelog

Dated trail

DateRoundChangeTouched by
2026-05-27R598Path 3 wiki + diagram shipped. NYC DOE B5875 used as worked example. Pillar bridge documented (5 -> 4).Mike + Claude
Schema

5 field tags for bid_rfp template

#strategyfieldexample (B5875)
1regex_after_labelbid_idB5875
2literal_constantcustomer_idNYC DOE
3table_with_headersitems_json246 spec rows
4regex_after_labeldecision_date2026-06-03
5formula / llm_with_schemabid_value$4.2M est

Pillar bridge field linkages

From (Data Tagger)To (Bid Center)Field
data_tagger_extractionsbid_external_pipelineextraction_id -> source_extraction_id
extracted customer_idbid_customerscustomer_id
extracted bid_idBid Center workflowbid_id
Runbook

It broke - what now

Scenario · bid extracted but not visible in Bid Center dashboard

Mike approved but the bid doesn't appear in gfs-pricing.pages.dev pipeline tile.

  1. Check bid_external_pipeline row: SELECT * FROM bid_external_pipeline WHERE bid_id='B5875'
  2. If missing: Bid Center mig 136 may not be landed. Check SELECT name FROM sqlite_master WHERE name='bid_external_pipeline'
  3. Check workflow ran: SELECT * FROM workflow_run_log WHERE workflow_type='bid_pipeline_review' ORDER BY started_at DESC LIMIT 5
  4. Force re-render dashboard

Scenario · Unknown buyer; HITL stuck

Sender domain doesn't match; Mike sees customer_id=NEW.

  1. Add to customers if it's a real new buyer
  2. Add alias to name_synonyms.json if it's an existing buyer with a new domain
  3. Re-run extraction: POST /api/data-tagger/apply against the R2 key

Logs to check

Backlog

Open items for Path 3