MolKit logo
Learn/Build AI Email Auto-Responder
Tools

Step 9 of 10

Log outcomes and track key metrics

Capture structured logs for every email the system processes so you can monitor system performance, catch regressions, and make data-driven improvements.

Why this matters

Without logs, you are flying blind. You cannot answer basic questions like 'what percentage of emails are being auto-sent?' or 'which intent category has the lowest classification confidence?' Without answers to those questions, you cannot improve the system, you can only guess. Logs turn your automation from a black box into a measurable system.

Build instructions

Add a log row at the end of each path

  1. Step 1

    In both Path A and Path B, add a Google Sheets 'Create Spreadsheet Row' action as the final step. Both paths log to the same sheet but in different row patterns.

  2. Step 2

    Create a sheet tab named 'Run Log' with these column headers: Timestamp | Sender | Subject | Intent | Confidence | Risk Flag | Route Taken | Send Result | Notes.

  3. Step 3

    In Path A, Route Taken = 'auto-sent'. In Path B, Route Taken = 'escalated'. For error paths, Route Taken = 'failed'. Map the remaining fields from the classification and trigger outputs.

Build an automated metrics tab

  1. Step 1

    Create a third sheet tab named 'Metrics'. This tab uses formulas to calculate performance metrics from the Run Log tab automatically.

  2. Step 2

    Add these formulas (adjust cell ranges to match your log): Auto-send rate: =COUNTIF('Run Log'!G:G,"auto-sent")/COUNTA('Run Log'!A:A). Escalation rate: =COUNTIF('Run Log'!G:G,"escalated")/COUNTA('Run Log'!A:A). Average confidence (auto-sent only): =AVERAGEIF('Run Log'!G:G,"auto-sent",'Run Log'!D:D).

  3. Step 3

    Add a 'Last updated' cell that shows =NOW() so you can see when the metrics were last calculated.

Define alert thresholds

  1. Step 1

    In the Metrics tab, add a column called 'Status' next to each metric. Use a formula to flag when a metric crosses a threshold: =IF(B2<0.60,"REVIEW","OK").

  2. Step 2

    Thresholds to consider: Auto-send rate below 60% → too many escalations, check classifier accuracy. Average confidence below 0.75 → classifier is uncertain, review training examples. Failure rate above 5% → pipeline issue, investigate send step errors.

  3. Step 3

    Review the Metrics tab every Monday morning. Any 'REVIEW' flag gets investigated before new features are added.

Common mistakes

  • Only logging successful auto-sends. Logging only the happy path gives you a misleadingly positive picture of system performance. Log every path: auto-sent, escalated, and failed.
  • Building the metrics tab manually (by reading the log and updating numbers). Manual metrics become stale within a day. Use formulas so metrics update automatically with every new log row.

Pro tips

  • Share the Metrics tab link with any stakeholders who care about the system's performance. A shared metrics view prevents 'is the system working?' questions from coming to you every week.

Before you continue

Run 10 test emails through the full workflow (mix of auto-sent, escalated, and at least one failed). Check the Run Log, all 10 should appear with complete data in every column. Check the Metrics tab, the auto-send rate and confidence average should reflect the 10 test runs accurately.

Step result

Every email processed by the system leaves a structured log entry. The metrics tab gives you a live view of system health at any time.