# Observability and Debugging in Airtable Scripts

Airtable automations offer limited post-run visibility. The automation history shows console output and whether the run succeeded or failed, but nothing else. There is no structured logging, no log search, no alerting on log content. This means every script must build its own observability layer using two tools: `console.log()` and Airtable field writes.

Beyond runtime observability, the scripts themselves must be self-documenting. Airtable's scripting editor has no file browser, no version history, no diff view, and no way to search across scripts. When someone new opens a script (or when you return to it after three months), the comments are the only context available.

---

## Commenting Best Practices

### Principle: Write for the person who did not write this script.

Airtable automation scripts live inside a UI that strips away every contextual cue a developer normally relies on: file names, directory structure, git history, README files. A script called "Step 2" inside "Automation 220a" tells the next person nothing. The comments must carry all of that context.

This is especially important for rotating headcounts, cross-functional handoffs, and any scenario where the person debugging a failure at 6pm is not the person who wrote the script.

### Script Header Block

Every script must start with a header comment block that answers five questions at a glance:

1. What is this script called and where does it sit in the pipeline?
2. What does it do (one sentence)?
3. What external APIs does it call?
4. What Airtable table(s) does it read from or write to?
5. Are there any known constraints or warnings?

```javascript
/**
 * ============================================================
 * AirtOPSy -- Marketo Forms Scout Script
 * ============================================================
 * Pipeline:  Forms Sync (script 1 of 2)
 * Purpose:   Fetch all Marketo form definitions, upsert into
 *            Forms table, flag records where updatedAt changed.
 * API:       GET /rest/asset/v1/forms.json (offset pagination)
 * Tables:    Reads: System Panel. Writes: Marketo Forms.
 * Note:      Time constants are battle-tested. Adjust with caution.
 * ============================================================
 */
```

For scripts that are part of a chained automation (1a, 1b, 2a, 2b), the header should note the chain position and what the previous/next steps expect:

```javascript
/**
 * ============================================================
 * 220 Script 1b -- GetActivityCountsByDay (days 5-8)
 * ============================================================
 * Pipeline:  220a/220b Activity Fetch (script 2 of 5)
 * Purpose:   Scan Marketo activity log for days 5-8, building on
 *            1a's scan of days 1-4. Outputs active dates for
 *            downstream detail-fetch scripts (2a, 2b).
 * Inputs:    leadId, ActivityIds, startOffsetDays (from 1a output)
 * Outputs:   activitySummary, activeDates, timedOut
 * API:       GET /rest/v1/activities/pagingtoken.json
 *            GET /rest/v1/activities.json
 * ============================================================
 */
```

### Section Dividers

Break the script body into clearly labeled sections. Use the five-asterisk pattern for visual scanning:

```javascript
/***** Inputs *****/
var config = input.config();
// ...

/***** Config *****/
var TABLE_NAME = "Marketo Forms";
var TIME_LIMIT_MS = 170000;
// ...

/***** Helpers *****/
function busyWait(ms) { /* ... */ }
// ...

/***** Guardrails *****/
if (!BASE_URL) throw new Error("Missing input: Base_URL");
// ...

/***** Step 1 -- Authenticate *****/
// ...

/***** Step 2 -- Paginate API *****/
// ...

/***** Step 3 -- Upsert to Airtable *****/
// ...

/***** Step 4 -- Reset System Panel *****/
// ...

/***** Summary *****/
// ...
```

The numbered steps serve double duty: they structure the code for the reader, and they map directly to the console log output, so someone reading the automation history can correlate log lines to code sections.

### Inline Comments: "Why," Not "What"

Do not comment what the code does if the code is readable on its own. Comment why it does something non-obvious:

```javascript
// BAD: restates the code
var half = BASE_DELAY * Math.pow(2, i) / 2;  // divide by 2

// GOOD: explains the design decision
var half = BASE_DELAY * Math.pow(2, i) / 2;  // equal jitter: minimum wait prevents stampeding

// BAD: obvious from context
await table.updateRecordAsync(recordId, payload);  // update the record

// GOOD: explains a constraint
await table.updateRecordAsync(recordId, payload);  // must happen before Resync clear to avoid data loss

// GOOD: flags a known API quirk
var cleaned = dateStr.replace(/Z\+0000$/, "Z");  // Marketo returns non-standard "Z+0000" suffix
```

### Comment Known Constraints and Gotchas

When a line of code exists because of a platform-specific constraint, comment the constraint, not the code:

```javascript
function busyWait(ms) {
  var end = Date.now() + ms;
  while (Date.now() < end) {}  // Airtable has no setTimeout
}

var config = input.config();  // can only be called once per script

var richContent = record.getCellValueAsString("Tokens (Staged)");  // richText does not serialize through input.config()

var TOKEN = input.secret("MarketoProdAirPulseClientId");  // secrets use input.secret(), not input.config()
```

These comments prevent the next developer from "fixing" something that is intentionally designed around a constraint.

### Comment the Automation Context

Scripts run inside automations that have triggers, conditions, and multiple steps. The script cannot see its own automation context. Document it in comments:

```javascript
/***** Automation Context *****/
// Trigger: "Resync" checkbox = true on System Panel record
// Condition: Run Status != "In Progress"
// This is step 1 of 1 in this automation.
// Downstream: none (standalone sync)
```

For chained automations:

```javascript
/***** Automation Context *****/
// Trigger: Record matches view "Pending Diagnosis" in D'AI-gnosis
// Step 1: Bearer token (separate script)
// Step 2: THIS SCRIPT (lead data fetch)
// Step 3: Conditional: if skipped != true
// Step 4: Activity history fetch (220a)
// Inputs from Step 1: bearerToken (output variable)
```

### Comment Density Guidelines

Not every script needs the same comment density:

**Heavy commenting (header + sections + inline):** Production scripts that multiple people touch, scripts in chained automations, scripts with retry/pagination/time-budget logic, anything that calls external APIs.

**Light commenting (header + sections only):** Simple one-off scripts, discovery/console-only scripts, scripts with fewer than 30 lines.

**No comments needed:** Formulas in Airtable fields (which do not support comments anyway), throwaway test scripts.

---

## Principle: Write Errors to Fields, Not Just Console

Console logs disappear into the automation run history. If nobody opens that specific run's log, the error is invisible. Airtable fields, on the other hand, are visible in views, filterable, and can trigger downstream automations or alerts.

**Every script should write its final state to at least one Airtable field.** On success, write a summary. On failure, write the error message. On partial completion, write both.

```javascript
try {
  // ... main logic ...
  await table.updateRecordAsync(recordId, {
    "Run Status": { name: "Done" },
    "Last Run Summary": "Processed " + count + " records in " + elapsed() + "ms",
    "Error Log": ""  // clear previous errors
  });
} catch (err) {
  try {
    await table.updateRecordAsync(recordId, {
      "Run Status": { name: "Error" },
      "Error Log": err.message + "\n\nAt: " + isoNow() + "\nRun ID: " + RUN_ID
    });
  } catch (writeErr) {
    console.log("CRITICAL: Could not write error to Airtable: " + writeErr.message);
    console.log("Original error: " + err.message);
  }
}
```

The nested try/catch on the error write is intentional. If the script fails because of a network issue, the error write might also fail. Log it to console as a last resort, but never let a write failure mask the original error.

---

## Console Logging Conventions

### Run Envelope

Every script should print a clear start block, a clear end block, and a summary. This makes it easy to scan automation history runs quickly.

```javascript
// START
console.log("========================================");
console.log("SCRIPT NAME - Startup");
console.log("========================================");
console.log("Run ID: " + RUN_ID);
console.log("Timestamp: " + isoNow());
console.log("Time budget: " + TIME_LIMIT_MS + "ms");

// ... all work happens here ...

// END
console.log("");
console.log("========================================");
console.log("SCRIPT NAME - Complete");
console.log("========================================");
console.log("   Records processed: " + processed);
console.log("   Records created: " + created);
console.log("   Records updated: " + updated);
console.log("   Records unchanged: " + unchanged);
console.log("   Errors: " + errorCount);
console.log("   API calls: " + fetchCount + "/" + FETCH_LIMIT);
console.log("   Runtime: " + elapsed() + "ms");
```

### Emoji Prefix Taxonomy

Consistent emoji prefixes make logs scannable. This is the convention used across AirtOPSy scripts:

| Prefix | Meaning | Example |
|---|---|---|
| `"[timestamp]"` | Elapsed time marker | `"[12.3s] Fetching page 3"` |
| Checkmark | Success / completion | `"Token acquired"` |
| Warning | Non-fatal issue, retryable | `"Rate limited (606), waiting 2.1s"` |
| X mark | Fatal error, cannot continue | `"Auth failure: invalid credentials"` |
| Magnifier | Discovery / lookup | `"Looking up lead by email"` |
| Document | Data payload info | `"HTTP 200 - 4521 chars"` |
| Clock | Time budget / delay | `"Time budget low: 8.2s remaining"` |
| Rocket | Script phase start | `"Scanning activities for Lead 127303611"` |
| Calendar | Date-specific operation | `"2026-02-19 - 0 activities (2 pages)"` |
| Chart | Summary statistics | `"Smart Lists loaded: 5759"` |
| Link | API endpoint | `"Endpoint: /rest/asset/v1/forms.json"` |
| Key | Authentication event | `"Requesting bearer token (refresh)"` |

Use emojis sparingly in log-dense scripts. The elapsed time prefix `[12.3s]` is more valuable than any emoji for debugging timeout issues.

### Elapsed Time in Every Log Line

For any script that might approach the 180s limit, include elapsed time:

```javascript
var RUN_START = Date.now();
function elapsed() {
  return ((Date.now() - RUN_START) / 1000).toFixed(1);
}

console.log("[" + elapsed() + "s] Starting page fetch");
console.log("[" + elapsed() + "s] Page 1 complete: 200 records");
console.log("[" + elapsed() + "s] Page 2 complete: 187 records");
```

This is the single most useful debugging signal. When a script times out at 180s, the last timestamp in the log tells you exactly which operation was running when the clock ran out.

---

## Run Summary Block

At the end of every script, output a structured summary. This serves two purposes: quick scan of "did it work?" in automation history, and data for the error-to-field write.

```javascript
var summary = {
  runId: RUN_ID,
  timestamp: isoNow(),
  runtimeMs: elapsed(),
  recordsProcessed: processed,
  recordsCreated: created,
  recordsUpdated: updated,
  recordsSkipped: skipped,
  errors: errorCount,
  apiCalls: fetchCount,
  exitReason: timedOut ? "timeout" : fetchCount >= FETCH_LIMIT ? "fetchLimit" : "completed",
  partial: timedOut || fetchCount >= FETCH_LIMIT
};

console.log("SUMMARY: " + JSON.stringify(summary));
```

The `exitReason` field is especially important for chained automations. The next script in the chain uses it to decide whether to continue processing.

---

## Partial Result Handling

When a script cannot complete (timeout, fetch limit, API error), it must preserve whatever work it has done. The worst outcome is a script that fetches 80% of the data, then throws an error on the 81st call and discards everything.

### Pattern: Accumulate and write incrementally

```javascript
var allResults = [];
var lastWrittenIndex = 0;
var WRITE_INTERVAL = 200;  // write to Airtable every 200 records

for (var page = 0; page < MAX_PAGES; page++) {
  if (timeRemaining() < 15000) {
    console.log("Time budget reached - writing partial results");
    break;
  }

  var data = await apiFetch(nextUrl);
  allResults = allResults.concat(data.result || []);

  // Periodic write checkpoint
  if (allResults.length - lastWrittenIndex >= WRITE_INTERVAL) {
    await writeBatch(allResults.slice(lastWrittenIndex));
    lastWrittenIndex = allResults.length;
    console.log("Checkpoint: " + allResults.length + " records written");
  }
}

// Final write of remaining records
if (allResults.length > lastWrittenIndex) {
  await writeBatch(allResults.slice(lastWrittenIndex));
}
```

### Pattern: Flag partial completion

Downstream consumers (AI diagnostics, reporting, alerting) need to know whether they are working with complete or partial data:

```javascript
await table.updateRecordAsync(recordId, {
  "Data Completeness": timedOut ? "Partial" : "Complete",
  "Pages Fetched": page + " of " + estimatedTotal,
  "Last Fetched Date": lastDateProcessed
});
```

---

## Debugging Failed Automation Runs

When investigating a failed run in Airtable's automation history:

### Step 1: Find the last timestamp

Scroll to the bottom of the console output. The last `[Xs]` timestamp tells you when the script stopped. If it is close to 180s, the script timed out. If it is early (under 30s), something threw an error.

### Step 2: Identify the operation

The log line immediately before the error (or immediately before the last timestamp) tells you what was happening: API fetch, Airtable write, token refresh, etc.

### Step 3: Check for cascading failures

Common cascade: API rate limit -> retry with delay -> timeout approaching -> attempted cleanup write -> cleanup write fails because timeout -> no error logged. The fix is always to check `timeRemaining()` before retries and skip the retry if there is not enough time.

### Step 4: Check concurrent runs

If the error is a rate limit (606) or concurrency limit (615), check whether other automations were running at the same time. Airtable's automation history shows run times for all automations in the base. Overlapping runs against the same external API are the most common source of rate limit errors.

### Step 5: Reproduce vs. transient

Rate limits, network timeouts, and API 500 errors are usually transient. If the Resync flag is still set (because the script errored before clearing it), manually trigger a re-run. If the same error recurs, the issue is structural.

---

## Airtable Automation History Limitations

Things you cannot do from the automation history UI:

- Search across console logs from multiple runs
- Filter runs by error type
- Export console logs
- Set alerts on specific log patterns
- See the exact input variable values passed to the script

To compensate, build observability into Airtable itself: error fields, status fields, last-run timestamps, and run ID fields. These are searchable, filterable, and can trigger alert automations.
