Temporal Workflows

GTM Clarity uses Temporal.io to orchestrate long-running, fault-tolerant operations. All connector sync and writeback operations run as Temporal workflows, providing durability, automatic retries, and observability.

Workflow Types

Connector Sync Workflow

The primary workflow that syncs data from an external source into GTM Clarity:

Workflow ID pattern: sync-{tenantId}-{connectorId}-{timestamp}

Task queue: connector-sync

Input:

interface ConnectorSyncInput {
  tenantId: string;
  connectorId: string;
  fullSync: boolean;  // true = re-import all data; false = incremental from cursor
}

Phases:

Phase	Description
authenticating	Decrypts stored credentials and validates access
syncing	Fetches batches from the source via cursor pagination
writing	Upserts records into the canonical schema
finalizing	Updates sync state (cursor, timestamp, stats)
complete	Sync finished successfully
failed	Unrecoverable error encountered

Scheduled Sync Workflow

A cron workflow that triggers sync for all connected connectors in a tenant:

interface ScheduledSyncInput {
  tenantId: string;
}

The scheduler:

1. Queries all connectors with status = connected for the tenant
2. Spawns a child connectorSyncWorkflow for each connector
3. Each child runs independently -- one failure does not block others
4. Returns aggregate results (processed count, successes, failures)

Child workflow ID pattern: sync-{tenantId}-{connectorId}-{timestamp}

Writeback Workflow

Pushes computed scores back to the source CRM (e.g., Salesforce custom fields). Uses the same Temporal infrastructure but with a separate workflow definition.

Activity Dispatch

Activities are the units of work executed by Temporal workers. The activity layer uses a dispatcher pattern that routes to connector-specific implementations:

Each connector type has four activity implementations:

Activity	Purpose	Timeout
authenticate	Decrypt tokens, validate access	5 minutes
syncBatch	Fetch a page of records from source	5 minutes
writeRecords	Upsert records into canonical schema	5 minutes
updateSyncState	Persist cursor and sync statistics	5 minutes

Retry Policies

Activity-Level Retries

Activities are configured with automatic retries:

Setting	Value
startToCloseTimeout	5 minutes
maximumAttempts	3
Backoff	Exponential (Temporal default)

Dead Letter Pattern

If a batch fails to write after 3 retries, the workflow does not fail entirely. Instead:

1. The failed batch is logged to a dead letter array (max 1,000 entries)
2. Error counts are incremented in the stats
3. The workflow continues with the next batch
4. Final sync state includes error counts for monitoring

// Dead letter entry structure
interface FailedRecord {
  batchIndex: number;
  error: string;
  timestamp: string;  // ISO 8601
}

Scheduler Retries

The scheduled sync activities use lighter retry settings:

Setting	Value
startToCloseTimeout	30 seconds
maximumAttempts	3

Cron Scheduling

The scheduledSyncWorkflow is registered as a Temporal cron workflow:

• Default schedule: Every 4 hours (0 */4 * * *)
• Scope: One schedule per tenant
• Behavior: Lists all connected connectors, starts child sync for each

tip

For high-volume data sources like Salesforce, consider running syncs more frequently (e.g., every 15 minutes) to keep engagement signals fresh for scoring.

Progress Tracking

The sync workflow exposes a Temporal query for real-time progress monitoring:

interface SyncProgress {
  entityType: string;
  recordsProcessed: number;
  errorsCount: number;
  phase: "authenticating" | "syncing" | "writing" | "finalizing" | "failed" | "complete";
}

Query the progress from any Temporal client:

const handle = client.workflow.getHandle(workflowId);
const progress = await handle.query(getSyncProgressQuery);

Worker Configuration

Temporal workers are configured to poll the connector-sync task queue:

Setting	Recommended Value
Task queue	connector-sync
Max concurrent activities	10
Max concurrent workflows	5

Workers should be deployed as a separate process from the Next.js web server. In production, use dedicated worker instances.

warning

Temporal workflows must be deterministic -- they cannot use Math.random(), Date.now(), or make external calls directly. All non-deterministic operations must be performed in activities.

Monitoring

Temporal Web UI

Access the Temporal Web UI at http://localhost:8233 (local development) or your production Temporal Cloud dashboard to:

• View running and completed workflows
• Inspect workflow history and event logs
• Query workflow state (including sync progress)
• Terminate stuck workflows

Key Metrics to Watch

Metric	Healthy Range	Alert Threshold
Sync duration	< 15 minutes	> 30 minutes
Failed batches per sync	0	> 10
Workflow failures per day	0	> 3
Activity retry rate	< 5%	> 20%

Related Pages

• Connector Management -- Configuring connectors that use these workflows
• Environment Setup -- Setting up the Temporal server
• Architecture -- Where workflows fit in the system