veritas-kanban/docs/features/polling.md

Efficient Polling

Optimized agent polling endpoint with ETag support and incremental change detection.

Overview

The /api/changes endpoint provides an efficient mechanism for agents to detect board changes without constantly fetching full task lists. Agents can poll periodically and receive only what's changed since their last check, dramatically reducing bandwidth and processing overhead.

Features

• Incremental updates — Only returns changes since last poll
• ETag support — HTTP caching with 304 Not Modified responses
• Change types — Task created, updated, deleted, archived, restored
• Efficient filtering — Query by timestamp, status, project
• Minimal payload — Only changed task IDs and metadata
• Rate-limit friendly — Designed for frequent polling without overwhelming the server

API Endpoint

GET /api/changes

# Initial poll (get current state)
curl http://localhost:3001/api/changes \
  -H "X-API-Key: YOUR_KEY"

# Poll for changes since timestamp
curl "http://localhost:3001/api/changes?since=2026-02-07T15:00:00Z" \
  -H "X-API-Key: YOUR_KEY"

# Poll with ETag (server returns 304 if nothing changed)
curl "http://localhost:3001/api/changes?since=2026-02-07T15:00:00Z" \
  -H "X-API-Key: YOUR_KEY" \
  -H "If-None-Match: \"abc123\""

Request Parameters

Parameter	Type	Required	Description
since	ISO 8601	❌	Return changes after this timestamp (UTC)
status	string	❌	Filter by task status (todo, in-progress, etc.)
project	string	❌	Filter by project ID

Response Schema

Success Response (200 OK)

{
  "timestamp": "2026-02-07T15:30:00Z",
  "changes": [
    {
      "type": "task.created",
      "taskId": "task_20260207_abc123",
      "timestamp": "2026-02-07T15:10:00Z",
      "data": {
        "title": "Implement feature X",
        "status": "todo",
        "project": "rubicon"
      }
    },
    {
      "type": "task.updated",
      "taskId": "task_20260207_def456",
      "timestamp": "2026-02-07T15:20:00Z",
      "data": {
        "status": "in-progress",
        "assignedAgents": ["TARS"]
      }
    },
    {
      "type": "task.completed",
      "taskId": "task_20260207_ghi789",
      "timestamp": "2026-02-07T15:25:00Z",
      "data": {
        "status": "done",
        "completedBy": "CASE"
      }
    }
  ]
}

No Changes (304 Not Modified)

When using ETag and nothing has changed, server responds with 304 Not Modified and no body. Agent should use cached data.

Empty Changes (200 OK)

{
  "timestamp": "2026-02-07T15:30:00Z",
  "changes": []
}

Change Types

Type	Description	Data Fields
task.created	New task created	title, status, type, project
task.updated	Task fields changed	changed fields only
task.completed	Task marked done	status, completedBy, duration
task.archived	Task moved to archive	archivedBy, archivedAt
task.restored	Task restored from archive	restoredBy, restoredAt
task.deleted	Task permanently deleted	deletedBy, deletedAt
agent.assigned	Agent assigned to task	agent, assignedBy
comment.added	Comment added to task	author, text (truncated)

ETag Behavior

The server includes an ETag header with every response:

HTTP/1.1 200 OK
ETag: "abc123"
Content-Type: application/json

Agents should save the ETag and include it in the next request:

curl "http://localhost:3001/api/changes?since=2026-02-07T15:00:00Z" \
  -H "X-API-Key: YOUR_KEY" \
  -H "If-None-Match: \"abc123\""

If nothing changed, server responds with:

HTTP/1.1 304 Not Modified
ETag: "abc123"

This saves bandwidth and processing time for both client and server.

Polling Strategy

Basic Polling Loop

#!/bin/bash

# Initial state
LAST_CHECK=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
ETAG=""

while true; do
  # Poll for changes
  RESPONSE=$(curl -s -w "\n%{http_code}" \
    "http://localhost:3001/api/changes?since=$LAST_CHECK" \
    -H "X-API-Key: $YOUR_KEY" \
    -H "If-None-Match: $ETAG")

  STATUS_CODE=$(echo "$RESPONSE" | tail -n1)
  BODY=$(echo "$RESPONSE" | head -n-1)

  if [ "$STATUS_CODE" -eq 200 ]; then
    # Parse changes
    echo "Changes detected: $BODY"

    # Update timestamp and ETag
    LAST_CHECK=$(echo "$BODY" | jq -r '.timestamp')
    ETAG=$(curl -I -s "http://localhost:3001/api/changes?since=$LAST_CHECK" \
      -H "X-API-Key: $YOUR_KEY" | grep -i 'etag:' | cut -d' ' -f2 | tr -d '\r\n')

  elif [ "$STATUS_CODE" -eq 304 ]; then
    echo "No changes since $LAST_CHECK"
  else
    echo "Error: HTTP $STATUS_CODE"
  fi

  # Wait 30 seconds before next poll
  sleep 30
done

Advanced Polling with Backoff

#!/bin/bash

LAST_CHECK=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
ETAG=""
POLL_INTERVAL=30
MAX_INTERVAL=300

while true; do
  RESPONSE=$(curl -s -w "\n%{http_code}" \
    "http://localhost:3001/api/changes?since=$LAST_CHECK" \
    -H "X-API-Key: $YOUR_KEY" \
    -H "If-None-Match: $ETAG")

  STATUS_CODE=$(echo "$RESPONSE" | tail -n1)
  BODY=$(echo "$RESPONSE" | head -n-1)

  if [ "$STATUS_CODE" -eq 200 ]; then
    CHANGE_COUNT=$(echo "$BODY" | jq '.changes | length')

    if [ "$CHANGE_COUNT" -gt 0 ]; then
      # Process changes
      echo "Processing $CHANGE_COUNT changes"

      # Reset poll interval (activity detected)
      POLL_INTERVAL=30
    else
      # No changes, increase interval (exponential backoff)
      POLL_INTERVAL=$((POLL_INTERVAL * 2))
      [ $POLL_INTERVAL -gt $MAX_INTERVAL ] && POLL_INTERVAL=$MAX_INTERVAL
    fi

    LAST_CHECK=$(echo "$BODY" | jq -r '.timestamp')

  elif [ "$STATUS_CODE" -eq 304 ]; then
    # No changes, increase interval
    POLL_INTERVAL=$((POLL_INTERVAL * 2))
    [ $POLL_INTERVAL -gt $MAX_INTERVAL ] && POLL_INTERVAL=$MAX_INTERVAL
  fi

  echo "Next poll in $POLL_INTERVAL seconds"
  sleep $POLL_INTERVAL
done

Common Use Cases

Detect New Task Assignments

# Poll for changes
CHANGES=$(curl -s "http://localhost:3001/api/changes?since=$LAST_CHECK" \
  -H "X-API-Key: YOUR_KEY")

# Filter for agent assignments
NEW_ASSIGNMENTS=$(echo "$CHANGES" | jq -r \
  '.changes[] | select(.type == "agent.assigned" and .data.agent == "TARS") | .taskId')

# Process each assignment
for TASK_ID in $NEW_ASSIGNMENTS; do
  echo "New assignment: $TASK_ID"
  vk begin "$TASK_ID"
done

Detect Completed Tasks

CHANGES=$(curl -s "http://localhost:3001/api/changes?since=$LAST_CHECK" \
  -H "X-API-Key: YOUR_KEY")

COMPLETED=$(echo "$CHANGES" | jq -r \
  '.changes[] | select(.type == "task.completed") | .taskId')

for TASK_ID in $COMPLETED; do
  echo "Task completed: $TASK_ID"
  # Post to squad chat
  curl -X POST http://localhost:3001/api/chat/squad \
    -H "Content-Type: application/json" \
    -H "X-API-Key: $YOUR_KEY" \
    -d "{\"agent\":\"VERITAS\",\"message\":\"Task $TASK_ID completed!\"}"
done

Monitor Specific Project

# Only check changes for rubicon project
CHANGES=$(curl -s "http://localhost:3001/api/changes?since=$LAST_CHECK&project=rubicon" \
  -H "X-API-Key: YOUR_KEY")

echo "Changes in Rubicon project:"
echo "$CHANGES" | jq '.changes[] | "\(.type): \(.taskId)"'

Performance Characteristics

Operation	Time Complexity	Notes
Poll with no changes	O(1)	ETag comparison only
Poll with changes	O(n)	n = number of changes since last poll
Filter by status/project	O(n)	Applied after change detection
Timestamp comparison	O(1)	Indexed by updatedAt

Rate Limiting

The /api/changes endpoint is designed for frequent polling and has relaxed rate limits compared to other endpoints:

• No rate limit for 304 responses — Cached responses are free
• Standard rate limit for 200 responses — 100 requests per minute per API key
• Recommended poll interval — 30-60 seconds for active monitoring

Caching Strategy

Agents should implement a two-tier cache:

1. ETag cache — Store last ETag for 304 checks
2. Data cache — Store last known state to avoid re-fetching unchanged tasks

# Pseudocode
if response.status == 304:
  return cached_data
elif response.status == 200:
  cached_data = merge(cached_data, response.changes)
  etag_cache = response.etag
  return cached_data

Comparison to Full Task List

Without Polling Endpoint

# Fetch entire task list every poll
curl http://localhost:3001/api/tasks \
  -H "X-API-Key: YOUR_KEY"
# Returns ~100KB for 50 tasks
# Requires parsing entire list to find changes

With Polling Endpoint

# Fetch only changes
curl "http://localhost:3001/api/changes?since=..." \
  -H "X-API-Key: YOUR_KEY"
# Returns ~2KB for 5 changes
# Changes are pre-identified

Bandwidth savings: 98% reduction
Processing time: 95% reduction

Security Notes

• Endpoint requires authentication (X-API-Key header)
• Respects agent permissions (only returns changes visible to the agent)
• Timestamps are server-authoritative (prevents tampering)
• Rate limiting prevents abuse

Troubleshooting

Changes Not Appearing

1. Verify timestamp is in UTC: date -u +"%Y-%m-%dT%H:%M:%SZ"
2. Check server logs for change events
3. Ensure agent has permission to view the tasks

304 Responses Not Working

1. Verify ETag is properly extracted and sent
2. Check for extra whitespace in ETag header
3. Ensure If-None-Match header format is correct: If-None-Match: "abc123"

High Bandwidth Usage

1. Implement exponential backoff during quiet periods
2. Use ETag to avoid re-fetching unchanged data
3. Filter by project/status to reduce change volume

Related Documentation

• Agent Task Workflow — Agent task lifecycle
• CLI Guide — CLI commands for task management
• REST API — Full API reference