Human-in-the-Loop Patterns

Not every agent action should be fully autonomous. HITL patterns let you add human approval gates, collect feedback, handle escalations, and keep humans in control of critical decisions while agents handle the routine work.

Overview

Human-in-the-loop (HITL) is a design pattern where agent workflows pause at specific points to request human input, approval, or review. Rekall supports HITL through execution memory -- pause a task, notify a human, capture their response, and resume.

HITL + Execution Memory

HITL patterns build on top of Execution Memory. Make sure you understand pause/resume/checkpoint before diving into HITL patterns.

When to Pause for Human Input

Common scenarios where you should add a human approval gate:

  • •Destructive operations -- Deleting data, dropping tables, force-pushing to main
  • •External actions -- Sending emails, posting to Slack, creating PRs
  • •Cost-incurring actions -- Cloud provisioning, API calls with usage fees
  • •Ambiguous situations -- When the agent is uncertain about the right action
  • •Compliance requirements -- Actions that require human sign-off for audit
Define HITL gates in a workflow
import Rekall from '@rekall/agent-sdk';


const rekall = new Rekall({ apiKey: 'rk_your_key', agentId: 'agent_abc123' });


// Define which actions require human approval
const hitlPolicy = await rekall.execution.setHITLPolicy({
  rules: [
    {
      action: 'delete',
      resource: '*',
      requireApproval: true,
      approvers: ['user:adam'],
    },
    {
      action: 'deploy',
      resource: 'production',
      requireApproval: true,
      approvers: ['user:adam', 'user:alice'],
      minApprovals: 1,
    },
    {
      action: '*',
      resource: '*',
      requireApproval: false,  // Everything else is auto-approved
    },
  ],
});

Approval Workflows

Single Approval Gate

The simplest HITL pattern: pause execution, notify a human, and wait for approval.

Single approval gate
// During task execution, pause for approval
async function deployToProduction(service: string, version: string) {
  const execution = await rekall.execution.create({
    name: `Deploy ${service}@${version} to production`,
    config: { onFailure: 'pause' },
  });


  await rekall.execution.start({ executionId: execution.id });


  // Step 1: Run tests (auto)
  await runTests(service);
  await rekall.execution.checkpoint({
    executionId: execution.id,
    step: 1,
    message: 'Tests passed',
  });


  // Step 2: Build (auto)
  const image = await buildImage(service, version);
  await rekall.execution.checkpoint({
    executionId: execution.id,
    step: 2,
    state: { image },
    message: 'Image built',
  });


  // Step 3: HITL - Request approval before deploying
  const approval = await rekall.execution.requestApproval({
    executionId: execution.id,
    title: 'Production Deployment Approval',
    description: `Ready to deploy ${service}@${version} to production.\n\nTests: PASSED\nImage: ${image}`,
    approvers: ['user:adam'],
    timeout: 3600000, // 1 hour to approve
    notifyVia: ['email', 'slack'],
  });


  // Execution is now paused. It will resume when approved.
  // The function returns when the approval is granted.


  if (approval.status === 'approved') {
    // Step 4: Deploy (approved by human)
    await deploy(service, version, 'production');
    await rekall.execution.complete({
      executionId: execution.id,
      result: { deployedBy: approval.approvedBy },
    });
  } else {
    // Approval denied or timed out
    await rekall.execution.complete({
      executionId: execution.id,
      result: {
        status: 'rejected',
        reason: approval.reason,
      },
    });
  }
}

Multi-Approval Chains

For critical actions, require multiple approvals from different people or roles.

Multi-approval chain
const approval = await rekall.execution.requestApproval({
  executionId: execution.id,
  title: 'Database Schema Migration',
  description: 'Dropping the legacy_users table and migrating data to users_v2',
  chain: [
    {
      step: 'Technical Review',
      approvers: ['user:alice', 'user:bob'],
      minApprovals: 1,
      timeout: 3600000,
    },
    {
      step: 'Manager Approval',
      approvers: ['user:carol'],
      minApprovals: 1,
      timeout: 7200000,
    },
  ],
  notifyVia: ['email', 'slack'],
});


// The execution pauses until all chain steps are approved
// Each step notifies the next group when the previous is approved
console.log(`Approval status: ${approval.status}`);
console.log(`Chain progress:`);
for (const step of approval.chainStatus) {
  console.log(`  ${step.name}: ${step.status} (${step.approvedBy?.join(', ') || 'pending'})`);
}

Human Feedback Integration

Beyond simple approve/reject, agents can collect structured feedback from humans and use it to improve their next action.

Feedback Loops

Collect structured feedback
// Request feedback with a structured form
const feedback = await rekall.execution.requestFeedback({
  executionId: execution.id,
  title: 'Review Generated Code',
  description: 'Please review the generated authentication module',
  artifacts: [
    { type: 'code', content: generatedCode, language: 'typescript' },
  ],
  form: {
    fields: [
      {
        name: 'quality',
        type: 'rating',
        label: 'Code Quality',
        min: 1,
        max: 5,
      },
      {
        name: 'issues',
        type: 'multiselect',
        label: 'Issues Found',
        options: [
          'Missing error handling',
          'Security concern',
          'Performance issue',
          'Style mismatch',
          'Logic error',
          'No issues',
        ],
      },
      {
        name: 'comments',
        type: 'text',
        label: 'Additional Comments',
        required: false,
      },
    ],
  },
  timeout: 86400000, // 24 hours
});


// Use feedback to improve
if (feedback.data.quality < 3) {
  // Re-generate based on feedback
  const issues = feedback.data.issues.join(', ');
  const comments = feedback.data.comments || '';


  // Store feedback as a memory for learning
  await rekall.memories.create({
    type: 'episodic',
    content: `Code generation feedback: quality ${feedback.data.quality}/5. Issues: ${issues}. ${comments}`,
    metadata: { tags: ['feedback', 'code-generation'] },
    importance: 0.8,
  });
}

Correction Patterns

When a human corrects an agent's output, store the correction as both an episodic memory and a preference signal.

Learning from corrections
// Human provides a correction
const correction = await rekall.execution.requestFeedback({
  executionId: execution.id,
  title: 'Correct Agent Output',
  artifacts: [{ type: 'code', content: agentOutput }],
  form: {
    fields: [
      { name: 'correctedCode', type: 'code', label: 'Your corrected version' },
      { name: 'explanation', type: 'text', label: 'What was wrong?' },
    ],
  },
});


// Store the correction as episodic memory
await rekall.memories.create({
  type: 'episodic',
  content: `Agent output was corrected. Issue: ${correction.data.explanation}`,
  metadata: {
    tags: ['correction', 'learning'],
    original: agentOutput,
    corrected: correction.data.correctedCode,
  },
  importance: 0.85,
});


// This feeds into preference learning automatically
// Next time, Rekall will detect the pattern and suggest a preference

Escalation Patterns

Confidence-Based Escalation

Define escalation rules based on agent confidence. Low-confidence actions escalate to humans; high-confidence ones proceed automatically.

Confidence-based escalation
// Define escalation policy
const policy = await rekall.execution.setEscalationPolicy({
  rules: [
    {
      // High confidence: proceed automatically
      confidenceRange: [0.9, 1.0],
      action: 'auto-approve',
    },
    {
      // Medium confidence: notify but proceed
      confidenceRange: [0.7, 0.9],
      action: 'notify-and-proceed',
      notifyVia: ['slack'],
    },
    {
      // Low confidence: pause and ask
      confidenceRange: [0.4, 0.7],
      action: 'request-approval',
      approvers: ['user:adam'],
      timeout: 3600000,
    },
    {
      // Very low confidence: escalate to senior
      confidenceRange: [0.0, 0.4],
      action: 'escalate',
      approvers: ['user:carol'],  // Manager
      notifyVia: ['email', 'slack'],
      priority: 'high',
    },
  ],
});


// During execution, agent declares its confidence
async function processTask(task: Task) {
  const confidence = await evaluateConfidence(task);


  const decision = await rekall.execution.checkEscalation({
    executionId: execution.id,
    action: task.action,
    confidence,
    context: {
      reasoning: 'Based on 3 similar past tasks, 2 succeeded with this approach',
      alternatives: ['Alternative A', 'Alternative B'],
    },
  });


  if (decision.approved) {
    await executeAction(task);
  } else {
    console.log(`Escalated: ${decision.reason}`);
    // Execution is paused, waiting for human
  }
}

Escalation feeds learning

Every escalation and its resolution is stored as an episodic memory. Over time, Rekall learns which situations the agent handles well and which need human review, automatically adjusting confidence thresholds.

Combining Execution Memory with HITL

Here is a complete example that combines execution checkpoints, HITL approval, human feedback, and escalation in a single workflow.

Complete HITL workflow
1async function codeReviewWorkflow(prNumber: number) {
2  const execution = await rekall.execution.create({
3    name: `Code review for PR #${prNumber}`,
4    config: {
5      checkpointInterval: 'per-step',
6      onFailure: 'pause',
7    },
8  });
9

10  await rekall.execution.start({ executionId: execution.id });
11

12  // Step 1: Automated analysis (no HITL)
13  const analysis = await analyzeCode(prNumber);
14  await rekall.execution.checkpoint({
15    executionId: execution.id,
16    step: 1,
17    state: { analysis },
18    message: `Analysis complete: ${analysis.issues.length} issues found`,
19  });
20

21  // Step 2: Check if critical issues need human review
22  const criticalIssues = analysis.issues.filter(i => i.severity === 'critical');
23

24  if (criticalIssues.length > 0) {
25    // HITL: Pause for human review of critical issues
26    const review = await rekall.execution.requestFeedback({
27      executionId: execution.id,
28      title: `Critical issues in PR #${prNumber}`,
29      description: `Found ${criticalIssues.length} critical issues that need human review.`,
30      artifacts: criticalIssues.map(issue => ({
31        type: 'code',
32        content: issue.codeSnippet,
33        title: issue.title,
34      })),
35      form: {
36        fields: [
37          {
38            name: 'action',
39            type: 'select',
40            label: 'Action',
41            options: ['Block PR', 'Approve with comments', 'False positive'],
42          },
43          { name: 'comments', type: 'text', label: 'Review comments' },
44        ],
45      },
46    });
47

48    await rekall.execution.checkpoint({
49      executionId: execution.id,
50      step: 2,
51      state: { analysis, humanReview: review.data },
52      message: `Human review: ${review.data.action}`,
53    });
54

55    if (review.data.action === 'Block PR') {
56      await postReview(prNumber, 'changes_requested', review.data.comments);
57      await rekall.execution.complete({ executionId: execution.id });
58      return;
59    }
60  }
61

62  // Step 3: Post automated review
63  await postReview(prNumber, 'comment', formatReview(analysis));
64  await rekall.execution.complete({
65    executionId: execution.id,
66    result: {
67      issues: analysis.issues.length,
68      criticalReviewed: criticalIssues.length,
69    },
70  });
71}

Next Steps

Rekall
rekall