Comprehensive Guardrails Integration Guide

Overview

ChainAlign's Comprehensive Guardrails System orchestrates all quality checks (financial, scenario, content, and RAG) into a unified validation framework. This ensures all AI-generated insights meet enterprise standards before deployment.

Key Achievement: The guardrails system now integrates:

• ✅ Financial Accuracy Guardrails (P&L, ratios, temporal consistency)
• ✅ Scenario Financial Guardrails (parameter bounds, outcome sanity, coherence)
• ✅ Language & Content Quality Guardrails (professional tone, factual accuracy, business appropriateness)
• ✅ RAG Evaluation Service (retrieval, generation, business task success)
• ✅ Comprehensive Coordinator (orchestrates all guardrails in unified pipeline)

---

Architecture

Four-Pillar Guardrails Framework

┌─────────────────────────────────────────────────────────────┐
│     ComprehensiveGuardrailsCoordinator (Orchestrator)       │
│   Unified validation pipeline for all AI-generated content  │
└────────────┬──────────┬─────────────┬────────────┬──────────┘
             │          │             │            │
    ┌────────▼──┐ ┌─────▼──────┐ ┌──▼──────────┐ ┌▼────────────┐
    │ Financial │ │  Scenario  │ │   Language  │ │     RAG     │
    │ Accuracy  │ │ Financial  │ │   Content   │ │ Evaluation  │
    │ Guardrails│ │ Guardrails │ │ Guardrails  │ │ Service     │
    └─────┬─────┘ └─────┬──────┘ └──┬──────────┘ └─┬────────────┘
          │              │           │             │
    ┌─────▼──────────────▼───────────▼─────────────▼──────┐
    │         Comprehensive Validation Report            │
    │  - Status: PASS / REVIEW_RECOMMENDED / FAIL        │
    │  - Risk Level: LOW / MEDIUM / HIGH / CRITICAL      │
    │  - Audit Trail Logged (SOX Compliance)            │
    └──────────────────────────────────────────────────────┘

Component Services

1. FinancialAccuracyEvaluationService

Location: /backend/src/services/FinancialAccuracyEvaluationService.js

Validates financial data integrity:

• P&L Reconciliation: Validates accounting equations (Revenue - COGS = Gross Profit, etc.)
• Ratio Sanity: Checks profitability, solvency, liquidity, efficiency ratios against industry benchmarks
• Temporal Consistency: Detects anomalies across periods (revenue spikes, margin shifts)

Key Methods:

validatePLReconciliation(financialData, tolerance)   // Tolerance default: 0.0001 (0.01%)
validateRatioSanity(financialData)                   // Returns errors + warnings
validateTemporalConsistency(financialDataSeries)     // Detects data quality issues
validateFinancialData(tenantId, userId, data)        // Full validation + audit log

2. ScenarioFinancialGuardrails

Location: /backend/src/services/ScenarioFinancialGuardrails.js

Prevents unrealistic financial scenarios:

• Parameter Validation: Revenue growth, COGS, OpEx, R&D, CapEx bounds checking
• Outcome Sanity: Ensures projections are mathematically plausible
• Coherence Check: Validates logical consistency (e.g., revenue decline with margin improvement is unusual)
• Feasibility & Risk: Identifies scenarios requiring executive approval

Key Methods:

validateScenarioParameters(scenarioParams)               // Parameter bounds
validateProjectionOutcomes(projections)                  // Outcome sanity
validateScenarioCoherence(scenarioParams, projections)   // Logical consistency
validateScenarioFeasibility(scenario, projections)       // Risk assessment
evaluateScenarioGuardrails(tenantId, userId, ...)      // Full evaluation + audit

3. LanguageContentQualityGuardrails

Location: /backend/src/services/LanguageContentQualityGuardrails.js

Ensures content meets enterprise standards:

• Professional Language: Detects marketing hype, qualifier overuse, passive voice >30%
• Factual Accuracy: Catches hallucinations, ensures required disclosures, validates citations
• Business Appropriateness: Ensures actionability, context relevance, domain language, urgency alignment

Key Methods:

validateProfessionalLanguage(content, options)          // Tone & language quality
validateFactualAccuracy(content, context)                // Hallucination detection
validateBusinessAppropriateness(content, context)        // Business relevance
validateContentQuality(tenantId, userId, content, opts)  // Full validation + audit

4. RAGEvaluationService

Location: /backend/src/services/RAGEvaluationService.js

Three-layer RAG quality evaluation:

• Layer 1: Curation feedback (human KB review decisions)
• Layer 2: Core RAG metrics (retrieval hit rate, precision, recall; generation faithfulness, relevancy)
• Layer 3: Business-level task success (realistic S&OP problem solving)

Key Methods:

logCurationFeedback(curationData)                   // Layer 1 data
evaluateRetrieval(retrievalTest)                   // Layer 2 retrieval metrics
evaluateGeneration(generationTest)                 // Layer 2 generation metrics
evaluateBusinessTaskSuccess(businessTest)          // Layer 3 business success
aggregateEvaluationResults(evalResults)            // Overall RAG score 0-100

5. ComprehensiveGuardrailsCoordinator (NEW)

Location: /backend/src/services/ComprehensiveGuardrailsCoordinator.js

Purpose: Orchestrates all guardrails in a unified pipeline.

Key Methods:

// Individual guardrail validation
validateFinancialData(financialData, options)
validateScenario(scenario, projections, options)
validateContent(tenantId, userId, content, context, options)
evaluateRAGQuality(ragTest, testType)

// Full insight validation pipeline
validateInsight(tenantId, userId, insight, options)

Comprehensive Report Structure:

{
  timestamp: Date,
  insightId: string,
  insightType: string,
  guardrailsRun: ['CONTENT_QUALITY', 'FINANCIAL_ACCURACY', 'SCENARIO_VALIDATION'],
  overallStatus: 'PASS' | 'REVIEW_RECOMMENDED' | 'FAIL',
  overallRiskLevel: 'LOW' | 'MEDIUM' | 'HIGH' | 'CRITICAL',
  requiresApproval: boolean,
  allIssues: Issue[],
  allWarnings: Warning[],
  recommendations: string[]
}

---

API Endpoints

Base URL

POST /api/guardrails/validate-financial
POST /api/guardrails/validate-scenario
POST /api/guardrails/validate-content
POST /api/guardrails/evaluate-rag
POST /api/guardrails/validate-insight
GET  /api/guardrails/status

1. Validate Financial Data

POST /api/guardrails/validate-financial
Content-Type: application/json

{
  "financialData": {
    "revenue": 1000000,
    "cogs": 600000,
    "gross_profit": 400000,
    "opex": 200000,
    "ebitda": 200000,
    "depreciation_amortization": 50000,
    "interest": 20000,
    "tax": 20000
  },
  "tolerance": 0.0001
}

Response:

{
  "success": true,
  "report": {
    "timestamp": "2025-10-23T...",
    "dataType": "financial",
    "guardrailsChecked": [
      {
        "name": "FINANCIAL_ACCURACY",
        "passed": true,
        "errors": [],
        "warnings": []
      },
      {
        "name": "RATIO_SANITY",
        "passed": true,
        "errors": [],
        "warnings": []
      }
    ],
    "overallStatus": "PASS",
    "overallRiskLevel": "LOW",
    "issues": [],
    "warnings": []
  }
}

2. Validate Scenario

POST /api/guardrails/validate-scenario
Content-Type: application/json

{
  "scenario": {
    "id": "scenario_123",
    "predicted_impact": {
      "revenue_growth_rate": 0.15,
      "cogs_pct_revenue": 0.55,
      "opex_growth_rate": 0.08,
      "rd_pct_revenue": 0.08,
      "capex_pct_revenue": 0.05
    }
  },
  "projections": {
    "period_1": {
      "revenue": 1150000,
      "cogs": 632500,
      "ebitda": 230000
    }
  }
}

Response:

{
  "success": true,
  "report": {
    "timestamp": "2025-10-23T...",
    "dataType": "scenario",
    "scenarioId": "scenario_123",
    "overallStatus": "PASS" | "REVIEW_REQUIRED" | "FAIL",
    "overallRiskLevel": "LOW" | "MEDIUM" | "HIGH" | "CRITICAL",
    "requiresApproval": false,
    "guardrailsChecked": [
      {
        "name": "PARAMETER_VALIDATION",
        "passed": true,
        "violations": []
      },
      {
        "name": "OUTCOME_SANITY",
        "passed": true,
        "violations": []
      },
      {
        "name": "COHERENCE_CHECK",
        "passed": true,
        "violations": []
      }
    ],
    "allIssues": [],
    "allWarnings": []
  }
}

3. Validate Content

POST /api/guardrails/validate-content
Content-Type: application/json

{
  "content": "Based on our analysis, we recommend increasing capacity by 25% to meet Q1 demand. This requires immediate action on raw material procurement.",
  "context": {
    "audience": "executive",
    "businessUnit": "supply_chain",
    "urgency": "critical",
    "decisionContext": "Capacity planning"
  },
  "options": {
    "languageOptions": {
      "targetAudience": "executive",
      "minProfessionalScore": 0.8
    },
    "factualContext": {
      "requiredDisclosures": ["assumes stable market conditions"]
    }
  }
}

Response:

{
  "success": true,
  "report": {
    "timestamp": "2025-10-23T...",
    "dataType": "content",
    "guardrailsChecked": [
      {
        "name": "PROFESSIONAL_LANGUAGE",
        "passed": true,
        "score": 0.95,
        "warnings": []
      },
      {
        "name": "FACTUAL_ACCURACY",
        "passed": true,
        "score": 0.88,
        "warnings": []
      },
      {
        "name": "BUSINESS_APPROPRIATENESS",
        "passed": true,
        "score": 0.92,
        "warnings": []
      }
    ],
    "overallStatus": "PASS",
    "overallScore": 92,
    "requiresReview": false,
    "recommendations": []
  }
}

4. Evaluate RAG Quality

POST /api/guardrails/evaluate-rag
Content-Type: application/json

{
  "ragTest": {
    "query": "What capacity constraints limit Q1 growth?",
    "retrievedDocs": [
      {
        "id": "doc_1",
        "title": "Production Capacity Report",
        "relevanceScore": 0.95
      },
      {
        "id": "doc_2",
        "title": "Q1 Demand Forecast",
        "relevanceScore": 0.87
      }
    ],
    "generatedAnswer": "Current production capacity of...",
    "groundTruth": "Reference answer here"
  },
  "testType": "retrieval"
}

Response:

{
  "success": true,
  "report": {
    "timestamp": "2025-10-23T...",
    "dataType": "rag",
    "testType": "retrieval",
    "overallScore": 87.5,
    "passesThreshold": true,
    "evaluation": {
      "hitRate": 0.95,
      "precisionAtK": 0.92,
      "recallAtK": 0.88,
      "mrr": 0.95,
      "ndcgAtK": 0.89
    }
  }
}

5. Comprehensive Insight Validation (Full Pipeline)

POST /api/guardrails/validate-insight
Content-Type: application/json

{
  "insight": {
    "id": "insight_456",
    "type": "scenario_impact",
    "content": "Implementing aggressive growth strategy increases risk...",
    "context": {
      "audience": "executive",
      "businessUnit": "finance",
      "decisionContext": "Strategic Planning",
      "urgency": "high"
    },
    "financialData": {
      "revenue": 1000000,
      "cogs": 550000,
      "ebitda": 200000
    },
    "scenario": {
      "id": "scenario_789",
      "predicted_impact": {
        "revenue_growth_rate": 0.25,
        "cogs_pct_revenue": 0.55
      }
    },
    "projections": {
      "period_1": { "revenue": 1250000, "ebitda": 250000 }
    }
  },
  "options": {}
}

Response:

{
  "success": true,
  "report": {
    "timestamp": "2025-10-23T...",
    "insightId": "insight_456",
    "insightType": "scenario_impact",
    "guardrailsRun": [
      "CONTENT_QUALITY",
      "FINANCIAL_ACCURACY",
      "SCENARIO_VALIDATION"
    ],
    "overallStatus": "PASS",
    "overallRiskLevel": "MEDIUM",
    "requiresApproval": false,
    "allIssues": [],
    "allWarnings": [
      {
        "guardrail": "SCENARIO_VALIDATION",
        "severity": "WARNING",
        "parameter": "revenue_growth_rate",
        "value": 25,
        "message": "Revenue growth >30% is aggressive - ensure clearly justified"
      }
    ],
    "recommendations": [
      "Provide clear justification for aggressive growth assumptions"
    ]
  }
}

6. Guardrails System Status

GET /api/guardrails/status

Response:

{
  "success": true,
  "status": {
    "timestamp": "2025-10-23T...",
    "guardrailsAvailable": [
      "FINANCIAL_ACCURACY",
      "SCENARIO_VALIDATION",
      "CONTENT_QUALITY",
      "RAG_EVALUATION"
    ],
    "components": {
      "financialAccuracy": {
        "checks": [
          "PL_Reconciliation",
          "Ratio_Sanity",
          "Temporal_Consistency"
        ],
        "status": "ACTIVE"
      },
      "scenarioFinancial": {
        "checks": [
          "Parameter_Validation",
          "Outcome_Sanity",
          "Coherence_Check"
        ],
        "status": "ACTIVE"
      },
      "languageContent": {
        "checks": [
          "Professional_Language",
          "Factual_Accuracy",
          "Business_Appropriateness"
        ],
        "status": "ACTIVE"
      },
      "ragEvaluation": {
        "checks": [
          "Retrieval_Quality",
          "Generation_Quality",
          "Business_Task_Success"
        ],
        "status": "ACTIVE"
      }
    },
    "auditLogging": "ENABLED"
  }
}

---

Audit Trail & SOX Compliance

All guardrails checks are logged via logAuditEvent for immutable audit trails:

await logAuditEvent(
  tenantId,
  userId,
  'COMPREHENSIVE_GUARDRAILS_CHECK',
  {
    insightType: 'scenario_impact',
    overallStatus: 'PASS',
    riskLevel: 'MEDIUM',
    guardrailsRun: ['CONTENT_QUALITY', 'FINANCIAL_ACCURACY'],
    issueCount: 0,
    warningCount: 1,
    requiresApproval: false
  },
  'Insight validation: PASS (Risk: MEDIUM)'
);

Logged Information:

• Timestamp of check
• User performing validation
• Insight or data being validated
• All issues and warnings
• Risk level assessment
• Whether approval is required
• Overall validation status

---

Integration Points

1. AI Insight Generation Pipeline

When AIManager generates insights, they should be validated:

// In AIManager or insight generation service
const insight = {
  id: 'generated_insight_123',
  type: 'scenario_impact',
  content: aiGeneratedText,
  context: { audience: 'executive', businessUnit: 'finance' },
  financialData: analyzedData
};

const validationReport = await ComprehensiveGuardrailsCoordinator.validateInsight(
  tenantId,
  userId,
  insight
);

if (validationReport.overallStatus === 'FAIL') {
  // Don't return the insight to user
  // Log the failure and alert operations team
} else if (validationReport.overallStatus === 'REVIEW_RECOMMENDED') {
  // Flag for human review before deployment
  await NotificationService.notify(tenantId, {
    type: 'INSIGHT_REVIEW_REQUIRED',
    insightId: insight.id,
    reasons: validationReport.allWarnings
  });
}

2. Scenario Decision Support

When users evaluate scenarios:

// In scenarioRoutes.js
router.post('/:id/calculate-financial-impact', async (req, res) => {
  // ... calculate financial impact ...

  const report = ComprehensiveGuardrailsCoordinator.validateScenario(
    scenario,
    projections,
    { tolerance: 0.0001 }
  );

  return res.json({
    financialImpact: calculations,
    guardrailsReport: report,
    approval_required: report.requiresApproval
  });
});

3. Content Quality Monitoring

Before publishing AI-generated insights:

// In InsightsService or similar
const contentValidation = await ComprehensiveGuardrailsCoordinator.validateContent(
  tenantId,
  userId,
  insight.description,
  { audience: 'executive', urgency: 'high' }
);

if (contentValidation.overallScore < 75) {
  insight.status = 'REVIEW_REQUIRED';
  insight.guardrails_issues = contentValidation.allIssues;
}

---

Configuration & Customization

Financial Accuracy Thresholds

// Tolerance for P&L reconciliation
validatePLReconciliation(financialData, 0.0001)  // 0.01% tolerance

// Ratio bounds checking
const ratios = {
  grossMargin: { min: 0.05, max: 0.95 },
  debtToEquity: { max: 2.0 },
  currentRatio: { min: 1.0, max: 5.0 }
};

Scenario Parameter Bounds

{
  revenue_growth_rate: { min: -0.5, max: 1.0 },        // -50% to +100%
  cogs_pct_revenue: { min: 0.15, max: 0.95 },          // 15% to 95%
  opex_growth_rate: { min: -0.5, max: 0.5 },           // -50% to +50%
  rd_pct_revenue: { min: 0.0, max: 0.2 },              // 0% to 20%
  capex_pct_revenue: { min: 0.0, max: 0.3 }            // 0% to 30%
}

Content Validation Options

{
  languageOptions: {
    targetAudience: 'executive' | 'analyst' | 'operational',
    minProfessionalScore: 0.8,  // 0-1 scale
    allowedBannedPhrases: []
  },
  factualContext: {
    prohibitedClaims: ['we are unaffected by market forces'],
    requiredDisclosures: ['assumptions about market stability']
  },
  businessContext: {
    businessUnit: 'sop' | 'finance' | 'supply_chain' | 'operations',
    urgency: 'critical' | 'high' | 'medium' | 'low',
    decisionContext: 'Capacity Planning'
  }
}

RAG Quality Thresholds

const thresholds = {
  retrieval: {
    hitRate: 0.7,           // 70% minimum
    precisionAtK: 0.75,
    recallAtK: 0.65
  },
  generation: {
    faithfulness: 0.8,      // 0-1 scale
    answerRelevancy: 0.75,
    contextRelevancy: 0.7
  },
  businessTask: {
    taskSuccessScore: 7     // 0-10 scale
  }
};

---

Error Handling

Validation Failures

Financial Data Fails P&L Reconciliation:

{
  "overallStatus": "FAIL",
  "allIssues": [
    {
      "guardrail": "FINANCIAL_ACCURACY",
      "severity": "ERROR",
      "check": "GROSS_PROFIT_RECONCILIATION",
      "expected": 400000,
      "actual": 399500,
      "variance": 500,
      "relativeVariance": 0.05,
      "message": "Gross Profit mismatch: Revenue 1000000 - COGS 600000 ≠ 399500"
    }
  ]
}

Scenario Parameters Out of Bounds:

{
  "overallStatus": "REVIEW_REQUIRED",
  "requiresApproval": true,
  "allIssues": [
    {
      "guardrail": "PARAMETER_VALIDATION",
      "severity": "GUARDRAIL",
      "parameter": "revenue_growth_rate",
      "value": 55,
      "threshold": "-50% to +100%",
      "message": "Revenue growth rate 55% is outside acceptable bounds. Allow? Requires executive approval."
    }
  ]
}

Content Lacks Professional Quality:

{
  "overallStatus": "REVIEW_RECOMMENDED",
  "requiresReview": true,
  "allWarnings": [
    {
      "guardrail": "PROFESSIONAL_LANGUAGE",
      "severity": "WARNING",
      "phrase": "revolutionary",
      "message": "Marketing/hype language detected: \"revolutionary\" - use objective, data-driven language"
    }
  ],
  "recommendations": [
    "Use objective, data-driven language instead of marketing hype phrases"
  ]
}

---

Monitoring & Observability

Key Metrics to Track

1. Guardrails Pass Rate: % of insights passing all checks
2. Issues by Type: Breakdown of failures (financial vs. content vs. RAG)
3. Approval Rate: % of insights requiring executive approval
4. Risk Distribution: % CRITICAL / HIGH / MEDIUM / LOW insights

Logging

All guardrails activity is logged to:

• Application logs (appLogger) - operational visibility
• Audit trail (auditService.logAuditEvent) - compliance & traceability

Dashboard Metrics

-- Top guardrail violations
SELECT
  guardrail_type,
  COUNT(*) as failure_count,
  DATE_TRUNC('day', timestamp) as day
FROM audit_events
WHERE event_type = 'COMPREHENSIVE_GUARDRAILS_CHECK'
  AND details->>'overallStatus' != 'PASS'
GROUP BY guardrail_type, day
ORDER BY day DESC;

-- Risk level distribution
SELECT
  details->>'riskLevel' as risk_level,
  COUNT(*) as count
FROM audit_events
WHERE event_type = 'COMPREHENSIVE_GUARDRAILS_CHECK'
GROUP BY risk_level;

---

Phase 2 Enhancements

Planned improvements:

1. Custom Guardrails Framework - Allow organizations to define custom validation rules
2. Self-Healing Guardrails - Automatic retry/repair of failed validations
3. Pre-Processing Compliance Filters - Early-stage regulatory risk detection
4. Advanced RAG Evals - Ragas framework integration, LLM-as-judge
5. Sensitivity Analysis - Scenario sensitivity testing for robustness
6. Data Quality Guardrails - Input validation, PII detection, schema enforcement

---

Testing

Unit Tests

npm test -- ComprehensiveGuardrailsCoordinator.test.js
npm test -- LanguageContentQualityGuardrails.test.js
npm test -- guardrailsRoutes.test.js

Integration Testing

# Test full insight validation pipeline
curl -X POST http://localhost:5000/api/guardrails/validate-insight \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @test-insight.json

---

Summary

The Comprehensive Guardrails System provides:

• ✅ Four-pillar validation: Financial, Scenario, Content, RAG
• ✅ Unified orchestration: Single pipeline for all checks
• ✅ Enterprise standards: Professional tone, factual accuracy, business appropriateness
• ✅ Risk assessment: LOW/MEDIUM/HIGH/CRITICAL risk levels
• ✅ Approval workflows: Flags scenarios/insights requiring executive review
• ✅ SOX compliance: Immutable audit trails for all validations
• ✅ Configurable thresholds: Customizable bounds and standards
• ✅ Comprehensive reporting: Actionable recommendations for improvements

This integrated approach ensures all AI-generated insights meet ChainAlign's quality standards before reaching decision-makers.