Developed MITCF Framework

Offline

MITCF Framework

Modular Inference-Time Control Framework — trustworthiness and adversarial robustness in agentic AI systems.

MITCF Pipeline

📥
Input
Task + BenchmarkItem
→
🔍
VerificationModule
Fact check, hallucination detection via LexicalHallucinationDetector
→
🧐
CritiqueModule
Bias detection, quality assessment, safety scoring
→
✏️
RevisionModule
Corrects errors from Verification + Critique feedback
→
🚪
DecisionGate
FuzzyDecisionGate: 5 inputs → 15 rules → centroid defuzz → action
→
📤
Output
Final response + GateResult + RunRecord

Source: mitcf_pipeline/controller.py — MITCFController.run_condition()

Ablation Conditions — Module Activation & Results

Baselinebaseline
LLM Call
MITCF-1mitcf_1_verification
LLM CallVerification
MITCF-2mitcf_2_verification_critique
LLM CallVerificationCritique
MITCF-3mitcf_3_verification_critique_revision
LLM CallVerificationCritiqueRevision
Full MITCFfull_mitcf
LLM CallVerificationCritiqueRevisionDecisionGate
VerificationModulecontrols.py

Runs LexicalHallucinationDetector on model output. Flags hallucinated tokens by comparing against benchmark reference. Sets hallucination_detected flag on the model output.

CritiqueModulecontrols.py

Evaluates output for bias, quality, and safety. Uses task type (FACT_CHECK, MATH, CODE) to determine appropriate critique strategy. Produces quality_score and safety_score.

RevisionModulecontrols.py

If verification or critique found issues, requests a revised response up to max_revision_attempts times. Tracks avg_revisions per run.

DecisionGatecontrols.py + fuzzy_gate.py

FuzzyDecisionGate with 5 fuzzy inputs → 15 Mamdani rules → centroid defuzzification → GateAction (ACCEPT/VERIFY/REVISE/ESCALATE/REFUSE).

LexicalHallucinationDetectordetectors.py

Calibrates a word-overlap threshold against a reference corpus. Detects hallucinations by checking if the output tokens overlap less than the calibrated threshold with the expected answer.

MITCFControllercontroller.py

Orchestrates all modules. Runs run_condition() for each of 5 ablation conditions. Stores per-item RunRecord to JSONL output.

Key Research Contributions

✓Pure-Python Mamdani fuzzy inference system — no external ML dependencies
✓Trustworthiness Index (TI) composite metric combining hallucination, bias, robustness, and decision reliability
✓20-variant neural network sweep using SHA-1 hash feature encoding
✓5-condition ablation study quantifying each module's contribution to TI
✓Bootstrap confidence intervals + McNemar test + Holm–Bonferroni correction for statistical validation