← Kalyan Venkatesh

Multi-Agent Inference Reliability Framework

2 linked studies on whether the safety layer around an LLM actually works. Part A builds a runtime monitor for generated code. Part B attacks the judges that grade LLM answers.

This is my graduate research at DePaul, led with 2 faculty co-authors and now under review. Both parts poke at the same assumption: that wrapping an LLM in checks makes it safer. Part A wraps a code-writing model in a critic and a fixer, runs everything locally through Ollama, and measures on real benchmarks whether the monitoring helped. Part B trains the other kind of check, preference judges of the sort LLM-as-a-judge pipelines rely on, then hits them with adversarial pairs to see which one breaks first. Both parts found the check itself failing in ways the clean metrics never showed.

Part A

A study on monitoring

agentic-llmops

A 3-agent runtime monitor for LLM code: a Critic scores each output for hallucination risk, a Fixer rewrites the risky ones, and a reversion gate keeps a rewrite only if it strictly improves. Measured by pass@1 on HumanEval, HumanEval+ and MBPP across 6 open models, with every run tracked in MLflow. The stronger the base model, the more the critic-fixer loop costs it.

≥ τ below τ · ships untouched, no Fixer call Planner writes the code Pre-Critic scores risk 0 to 1 Fixer rewrites it Reversion gate keep only if better Post-Critic final score
Fresh Planner output, not yet scored.
Cleared. Either it scored below τ = 0.70 and skipped the Fixer, or its rewrite survived the gate.
Flagged at or above τ. Sent to the Fixer for a rewrite.
Rewrite reverted. The gate threw it away and kept the original, which is what happens to 73.4% of them.

By capability

Change in pass@1 from adding the monitor, on code that parses cleanly.

How part A works

System design, the stages in run order

Read it top to bottom. On the left are the models and data, all served locally through Ollama with no retraining. The centre spine is the monitoring pipeline in run order: Planner, hybrid Pre-Critic, the Fixer behind a threshold, a reversion gate, Post-Critic and the test. The right shows what each run writes to disk, ending in the headline result. A baseline with no monitoring runs alongside for a clean before-and-after.

system.blueprint
hover a chip to spotlight its stage
Fresh planner code, in flight
Flagged by the critic, routed to the Fixer
Cleared the gate, or passed the tests
Stage · model · artifact

Where monitoring stops helping

Monitoring is not a free win. The two mid-capability models moved by less than 2 points, neither distinguishable from zero. The real damage shows up when the critic false-alarms on a strong model and the fixer rewrites code that was already correct.

What the gate is actually for

The reversion gate throws away 73.4% of the Fixer's rewrites, keeping only strict improvements. The ablation shows problem context and fixer scale drive the gains; the gate mostly protects the downside.

The critic is a hybrid

A static AST pass catches syntax errors and undefined names first and sets a floor. The LLM critic can only push the risk score higher, never talk a real defect down to clean.

Part A notes

The headline number did not survive its audit.

The old extractor only matched python-tagged fences, so ~54% of Code Llama's valid code was scored as failure. The corrected table keeps the models with clean re-runs and retires the rest.

The gate exists to prevent harm.

It reverts 73.4% of the Fixer's rewrites, keeping only strict improvements.

False alarms cost more than gains help.

When the critic wrongly flags code a strong model already got right, the fixer can gut it: one slice fell from 92.9% to 50.0%.

Part B

A study on LLM-as-a-judge

Inference-Lens

The judges here are the checks most LLM pipelines trust: models that pick the better of 2 answers. I trained 3 of them on Anthropic's HH-RLHF human preference data, a logistic regression, an XGBoost, and a fine-tuned DeBERTa transformer in PyTorch, then applied each one zero-shot to LLM-Bar's 419 adversarial pairs to find which judge breaks first.

Try it

Paste 2 answers, or load the sample. 2 trained models pick the better one. Watch a padded answer win.

How part B works

A live map of the build.

System design, the stages in run order

Left: the datasets and pretrained models that feed in. Centre: the pipeline in run order. Right: what each stage writes to disk. The amber path is LLM-Bar. It stays out of training and enters only at the stress-test, so nothing a model saw prepared it for that path.

system.blueprint
hover a chip to spotlight its stage
HH-RLHF data in flight (training path)
LLM-Bar adversarial pairs (held out, stress-test only)
Trained / served output
Stage · resource · artifact

Part B notes

The best model failed hardest

DeBERTa led the clean benchmark at 0.622 AUC and lost the most under attack, 19.3pp.

Longer answers scored worse

The most verbose archetype had the lowest human pick rate, 48.9%, below the 50.1% of the shortest.

Every judge flipped

All 3 models fell below 50% on adversarial pairs. On a forced choice that is systematic inversion.