Haoran Xu
About
I am a PhD student in the James Watt School of Engineering at the University of Glasgow, supervised by Prof. Lei Zhang and Prof. Iadh Ounis.
I study how LLMs and multi-agent LLM systems use evidence to reach decisions. Much of my work looks at over-commitment: LLM judges and agents giving definitive answers when the evidence is partial or conflicting, and agent pipelines that succeed at a task without the evidence to support the action they took.
I build evaluations that make these failures visible, tracing evidence through agent reports, decisions and outcomes so that people can tell a well-supported decision from a lucky one, and know when human review is needed.
Beyond diagnosing these failures, I am moving towards fixing them: using reinforcement learning and post-training to teach models to keep their decisions within what the evidence supports, with the evaluations above serving as targets and checks for that training.
Research agenda
My work so far addresses the first three questions; the fourth is where I am heading next.
- When do LLM judgements go beyond the evidence? Measuring when LLM judges and agents issue definitive verdicts on evidence that is conflicting or insufficient, and separating whether they recognise the conflict from what they finally decide.
- How does evidence flow through multi-agent decisions? Tracing evidence through agent reports, decisions and task outcomes to locate information loss and unsupported claims, and to distinguish success supported by the available information from success achieved without it.
- How can evaluation support human oversight? Designing evaluations that help people recognise uncertainty, question unsupported conclusions and decide when an automated judgement needs human review.
- Can training close these gaps? Using reinforcement learning and post-training to improve how models and agents act on conflicting or incomplete evidence, such as abstaining or flagging a conflict when warranted, and measuring whether gains hold on evaluations that separate supported from unsupported success.
Publications and preprints
-
Certifiable Semantic Agreement Among LLM Agents: What the Admissibility Instrument Decides
-
Cherry-pick Override: LLM Judges Under-use the Non-Directional Verdicts Their Contract Authorizes
-
A Multi-Round Voting Consensus Framework via Evolutionary Game Theory and Score Completion
Selected projects
-
Fact2Act (in progress)Process-based evaluation of multi-agent LLM decisions: tracing evidence through role-specific agent reports, decisions and task outcomes, to separate success supported by the available information from success achieved without sufficient evidence.preprint in preparation
-
CRSE / FBA EncoderSemantic encoder fine-tuned with supervised contrastive learning and adversarial hard negatives, used to check whether agents agree on supporting meaning rather than only on verdicts.
-
LLM API DemosLightweight demos for structured LLM API usage, agent workflows, and research prototyping.
-
Work PlannerA local-first academic workflow tool for research planning and execution tracking.
Education
University of Glasgow
PhD, James Watt School of Engineering, Oct 2024–present
University of Glasgow
BEng in Electronics and Electrical Engineering, Sep 2020–Jul 2024
University of Electronic Science and Technology of China (UESTC)
BEng in Communication Engineering, Sep 2020–Jul 2024
Dual-degree programme with the University of Glasgow
Talks
Invited talk on the multi-round voting consensus framework, Autonomous Systems & Connectivity Research Programme, University of Glasgow, Mar 2025
Teaching
Graduate Teaching Assistant (course development), University of Glasgow, Apr 2025–Jul 2025. Revised undergraduate and postgraduate lecture slides with lecturers and designed assignments.