Ph.D. candidate · UC Berkeley

Junyu (James) Guo

I study how AI agents should decide under uncertainty: safe reinforcement learning, reasoning and uncertainty in large language models, and auditing the agents we build on them.

Read my research Download CV

Berkeley, CA

01 / About

About me

I’m Junyu Guo, James to most people. I’m a Ph.D. candidate at UC Berkeley, advised by Prof. Javad Lavaei. I design efficient decision-making algorithms for agents that learn from data and act under uncertainty, from safe reinforcement learning to the way large language models reason, and I care about making the resulting systems ones we can actually trust.

My work so far runs along three threads. In reinforcement learning, DRCORL (NeurIPS 2025) shows that an offline policy can keep its safety promises without paying for them in reward or inference speed; the title is the thesis, don’t trade off safety. In language models, I’m drawn to the habits of a careful thinker: knowing when to say “I’m not sure”, picking a reasoning style that fits the problem rather than defaulting to one, and noticing when a fact has changed halfway through a conversation. And in trustworthy AI, my most recent paper, Groundability, Not Scale Alone, asks when a small model can reliably audit the work of a much stronger coding agent. The answer has less to do with how big the reviewer is than with whether its evidence can actually be checked.

None of this is solo work. I’m fortunate to collaborate with Prof. Costas Spanos (Berkeley EECS), Prof. Ming Jin (Virginia Tech) and Dr. Shangding Gu, and with Liyuan Liang and Yuchen Fang from our group.

I came to all of this by way of mathematics. At Tsinghua University, Prof. Chenxu Li and Prof. Yiwen Shen showed me what research feels like, through the problem of allocating a portfolio efficiently when markets are incomplete. An exchange semester at Cornell with Prof. Andreea Minca and Prof. Qiaomin Xie, working on mean field games, was my first taste of reinforcement learning, and I never quite left.

Off the clock I’m a football fan with a divided heart (Manchester City and FC Barcelona), and I like to travel, hike, listen to music, and play a bit of soccer myself.

02 / Focus

Research interests

Safe & Efficient Reinforcement Learning

Offline and constrained RL that keeps policies safe without giving up reward or inference speed, and algorithms that adapt quickly to changing environments.

LLM Reasoning & Uncertainty

How language models should choose a reasoning style, say what they are unsure about, and keep track of information that changes within a conversation.

Trustworthy AI Agents

Checks that let weaker reviewers audit stronger agents, from coding agents to decision-making systems, so that AI remains verifiable, fair and privacy-preserving as it scales.

03 / Research

Selected papers

All research

2026

PreprintLLM agents

Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents

Junyu Guo, Shangding Gu, Ming Jin, Javad Lavaei

Can a weaker model reliably decide whether a strong coding agent's patch actually solves its issue? Longer traces, more context and better-organized evidence do not make the judgment reliable. What does is groundability: whether an independent check, such as a runnable test or sound static analysis, can answer the disputed question. Given groundable evidence, weak reviewers catch the omissions that confident summaries hide, and reviewer size stops predicting quality. The open problem is producing such checks when official tests are not available.

arXiv
2026

PreprintLLM reasoning

When Context Changes: Understanding Update Failures in LLMs

Junyu Guo, Yuchen Fang, Shangding Gu, Costas Spanos, James Demmel, Javad Lavaei

LLMs can answer with an old value of a variable even when the updated one is still in context, a failure we call stale binding. We introduce the Controlled In-Context Memory (CICM) benchmark to isolate it and trace it to attention drift: when attention scores are similar, several old values together outweigh the current one. Redirecting attention toward the current value at inference time, with no further training, corrects most of these errors.

arXiv
2026

PreprintLLM reasoning

LLMs Should Express Uncertainty Explicitly

Junyu Guo, Shangding Gu, Ming Jin, Costas Spanos, Javad Lavaei

LLMs often give confident but wrong answers. We show that post-training can make a model's self-assessment explicit in its own response, and ask where that signal belongs: a verbalized confidence score once the answer is formed, or an uncertainty marker emitted mid-reasoning whenever the current step looks unreliable. Both sharply reduce overconfident errors while improving answer quality, through different mechanisms: the first sharpens a confidence structure already present in the pretrained model, the second teaches the model to flag high-risk reasoning steps.

arXiv
2025

NeurIPS 2025Safe RL

Don't Trade Off Safety: Diffusion Regularization for Constrained Offline RL

Junyu Guo, Zhi Zheng, Donghao Ying, Ming Jin, Shangding Gu, Costas Spanos, Javad Lavaei

Constrained offline RL must learn safe, high-reward policies from a fixed dataset, with no unsafe exploration. DRCORL uses a diffusion model to capture the behavior policy in the data, distills it into a simple policy for fast inference, and applies gradient manipulation to balance reward against constraint satisfaction. The result reliably meets cost limits without giving up return or inference speed, and does so with the same hyperparameters across tasks.

arXiv NeurIPS
2025

NeurIPS MATH-AI Workshop 2025LLM reasoning

Meta Thinker: Thinking What AI Thinks

Junyu Guo, Shangding Gu, Costas J. Spanos, Javad Lavaei

Reasoning strategies such as Chain-of-Thought and Tree-of-Thought help, but which one works depends on the problem. We ask whether an LLM can pick its own thinking style. After analyzing five reasoning paradigms across mathematical, logical and commonsense tasks, we propose a meta-thinking prompt algorithm that lets the model select, or synthesize, a reasoning style from the input itself, improving both accuracy and token efficiency.

OpenReview Poster page Code
2025

PreprintLLM reasoning

StyleBench: Evaluating Thinking Styles in Large Language Models

Junyu Guo, Shangding Gu, Ming Jin, Costas Spanos, Javad Lavaei

How well a reasoning style works depends on the model and the task, yet this interplay was poorly understood. StyleBench systematically evaluates five styles of thought across diverse tasks and open-source models of many sizes. The takeaway: no single style is universally best. Search-based styles pay off on open-ended problems but need large models, concise styles bring large efficiency gains on well-defined tasks, and small models often ignore the format and guess. Robust reasoning emerges with scale.

arXiv Code