Portrait of Yutao Mou

Yutao Mou (牟宇滔)

PhD Student · AI Safety & Agent Security

北京大学 · Peking University
KCL Lab

Master’s degree (2024) · Bachelor’s degree (2021)
北京邮电大学
Beijing University of Posts and Telecommunications

About Me

关于我

Supervisor

I am a PhD student at KCL Lab, Peking University, supervised by Professors Wei Ye and Shikun Zhang.

Research Interests

I work on building safe, reliable, and scalable AI systems, with an emphasis on the safety evaluation and alignment of large language models and AI agents.

01

Safety Evaluation & Red Teaming

Comprehensive safety benchmarks for text generation, code agents, and multimodal tasks; jailbreak and automated red-teaming methods; and interpretable analysis of model vulnerabilities.

02

Post-Training & Safety Alignment

Safety alignment that preserves general capabilities, with a focus on reasoning models, safety-oriented reward models, and automated synthesis and augmentation of safety training data. Future directions include multimodal safety alignment.

Feel free to contact me for research discussions and collaboration.

News

最新进展
  1. 入选 2026年中国科协青年科技人才培育工程博士生专项计划。

  2. 完成一篇通用智能体安全综述论文。项目与论文列表 ↗

  3. 完成一篇可执行对抗环境扩展驱动智能体内生安全对齐的工作。论文 ↗

Publications

学术论文
Overview of ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback

ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback

Yutao Mou*, Zhangchi Xue, Lijun Li, Peiyang Liu, Shikun Zhang, Wei Ye, Jing Shao
ACL 2026 (Findings)
[Code] / [Paper]
Abstract

In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents. We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning. The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history. It assesses request harmfulness and action-attack correlations, producing interpretable and generalizable safety judgments and feedback. Furthermore, we introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65 percent on average and improves benign task completion by approximately 10 percent under prompt injection attacks.

Overview of Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models

Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models

Yutao Mou*, Xiaoling Zhou, Yuxiao Luo, Shikun Zhang, Wei Ye
Preprint
[Code] / [Paper]
Abstract

In this work, we show that LoRA-based Refusal-training enables performance-preserving safety alignment even when trained solely on safety data, demonstrating that LoRA serves as cost-efficient, performance-preserving, and plug-and-play safety patches. Beyond empirical findings, we provide both theoretical and experimental evidence that LoRA effectively decouples safety into a low-rank subspace largely orthogonal to the model's intrinsic transformation space, ensuring that safety enhancements do not interfere with inherent capabilities.

Overview of SaRO: Enhancing LLM Safety through Reasoning-based Alignment

SaRO: Enhancing LLM Safety through Reasoning-based Alignment

Yutao Mou*, Yuxiao Luo, Shikun Zhang, Wei Ye
ACL 2026 (Findings)
[Code] / [Paper]
Abstract

In this study, we address the prevalent issue of shallow alignment in current LLM safety alignment methods by proposing the Safety-oriented Reasoning Optimization Framework (SaRO). SaRO introduces a System 2-style alignment to internalize a deliberative, slow-thinking reasoning approach, enabling models to reflect on safety policies more effectively. This framework enhances safety performance without compromising the general capabilities of the model.

Overview of Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective

Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective

Yutao Mou*, Xiao Deng, Yuxiao Luo, Shikun Zhang, Wei Ye
ACL, 2025 (CCF-A)
[Code] / [Paper]
Abstract

In this paper, we construct a benchmark dataset named CoV-Eval, which encompasses various task types and covers a wide range of code security vulnerabilities. From the perspective of code security, we evaluate LLMs and various coding assistants, focusing on assessing the security vulnerabilities in code generated by LLMs, as well as their capabilities in vulnerability detection and repair.

Overview of SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

Yutao Mou*, Shikun Zhang, Wei Ye
NeurIPS, 2024 (CCF-A)
[Code] / [Paper]
Abstract

This paper introduces SG-Bench, a new benchmark for evaluating LLM safety, which pioneers the concept of safety generalization—assessing how variations in task and prompt types affect model safety. SG-Bench combines both generative and discriminative tasks and expands test data to evaluate the impact of prompt engineering and jailbreak attacks

Overview of UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle

UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle

Yutao Mou*, Kexiang Wang, Jianhe Lin, Dehong Ma, Jun Fan, Daiting Shi, Zhicong Cheng, Gu Simiu, Dawei Yin, Weiran Xu
NAACL, 2024 (Findings)
[Code] / [Paper]
Abstract

This paper first propose a new paradigm: knowledge rekindle, which aims to re-incorporate the fine-tuned expert model into the training cycle and break through the performance upper bounds of experts without introducing additional annotated data. Then we further propose a unified expert-guided pre-training (UEGP) framework for knowledge rekindle.

Overview of Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery

Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery

Yutao Mou*, Keqing He*, Pei Wang, Yanan Wu, Jingang Wang, Wei Wu, Weiran Xu
EMNLP, 2022 (CCF-B)
[Code] / [Paper]
Abstract

This paper focuses on new intent discovery and clustering task, and propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents. Specifically, we design a novel K-nearest neighbor contrastive learning objective (KCL) for in-domain pre-training, and a hard negative mining strategy for self-supervised representation learning on unlabeled out-of-domain data.

Overview of UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning

UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning

Yutao Mou*, Pei Wang*, Keqing He*, Yanan Wu, Jingang Wang, Wei Wu, Weiran Xu
EMNLP, 2022 (short paper)
[Code] / [Paper]
Abstract

This paper focuses on out-of-domain (OOD) intent detection task, and propose a unified neighborhood learning framework (UniNL) to detect OOD intents. propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents. Specifically, we design a K-nearest neighbor contrastive learning objective for training and introduce a KNN-based scoring function for confidence estimation. We aim to align training objective with confidence function in inference stage.

Overview of Generalized Intent Discovery: Learning from Open World Dialogue System

Generalized Intent Discovery: Learning from Open World Dialogue System

Yutao Mou*, Keqing He*, Yanan Wu, Pei Wang, Jingang Wang, Wei Wu, Yi Huang, Junlan Feng, Weiran Xu
COLING, 2022 (CCF-B)
[Code] / [Paper]
Abstract

This paper defines a new task, Generalized Intent Discovery (GID), which aims to extend an IND intent classifier to an open-world intent set including IND and OOD intents. We hope to simultaneously classify a set of labeled IND intent classes while discovering and recognizing new unlabeled OOD types incrementally. We construct three public datasets for different application scenarios and propose two kinds of frameworks, pipeline-based and end-to-end for future work.

Overview of Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning

Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning

Yutao Mou*, Keqing He*, Yanan Wu*, Zhiyuan Zeng, Hong Xu, Huixing Jiang, Wei Wu, Weiran Xu
ACL, 2022 (short paper)
[Code] / [Paper]
Abstract

Discovering Out-of-Domain (OOD) intents is essential for developing new skills in a task-oriented dialogue system. The key challenge is how to transfer prior IND knowledge to OOD clustering. This paper proposes a decoupled knowledge transfer framework for new intent discovery and clustering, which unifies the two-stage learning process of in-domain and out-of-domain data into instance discrimination and clustering discrimination tasks and bridges the gap between in-domain and out-of-domain data.

Research Experience

科研实习

Selected Honors

荣誉奖励

Academic Service

学术服务

Reviewer   /   EMNLP 2022, ACL 2023, EMNLP 2023, ARR