Shatong Zhu 「朱沙桐」

Research Assistant at the Tongji TiEV Lab

Undergraduate B.Eng Student,
Data Science and Big Data Technology,
Tongji University, Shanghai, China.

I am a fourth-year B.Eng student in Data Science at the School of Computer Science, Tongji University, adviced by Junqiao Zhao. I have been honored with National Scholarship twice.

Research Interests: I have a broad interest in advancing the next generation of artificial intelligence, with a primary focus on reinforcement learning (RL) and robotic manipulation. Currently, I am a Research Assistant at the Tongji TiEV Lab, where I explore the generalization capabilities of robotic manipulation through offline meta-reinforcement learning.

Data Science: As a Data Science student, I am passionate about applying mathematical techniques to solve real-world challenges, particularly in areas such as data analysis, data mining, and operations research.

Previously: I have interned as an algorithm engineer at Meetsocial and ZhongAn Insurance. Additionally, I have contributed to research on Multi-Agent Reinforcement Learning (MARL) as a Research Assistant at the Tongji TiEV Lab, and worked on fine-tuning Large Language Models (LLM) at the Tongji FinLab.

Email: shatong.zhu[AT]gmail.com

News

Nov 03, 2024	🏆 I was awarded the National Scholarship for the 2024 academic year! (Top 0.2% nationwide)
Sep 26, 2024	📄 Our paper UNICORN is accepted at NeurIPS 2024 as a spotlight!

Selected Publications [full list]

(*) denotes equal contribution

2024

arXiv

POWQMIX: Weighted Value Factorization with Potentially Optimal Joint Actions Recognition for Cooperative Multi-Agent Reinforcement Learning

Chang Huang*, Shatong Zhu*, Junqiao Zhao, Hongtu Zhou, Chen Ye, Tiantian Feng, and Changjun Jiang

In arXiv preprint arXiv:2405.08036. Under review, 2024

Abs PDF

Value function factorization methods are commonly used in cooperative multi-agent reinforcement learning, with QMIX receiving significant attention. Many QMIX-based methods introduce monotonicity constraints between the joint action value and individual action values to achieve decentralized execution. However, such constraints limit the representation capacity of value factorization, restricting the joint action values it can represent and hindering the learning of the optimal policy. To address this challenge, we propose the Potentially Optimal Joint Actions Weighted QMIX (POWQMIX) algorithm, which recognizes the potentially optimal joint actions and assigns higher weights to the corresponding losses of these joint actions during training. We theoretically prove that with such a weighted training approach the optimal policy is guaranteed to be recovered. Experiments in matrix games, difficulty-enhanced predator-prey, and StarCraft II Multi-Agent Challenge environments demonstrate that our algorithm outperforms the state-of-the-art value-based multi-agent reinforcement learning methods.
NeurIPS Spotlight

Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning

Lanqing Li*, Hai Zhang*, Xinyu Zhang, Shatong Zhu, Yang YU, Junqiao Zhao, and Pheng-Ann Heng

In The 38th Annual Conference on Neural Information Processing Systems, 2024

Spotlight Presentation [Top 1%]
Awarded Abs PDF Code Poster

Spotlight

As a marriage between offline RL and meta-RL, the advent of offline meta-reinforcement learning (OMRL) has shown great promise in enabling RL agents to multi-task and quickly adapt while acquiring knowledge safely. Among which, context-based OMRL (COMRL) as a popular paradigm, aims to learn a universal policy conditioned on effective task representations. In this work, by examining several key milestones in the field of COMRL, we propose to integrate these seemingly independent methodologies into a unified framework. Most importantly, we show that the pre-existing COMRL algorithms are essentially optimizing the same mutual information objective between the task variable M and its latent representation Z by implementing various approximate bounds. Such theoretical insight offers ample design freedom for novel algorithms. As demonstrations, we propose a supervised and a self-supervised implementation of I(Z;M), and empirically show that the corresponding optimization algorithms exhibit remarkable generalization across a broad spectrum of RL benchmarks, context shift scenarios, data qualities and deep learning architectures. This work lays the information theoretic foundation for COMRL methods, leading to a better understanding of task representation learning in the context of reinforcement learning.