Wenyu Du

PhD Candidate

The University of Hong Kong

Biography

I am a PhD candidate at the University of Hong Kong, under the supervision of Professor Reynold Cheng. I work on natural language processing and machine learning. Before joining HKU, I received my MSc and BSc from University of Southampton and University of Nottingham respectively. Please refer to CV for detail.

Current Interests

Efficient training and deployment of large-scale models
Interpreting internal (reasoning) mechanisms for neural models
System2-centric (multimodal) models

Education

MSc in Data Science

University of Southampton
BSc in Computer Science

University of Nottingham

Selected Publications

* = equal contribution

Wenyu Du*, Tongxu Luo*, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu. Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training. The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS Spotlight), 2024.

WebSite Paper Code Twitter Talk@MILA

Wenyu Du*, Shuang Cheng*, Tongxu Luo, Zihan Qiu, Zeyu Huang, Ka Chun Cheung, Reynold Cheng, Jie Fu. Unlocking Continual Learning Abilities in Language Models. The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2024.

Paper Code Hugging Face Daily Papers

Yuqiao Wen, Zichao Li, Wenyu Du, Lili Mou. f-Divergence Minimization for Sequence-Level Knowledge Distillation. The 61st Annual Meeting of the Association for Computational Linguistics (ACL, Main), 2023.

Paper Code

Contact

University of Hong Kong, MW104, Meng Wah Complex Building, Pokfulam Road, Hong Kong