Wenyu Du

Wenyu Du

PhD Candidate

The University of Hong Kong

Biography

I am a PhD candidate at the University of Hong Kong, under the supervision of Professor Reynold Cheng. I work on natural language processing and machine learning. Before joining HKU, I received my MSc and BSc from University of Southampton and University of Nottingham respectively. Please refer to CV for detail.

Current Interests

  • Efficient training and deployment of large-scale models
  • Interpreting (reasoning) mechanisms for neural models
  • System2-centric (multimodal) models

Education

  • Doctor of Philosophy

    University of Hong Kong

  • MSc in Data Science

    University of Southampton

  • BSc in Computer Science

    University of Nottingham

Selected Publications

* = equal contribution

Wenyu Du*, Tongxu Luo*, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu. Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training. The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS Spotlight), 2024.

WebSite Paper Code Twitter Talk@MILA

Wenyu Du*, Shuang Cheng*, Tongxu Luo, Zihan Qiu, Zeyu Huang, Ka Chun Cheung, Reynold Cheng, Jie Fu. Unlocking Continual Learning Abilities in Language Models. The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2024.

Paper Code Hugging Face Daily Papers

Yuqiao Wen, Zichao Li, Wenyu Du, Lili Mou. f-Divergence Minimization for Sequence-Level Knowledge Distillation. The 61st Annual Meeting of the Association for Computational Linguistics (ACL, Main), 2023.

Paper Code

Contact

  • University of Hong Kong, MW104, Meng Wah Complex Building, Pokfulam Road, Hong Kong