I am currently a third-year Master’s student at Shanghai Jiao Tong University, under the supervision of Prof. Hai Zhao, majoring in computer science. Before that, I received my B.S. degree in the IEEE honor class at SJTU in 2022, majoring in computer science. I also collaborated with Dr. Zhuosheng Zhang during my undergraduate years.

I used to work as a research intern at Microsoft Research Asia (MSRA) in the NLC group, guided by Dr. Lei Cui. I worked on several interesting research topics related to Document AI, including webpage understanding and document Image foundation models.

During May 2023 to Feb 2024, I worked closely with Prof. Pengfei Liu at GAIR on various aspects of Large Language Models (LLMs), primarily focusing on the evaluation and alignment of LLMs.

Currently, I am doing some research under the supervision of Prof. Junxian He.

Publications

(* indicates equal contribution)

Diving into Self-Evolving Training for Multimodal Reasoning

Wei Liu*, Junlong Li*, Xiwen Zhang, Fan Zhou, Yu Cheng, Junxian He
Preprint, 2024 | PDF | Code | Page | HF Repo

Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale

Fan Zhou*, Zengzhi Wang*, Qian Liu, Junlong Li, Pengfei Liu
Preprint, 2024 | PDF | Code | Page | HF Repo

Dissecting Human and LLM Preferences

Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, Pengfei Liu
ACL 2024 | PDF | Demo | Code | Dataset

Reformatted Alignment

Run-Ze Fan, Xuefeng Li, Haoyang Zou, Junlong Li, Shwai He, Ethan Chern, Jiewen Hu, Pengfei Liu
EMNLP 2024, Findings | PDF | Code | Page

Extending LLMs' Context Window with 100 Samples

Yikai Zhang, Junlong Li, Pengfei Liu
Preprint, 2024 | PDF | Code

The Critique of Critique

Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu
ACL 2024, Findings | PDF | Code

Generative Judge for Evaluating Alignment

Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, Pengfei Liu
ICLR 2024 | PDF | Code | Page

Generative AI for Math: Abel

Ethan Chern*, Haoyang Zou*, Xuefeng Li*, Jiewen Hu*, Kehua Feng, Junlong Li, Pengfei Liu
Preprint, 2023 | Code | Page

Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning

Jinyuan Wang, Junlong Li, Hai Zhao
EMNLP 2023, Findings | PDF |Code

Self-Prompting Large Language Models for Zero-Shot Open-Domain QA

Junlong Li, Jinyuan Wang, Zhuosheng Zhang, Hai Zhao
NAACL 2024 | PDF

Dialogue-adaptive language model pre-training from quality estimation

Junlong Li, Zhuosheng Zhang, Hai Zhao
Neurocomputing, 2022 | PDF | Code

DiT: Self-supervised pre-training for document image transformer

Junlong Li, Yiheng Xu, Tengchao Lv, Lei Cui, Cha Zhang, Furu Wei
ACM Multimedia 2022 | PDF | Code

Markuplm: Pre-training of text and markup language for visually-rich document understanding

Junlong Li*, Yiheng Xu*, Lei Cui, Furu Wei
ACL 2022 | PDF | Code

Multi-turn dialogue reading comprehension with pivot turns and knowledge

Zhuosheng Zhang, Junlong Li, Hai Zhao
TASLP, 2021 | PDF | Code

Educations

2022.09 - 2025.03 (expected), M.S.@SJTU, Computer Science & Engineering
2018.09 - 2022.06, B.S.@SJTU, IEEE honor class, Computer Science.

Talks

2023.10, Invited talk at CIKM 2023 Workshop on Document Intelligence and Understanding.

Service

Reviewer: ACM MM, ARR, ICLR

Awards

Yang Yuanqing Education Scholarship (top 0.3%), 2024
MSRA Stars of Tomorrow (Award of Excellent Intern), 2022
National Scholarship (top 2%), 2022
SJTU Zhiyuan Honors Scholarship (top 5%), 2019-2021