Hyunji Amy Lee

About

Hello! I am a final-year PhD Student at the Language & Knowledge Lab at KAIST, advised by Minjoon Seo and an incoming Postdoc at UNC working with Mohit Bansal. I am interested in building a robust semi-parametric model that aligns with external knowledge modules thereby capitalizing on the strengths of both the model’s parametric knowledge space and external nonparametric knowledge space, aiming to achieve the optimal performance.

I've been focusing on three main research areas:
1. Retrieving the correct knowledge source: generalizable retrieval models robust on out-of-domain (RouterRetriever) and generative retrieval models (GMR, DynamicGR).
2. Aligning knowledge modules with language model: training objective to enhance grounding ability (CINGS), superposing model's parametric space and nonparametric space (Semiparametric Token-Sequence Co-Suprervision), and developing decoding method to effectively utilize nonparametric space (Np Decoding).
3. Understanding model’s parametric knowledge: investigating how models truly ground to given knowledge (Truly Ground), improving grounding performance in complex contexts (CORG), and studying knowledge utilization during pretraining (Knowledge Entropy).
Additionally I've worked on expanding these approaches to other modalities (VLM Safety, ZeroTA).

Previously, I graduated with M.S. at the Language & Knowledge Lab at KAIST, advised by Minjoon Seo, and B.S. from KAIST in computer science and electrical engineering (double major), where I was advised by Edward Choi.

I am currently interning at Microsoft Research with Lucas Caccia, Marc-Alexandre Côté, Eric Xingdi Yuan, and Alessandro Sordoni. I've previously interned at Tencent AI (Winter 2024) with Wenhao Yu, Kaixin Ma, and Hongming Zhang; Adobe Research (Summer 2024) with David Seunghyun Yoon, Franck Dernoncourt, and Trung Bui; AI2 Semantic Scholar Team (Summer 2023) with Kyle Lo and Luca Soldaini. See my Vitæ page for details.

News

• [June 2025] I will be starting a Postdoc with Mohit Bansal at the University of North Carolina, Chapel Hill from Fall 2025! Looking forward to lots of collaborations with the amazing students and faculty of UNC NLP and UNC CS!
• [June 2025] Successfully defended PhD thesis 'Aligning Knowledge Modules with Language Models'. Many thanks to my advisor Minjoon Seo and the committee members, Arman Cohan, Edward Choi, Eunsol Choi, and James Thorne!
• [June 2025] New preprint! Context-Informed Grounded Supervision introduces a new supervision method to enhance grounding ability in text and vision-language domain.
• [May 2025] Joining Microsoft Research as a Research Intern for Summer 2025. I'll be working on enhancing LLM code understanding!
• [March 2025] Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition wins best paper award at KnowFM 2025!
• [January 2025] Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition has been selected as a ICLR 2025 Oral!
• [January 2025] COrg: Generating Answers from Complex, Interrelated Contexts has been accepted to NACCL 2025!
• [January 2025] Two papers (Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition, How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?) accepted to ICLR 2025!
• [December 2024] RouterRetriever: Exploring the Benefits of Routing over Multiple Expert Embedding Models has been accepted to AAAI 2025!
• [September 2024] Exploring the Practicality of Generative Retrieval on Dynamic Corpora has been accepted to EMNLP 2024!
• [October 2024] Joining Tencent AI (US) as a Research Intern for Winter 2024. I'll be working on hybrid models (linear models + attention-based models) to enhance recall ability with high efficiency!
• [August 2024] How Well Do Large Language Models Truly Ground? wins best student paper award at KnowledgeNLP 2024!
• [June 2024] INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models has been accepted to KnowledgeNLP 2024 Oral!
• [May 2024] Semiparametric Token-Sequence Co-Supervision has been accepted to ACL 2024!
• [May 2024] Joining Adobe Research as a Research Intern for Summer 2024. I'll be working on enhancing answering over complex, interrelated contexts!
• [March 2024] Two papers (How Well Do Large Language Models Truly Ground? and KTRL+F: Knowledge-Augmented In-Document Search) has been accepted to ACL 2024!
• [January 2024] Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis has been accepted to TACL 2024!
• [November 2023] New preprint! Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders introduces a simple yet effective recipes to improve out-of-domain retrieval performance in dense encoders.
• [July 2023] New preprint! Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment introduces ZeroTA, a novel method for dense video captioning in a zero-shot manner.
• [May 2023] Joining Allen Institute for AI (AI2) as a Research Intern for Summer 2023. I'll be working on enhancing retrieval models on out-of-domain!
• [May 2023] Nonparametric Decoding for Generative Retrieval has been accepted to ACL Findings 2023!
• [February 2023] Local 3D Editing via 3D Distillation of CLIP Knowledge has been accepted to CVPR 2023!
• [February 2023] Graduated with M.S. from KAIST and started PhD program
• [October 2022] Generative Multi-hop Retrieval has been accepted to EMNLP 2022 Oral!
• [August 2021] Cost-effective End-to-end Information Extraction for Semi-structured Document Images has been accepted to EMNLP 2021 (Short)!
• [March 2021] Started M.S. program at KAIST Language & Knowledge Lab

Publications