ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers
Saptarshi Sengupta†, Zhengyu Zhou, Jun Araki, Xingbo Wang, Bingqing Wang, Suhang Wang, Zhe Feng
EACL 2026 (Oral).
Xingbo Wang
Researcher building useful and reliable AI systems
- Large Language Models
- AI Agents
- Human–AI Interaction
About Me
I am Xingbo Wang, an AI researcher working at the intersection of large language model (LLM) agents and human–AI interaction. I am currently a research scientist at Bosch
, developing LLM-based agentic systems for complex, knowledge-intensive tasks. More broadly, I build interactive tools that help people understand, evaluate, and collaborate with AI.
I received my PhD in Computer Science and Engineering from HKUST, advised by Prof. Huamin Qu. I later worked with Prof. Fei Wang at Weill Cornell Medicine and Prof. Tanzeem Choudhury at Cornell Tech. My work spans industry, healthcare, and education, with the goal of making AI systems more useful, reliable, and grounded in human reasoning.
Current Focus
- Agent optimization: developing meta-agents, meta-prompting, and self-improvement methods that enable agents to optimize their own behavior.
- Tool-integrated reasoning: enabling agents to create, retrieve, and use tools effectively.
- Knowledge extraction and grounding: turning structured and unstructured evidence into grounded, actionable knowledge.
- Human–agent interaction and evaluation: designing intuitive interfaces, oversight, and evaluation methods for effective and reliable collaboration.
Beyond Research
Outside research, I enjoy discovering local coffee shops (30+ in New York City so far), exploring new trails, and playing video games. My partner and I also share our home with three cats: Whisky , Coffee , and DuoLe .
News
- Jan. 2026 📢 : Our intern's work ToolDreamer on tool retrieval with synthetic tool generation is accepted to EACL 2026. Congrats!
- Nov. 2025 📢 : One paper about visual analytics for online user behavior progression exploration is accepted to TVCG. Checkout details here Congrats to Rui and the team!
- Sep. 2025 📢 : My paper about interactive strcutured data extraction from scientific literature is accepeted to Campbell Systematic Reviews. Welcome to check our code here!
- Aug. 2025 📢 : One paper about multimodal visual analytics for E-commerce fashion design is accepted to IEEE CG&A. Congrats to Quan's team!
- Jul. 2025 📢 : I will serve as a program committee member for ACM CSCW 2026.
- Jul. 2025 📢 : Our paper: TrialCompass: visual exploration of clinical trial design criteria is accepted to IEEE VIS' 25! Congrats Rui and the team!
- Jan. 2025 📢 : I am excited that my year-long project on Multi-agent LLMs for personalized sleep health is conditionally accepted to CHI' 25. Stay tuned for more details!
- Feb. 2025 📢 : I will serve as a program committee member for IEEE VIS 2025.
- Jan. 2025 📢 : Oue paper about prompt optimization for enhancing multimodel Reasoning of LLMs is accepted to Proc. of PacificVis 2025.
- Jan. 2025 📢 : One paper about problem-driven design space and pattern analysis for visual analytics research is accepted by TVCG 2025!
- Dec. 2024 📢 : I will serve as a Associate Chair for CHI 2025 Late-Breaking Work. Thanks for the invitation!
- Dec. 2024 📢 : Our paper about Multi-agent LLMs for data insights management and navigation for exploratory data analysis is accepted to TVCG (Proc. of PacificVis 2025).
- Nov. 2024 📢 : One paper about multimodel presentation delivery support system is accepted to Chinese CHI 2024 and won the Best Paper Award 🥇. Cograts to Yuchen and Quan's team!
- Jul. 2024 📢 : Two co-athored papers with SYSU's teams on LLMs for (1) supporting teachers' teaching plans and (2) proactive peer support are accepted to UIST 2024.
- Jun. 2024 📢 : I will serve as an Associate Chair for CSCW 2025. Thanks for the invitation!
- May. 2024 📢 : One paper about multimodal vocabulary learning via story retelling is accepted to DIS 2024.
- Feb. 2024 📢 : One scoping review paper about LLM interpretability and applications in biomedicine is published in iScience.
- Jan. 2024 📢 : One viewpoint paper about LLM in healthcare is published in Journal of General Internal Medicine (JGIM).
- Dec. 2023 📢 : I will serve as an Associate Chair for CHI LBW 2024. Thanks for the invitation!
- Nov. 2023 📢 : Built a web app for personalized health data analysis: link.
- Aug. 2023 📢 : Built a web app for interactive visual data analysis, where you can use chat with your data through natural language: link.
- July. 2023 🎉 : Four papers (one about explaning commonsense knowledge and reasoning capabilities of NLP models, and three about multimodal data analysis) are accepted to IEEE VIS 2023 @ Melbourne. 👏 More details to come later.
- July. 2023 🎉 : One paper about story co-writing for vocabulary learning is conditionally accepted to ACM UIST 2023@San Francisco. 👏
- Jun. 2023 📢 : I started my postdoc @ Weill Cornell Medicine.
- Jun. 2023 📢 : Our 30 Million QEF Project AI-assisted VR English Speaking Program, where I act as a Co-PI, is funded! 👏
- Mar. 2023 📢 : Our MultiViz paper is accepted to CHI LBW 23. 👏
- Feb. 2023 📢 : I gave a talk at Weil Cornell Medicine titled "Unlease the Power of Data-driven AI through Explanations and Interactions".
- Feb. 2023 📢 : One paper about multimodal satisfaction analysis is acceoted to IEEE TVCG. 👏 (Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events)
- Jan. 2023 📢 : One paper about interpretability analysis of multimodal models is accepted to ICLR 2023. 👏 (MultiViz: Towards Visualizing and Understanding Multimodal Models)
- Jan. 2023 📢 : One paper about explanation and diagnosis of NLI is accepted to IEEE TVCG. 👏 (XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis)
- Jan. 2023 📢 : One paper about shortcut exploration in NLP datasets is accepted to IEEE TVCG. 👏 (ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset)
- Dec. 2022 Three papers receive minor revision at IEEE TVCG.
- Jul. 2022 📢 : One poster about pre-screening tool for students with specific learning disabilities is accepted by ASSETS 2022 👏
- Jul. 2022: I will serve as a student volunter captain at IEEE VIS 2022.
- Jan. 2022 📢 : One paper about persuasive strategies in online discussion is accepted with minor revision by CSCW 2022 👏 (Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion)
- Jan. 2022 📢 : One paper about multimodal gesture analysis receives minor revision at IEEE TVCG 👏 (GestureLens: Visual Analysis of Gestures in Presentation Videos)
- Oct. 2021 We release the project pages of DeHumor and M2Lens. Welcome to check them out.
- Sep. 2021 Our paper "M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis" receives Best Paper Honorable Mention !
- Jul. 2021 I present "M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis" @China VIS 2020, Wuhan! (, )
- Jul. 2021 I visit IDG group of Zhejiang Univeristy, thank professor Wu for his hospitality! ()
- Jul. 2021 Our XAI paper "M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis" is accepted by IEEE VIS 2021, Virtual! See you online!
- Jul. 2021 Our paper "DeHumor: Visual Analytics for Decomposing Humor" is finally accepted by IEEE TVCG, 2021! Stay tuned!
- Jan. 2021 Our team won Silver Award of the 6th China International College Students' "Internet+" Innovation and Entrepreneurship Competition (, , ) Announcement
- Dec. 2020 Our team won Silver Award of 12th Challenge Cup National College Student Business Plan Competition (, ) Announcement
- Jun. 2020 I passed the PhD Qualifying Exam (PQE). And now I am a PhD Candidate!
- !!! Jun. 2020 !!! Our project won the First Prize for the 6th Hong Kong University Student Innovation and Entrepreneurship Competition
- **!!! May. 2020 !!! Welcome to check the online presentation of VoiceCoach for CHI 2020.
- Dec. 2019 Our paper "VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking" has been conditionally accepted by CHI 2020.
- Jul. 2019 Our paper "EmoCo: Visual Analysis of Emotion Coherence in Presentation Videos" is accepted by IEEE VIS 2019 @Vancouver, Canada!
- Jun. 2019 Our project FLY healthtech wins Student Team Award @HKUST-SINO One Million Dollar Entrepreneurship Competition 2019!
- Aug. 2018 I join Vislab @HKUST.
- Jun. - Aug.2018 I visit at IDG @ZJU.
- Jun. 2018 I graduate from WHU.
Publications
User interfaces for human-AI interactions and human-data interactions
(*: equal contribution; +: corresponding author; †: intern(s))
Filter by topic
SciDaSynth: Interactive Structured Data Extraction from Scientific Literature with Large Language Model NEW
Xingbo Wang*, Samantha L. Huey*, Rui Sheng, Saurabh Mehta, Fei Wang
Campbell Systematic Reviews 2025 (to appear).
code | pdf | arXiv | Wiley Online
POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
Jianben He, Xingbo Wang, Shiyi Liu, Guande Wu, Claudio Silva, Huamin Qu
IEEE Pacific Visualization Conference (PacificVis), 2025.
InsightLens: Augmenting LLM-Powered Data Analysis with Interactive Insight Management and Navigation
Luoxuan Weng, Xingbo Wang, Junyu Lu, Yingchaojie Feng, Yihan Liu, Haozhe Feng, Danqing Huang, Wei Chen
IEEE Transactions on Visualization and Computer Graphics (Proceedings of PacificVis 2025) (to Appear).
CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
Xingbo Wang, Renfei Huang, Zhihua Jin, Tianqing Fang, and Huamin Qu
IEEE Transactions on Visualization and Computer Graphics (VIS 2023).
PDF | arXiv | IEEEXplore | 💻/🌐 Project Page & Demo | video | Slides
VideoPro: A Visual Analytics Approach for Interactive Video Programming
Jianben He, Xingbo Wang+, Kam Kwai Wong, Xijie Huang, Changjian Chen, Zixin Chen, Fengjie Wang, Min Zhu, Huamin Qu.
IEEE Transactions on Visualization and Computer Graphics (VIS 2023).
PDF | arXiv | IEEEXplore
PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
Yingchaojie Feng, Xingbo Wang, Kam Kwai Wong, Sijia Wang, Yuhong Lu, Minfeng Zhu, Baicheng Wang, Wei Chen.
IEEE Transactions on Visualization and Computer Graphics (VIS 2023).
M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
Xingbo Wang, Jianben He, Zhihua Jin, Muqiao Yang, Yong Wang, Huamin Qu.
IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS'21).
🏆 Best Paper Honorable Mention
PDF | arXiv | IEEEXplore | 💻 Project Page | 🌐 Demo