UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Reason-and-Execute Prompting: Enhancing Multi-Modal Large Language Models for Solving Geometry Questions
Duan, Xiuliang1; Tan, Dating1; Fang, Liangda1,2; Zhou, Yuyu1; He, Chaobo3; Chen, Ziliang4; Wu, Lusheng1; Chen, Guanliang5; Gong, Zhiguo6; Luo, Weiqi1; Guan, Quanlong1
2024-11
Conference Name32nd ACM International Conference on Multimedia, MM 2024
Source PublicationMM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia
Pages6959-6968
Conference Date28 October 2024 - 1 November 2024
Conference PlaceMelbourne, VIC
CountryAustralia
Publication PlaceNew York, NY, USA
PublisherAssociation for Computing Machinery, Inc
Abstract

Multi-Modal Large Language Models (MM-LLMs) have demonstrated powerful reasoning abilities in various visual question-answering tasks. However, they face the challenge of lacking rigorous reasoning and precise arithmetic, when solving geometry questions. To address this challenge, we propose a novel prompting method, namely Reason-and-Execute (R&E), to enhance the accuracy of solving geometry questions by MM-LLMs. Specifically, the R&E method includes two templates: reasoning template and execution template. We first adopt a reverse-thinking approach to construct a rigorous reasoning template so that it guides MM-LLMs to start reasoning from the most relevant domain knowledge of the question and ultimately identify the arithmetic requirements. We then make use of program-assisted thought to construct execution template in order to guide MM-LLMs to understand the arithmetic requirements from reasoning template and generate executable code block. The answer is finally obtained by executing the code block. We evaluate our prompting method on 9 models in answering questions on 6 datasets (including four geometry datasets and two science datasets) compared to Chain-of-Thought (CoT) and Program-Aided Language (PAL) prompting methods. R&E method shows up to 12.8% improvement compared to CoT and PAL, proving strong reasoning and arithmetic abilities for solving geometry questions of our method. Moreover, we further analyze the answering accuracy from the different perspectives on solving geometric questions, including domain knowledge, geometry shapes, question length, and language. Through multiple analysis, our method is able to enhance the ability of MM-LLMs to solve geometry questions.

KeywordGeometry Questions Multi-modal Large Language Models Prompting Method
DOI10.1145/3664647.3681484
URLView the original
Language英語English
Scopus ID2-s2.0-85209821670
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Affiliation1.Jinan University, Guangzhou, China
2.Pazhou Laboratory, Guangzhou, China
3.South China Normal University, Guangzhou, China
4.Peng Cheng Laboratory, Shenzhen, China
5.Monash University, Melbourne, Australia
6.University of Macau, Macao
Recommended Citation
GB/T 7714
Duan, Xiuliang,Tan, Dating,Fang, Liangda,et al. Reason-and-Execute Prompting: Enhancing Multi-Modal Large Language Models for Solving Geometry Questions[C], New York, NY, USA:Association for Computing Machinery, Inc, 2024, 6959-6968.
APA Duan, Xiuliang., Tan, Dating., Fang, Liangda., Zhou, Yuyu., He, Chaobo., Chen, Ziliang., Wu, Lusheng., Chen, Guanliang., Gong, Zhiguo., Luo, Weiqi., & Guan, Quanlong (2024). Reason-and-Execute Prompting: Enhancing Multi-Modal Large Language Models for Solving Geometry Questions. MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia, 6959-6968.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Duan, Xiuliang]'s Articles
[Tan, Dating]'s Articles
[Fang, Liangda]'s Articles
Baidu academic
Similar articles in Baidu academic
[Duan, Xiuliang]'s Articles
[Tan, Dating]'s Articles
[Fang, Liangda]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Duan, Xiuliang]'s Articles
[Tan, Dating]'s Articles
[Fang, Liangda]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.