UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle
Li, Bixuan; Chen, Qifu; Lin, Jinlin; Li, Sai; Yen, Jerome
2024
Conference Name8th International Conference on Data Mining and Big Data, DMBD 2023
Source PublicationCommunications in Computer and Information Science
Volume2017 CCIS
Pages165-180
Conference Date9 December 2023through 12 December 2023
Conference PlaceSanya
PublisherSpringer Science and Business Media Deutschland GmbH
Abstract

In recent years, the advancement of Artificial Intelligence (AI) technology has brought both convenience and panic. One of the most notable AI systems in recent years was ChatGPT in 2022. In 2023, GPT-4 was released as the latest version. Scholars are increasingly investigating the potential of ChatGPT/GPT-4 for text generation and summarization. Inspired by the principle of “Faithfulness, Expressiveness, and Elegance” in translation, this study investigates the writing and summarizing capabilities of GPT-4, one of the latest AI chatbots. For this purpose, we collected 60 articles from top financial and technology journals, extracted the abstract part, and fed it into GPT-4 to generate abstracts. Three evaluation metrics were created for evaluation: the Text Relevance Score, the AI Detector Score, and the Plagiarism Detector Score. Our findings indicate that abstracts generated by GPT-4 closely resemble the original abstracts without being detected by the plagiarism detector Turnitin in most cases. This implies that GPT-4 can produce logical and reasonable abstracts of articles on its own. Also, we conducted a cross-temporal analysis of GPT-4’s effectiveness and observed continuous and significant improvement. Nevertheless, with the advancement of AI detectors, the abstracts generated by GPT-4 can broadly be recognized as AI-generated. Furthermore, this paper also discusses ethical concerns and future research directions.

KeywordAbstract Generation Artificial Intelligence Chatbot Gpt-4 Large Language Models
DOI10.1007/978-981-97-0837-6_12
URLView the original
Language英語English
Scopus ID2-s2.0-85187708152
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
AffiliationUniversity of Macau, Taipa, Avenida da Universidade, Macau, China
First Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Li, Bixuan,Chen, Qifu,Lin, Jinlin,et al. Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle[C]:Springer Science and Business Media Deutschland GmbH, 2024, 165-180.
APA Li, Bixuan., Chen, Qifu., Lin, Jinlin., Li, Sai., & Yen, Jerome (2024). Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle. Communications in Computer and Information Science, 2017 CCIS, 165-180.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Li, Bixuan]'s Articles
[Chen, Qifu]'s Articles
[Lin, Jinlin]'s Articles
Baidu academic
Similar articles in Baidu academic
[Li, Bixuan]'s Articles
[Chen, Qifu]'s Articles
[Lin, Jinlin]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, Bixuan]'s Articles
[Chen, Qifu]'s Articles
[Lin, Jinlin]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.