From June 16th to 21st, 2024, the North American Chapter of the Association for Computational Linguistics (NAACL) hosted its annual conference, a premier event in Natural Language Processing (NLP). This year's conference, held in the vibrant city of Mexico City, featured cutting-edge research and innovation in computational linguistics. NYU Shanghai is proud to announce that undergraduate students Xiaocheng Yang and Bingsen Chen had their papers accepted and were invited to attend the conference on-site.
At the conference, Yang and Chen presented their study “Arithmetic Reasoning with LLM: Prolog Generation & Permutation” through an engaging poster display. Their research, supervised by Professor Yik-Cheung (Wilson) Tam, Associate Director of the Shanghai Frontiers Science Center of AI and DL, explores utilizing large language models (LLMs) to generate Prolog programs for solving elementary school math problems. This innovative work attracted interest from experts both domestically and internationally.
Left image: Xiaocheng Yang(left) and Bingsen Chen(right) at the meeting;
Right image: Xiaocheng Yang presenting their study to the scholar
Challenging the Status Quo
Scientists have long been researching how to use artificial intelligence to solve math problems. Traditionally, large language models(LLMs) —-AI systems trained on vast amounts of text data to generate human-like language—have used a method called Chain of Thought (CoT) to solve these problems. CoT involves the AI generating a series of calculations step-by-step, much like how a human might solve a math problem. However, this method often leads to errors, making the final answer incorrect.
Yang and Chen, both student researchers at NYU Shanghai, proposed a novel approach: instead of having the AI do all the calculations, they focus on understanding the problem and setting up the steps. The actual arithmetic calculations are then handled by a separate program specifically designed to perform calculations.
Why Prolog？
Their research found that using LLMs to create Prolog programs works much better. Prolog is a type of logic programming language that excels at handling complex reasoning tasks by defining a set of rules and facts and using them to derive conclusions. It allows for defining rules and facts in a way that doesn’t depend on their order. This means LLMs can focus on identifying the key parts of a math problem and translating them into Prolog code. Once the code is written, a Prolog interpreter (a tool that understands and runs Prolog code) can solve the problem accurately. This method overcomes the natural language models' difficulties with direct calculations and symbolic reasoning.
“We’re very excited about the methodology we’ve put together and the potential applications it could lead to,” says Yang. "Our passion for advancing AI research and solving real-world problems is what drives us."
About NAACL
The NAACL conference is renowned for its low acceptance rate, which stood at 23.2% this year, reflecting the high caliber of research presented. As the North American Association for Computational Linguistics (ACL) branch, NAACL is instrumental in advancing NLP research and technology. NAACL 2024 emphasized a unique theme: "Languages of Latin America." The conference covered various topics, including deep learning, NLP techniques, data mining, and corpus linguistics. It focused on interdisciplinary approaches combining insights from linguistics, anthropology, and education to address the unique challenges and opportunities presented by the languages of Latin America and the Caribbean.