Best L3S Publication of the Quarter (Q1/2024 )
Category: NLP, LLM
Does a language model “understand” high school math?
A survey of deep learning based word problem solvers
Authors: Sowmya S Sundaram, Sairam Gurajada, Deepak Padmanabhan, Savitha Sam Abraham, Marco Fisichella:
Published in the Q1 journal “Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery”
https://wires.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/widm.1534
The paper in a nutshell:
Our work discusses the development and evaluation of algorithms that automatically solve mathematical word problems (MWPs), a task combining text pattern recognition with mathematical reasoning. Despite significant research, achieving robust representations for basic math word problems remains challenging. While solving simple arithmetic word problems is considered a solved problem, we demonstrate how many nuances are yet to be addressed. Over the years, various models have evolved from hand-crafted rule-based systems to deep- learning models that include sequence-to-sequence paradigms, graph-based models, and large language models (LLMs).
Which problem does the research solve?
We review deep- learning approaches to MWPs, analyzing their strengths and weaknesses, and note that while they show promising results, there are gaps to be addressed such as stability, correctness, and confidence. It includes a critical examination of the experimental designs and the datasets commonly used in MWP research.
Historical and recent progress in the field is also addressed, noting the numerous attempts to solve MWPs using various computational techniques. We propose future directions for research, which include improving model robustness, creating more representative datasets, and refining evaluation metrics to better assess the models’ understanding of mathematical concepts.
