How AI's Triumph over PIAAC May Soon Disrupt Traditional Teaching and Learning

The Organization for Economic Co-operation and Development (OECD) has released a report that assesses AI progress in reading and mathematics, two core domains essential for success in education, work, and life. Titled “Is Education Losing the Race to Technology?” the report highlights AI’s ability to perform well on the PIAAC, an international assessment of adults' literacy, numeracy and problem solving skills.

source: DeepMind

According to the study, AI could answer around 80% of PIAAC literacy questions as of 2021. It can solve most easy questions, such as locating information in short texts and identifying basic vocabulary. AI can also handle many of the more difficult questions, in which answers must be formulated across large chunks of text. In addition, AI could solve around two-thirds (66%) of the PIAAC numeracy test, which involves everyday mathematical situations. Easier questions involve single-step processes, while harder ones require multiple steps and various types of mathematical content, such as images, symbolic notations, formulae, diagrams, graphs, tables and maps.

The OECD report mostly focuses on AI’s progress in literacy and numeracy from 2016 to 2021, or before ChatGPT was released in 2022. The report reveals a 25% improvement in AI’s literacy capabilities from 2016 to 2021, reflecting technological breakthroughs in natural language processing (NLP). Yet, the OECD experts found that AI’s numeracy capabilities did not change much during the same five-year period.

According to the report, AI could outperform large portions of the OECD population in both reading and math as of 2021. Moreover, the OECD experts agree that AI will be able to solve the entire literacy and numeracy tests by 2026 -- and some predict this will happen sooner.

Generally, artificial intelligence has struggled with math more than literacy, especially math word problems. Yet, the researchers point out that large language models such as GPT have been “fine-tuned” for quantitative problem analysis over the last year or so. The three experts who gave AI middle-to-high ratings concluded that, given the recent advancements in the field, AI is close to solving the PIAAC numeracy test. They suggested, hypothetically, this would occur in less than one year. The one expert who gave AI the lowest ratings focused on the state of AI techniques at the time of the study. But even this expert agreed that, given recent advances, AI will be able to solve the numeracy test within a year.

Furthermore, the study was concluded well before the introduction of GPT-4, a major update to the large language model that helps power ChatGPT. Introduced last month, GPT-4 immediately achieved “stunning results” on standardized educational assessments. For instance, The ABA Journal reported that ChatGPT-4 passed the Uniform Bar Examination in the 90th percentile (GPT-3.5 was around the 10th percentile). It also scored in the 88th percentile on the LSAT (up from GPT-3.5’s 40th percentile).

GPT-4 also excelled on standardized secondary-level exams. CNBC reported that on the Scholastic Aptitude Test (SAT), used to measure a high school student's readiness for college, AI “aced” the SAT Reading & Writing section with a score of 710 out of 800, in the 93rd percentile of test-takers. For the math section, GPT-4 earned a 700 out of 800, ranking among the 89th percentile of test-takers. In total, GPT-4 scored 1410 out of a possible 1600 points, significantly better than the average score of 1060.

GPT-4 also passed a slew of Advanced Placement (AP) tests. In AP exams, scores range from 1 to 5, with 3 and above generally considered passing grades. GPT-4 received a 5 on AP Art History, AP Biology, AP Environmental Science, AP Macroeconomics, AP Microeconomics, AP Psychology, AP Statistics, AP US Government and AP US History, according to OpenAI. On AP Physics 2, AP Calculus BC, AP Chemistry, and AP World History, GPT-4 received a 4.

AI's progress in literacy, numeracy and problem solving has huge implications for employment and education. It highlights the need to strengthen workforce skills and prepare humans to work together with AI in key domains. With AI's ability to outperform humans in reading and mathematics, it becomes increasingly important for educators to adapt their teaching methods and curricula to better prepare students for a future where they must coexist with and leverage AI systems. The OECD suggests that this may involve shifting the focus of education “towards teaching students how to effectively use AI systems in performing literacy and numeracy tasks," ideally fostering a symbiotic relationship between human and AI capabilities.

In any event, the rapid advance of AI raises big questions about the potential impact on the role of teachers and the importance of human interaction in the learning process. As I've argued in a previous post, it is crucial for schools to construct a framework for leveraging technology to enhance teaching and learning, while also nurturing the human elements that have been an integral part of the educational experience for centuries.

This should involve continuous professional development for educators, emphasizing the importance of creativity and social-emotional intelligence in teaching and learning. As I wrote in my last post, educational occupations are among the most exposed to advances. So, time is of the essence. In the end, the future of education will likely depend on forging a harmonious collaboration between humans and AI, ensuring that students are well-prepared to thrive in an increasingly technology-driven world.