Generative AI for UNT: Assessing the Quality of Mathematics Item Cloning

Abstract

The paper presents the results of a study devoted to analyzing the quality and reliability of mathematics test items with a single correct answer generated using artificial intelligence language models. The main objective of the research was to evaluate how well such items correspond to the requirements of national testing and whether they can be effectively used in educational practice. Within the framework of the study, a controlled cloning of 200 test items in the format of the Unified National Testing (UNT) was carried out based on a previously verified bank of test materials. After generation, the items underwent a multi-stage expert evaluation process, which included checking mathematical correctness, determining the cognitive level of difficulty, and analyzing compliance with official testing specifications.Modern artificial intelligence language models were used to generate the tasks, including GPT-4, DeepSeek, Qwen versions 2.5 and 3, and Claude 3. Comparative analysis showed that the vast majority of the generated items demonstrated a high level of quality. According to the results of the expert review, 97% of the tasks were recognized as suitable for further use. Moreover, 50.5% of them did not require any additional revision and could be used in their original form. The obtained results indicate a low defect rate and confirm the stability of the applied algorithms for generating test items. Thus, the findings of the study demonstrate the significant potential of integrating artificial intelligence technologies into the processes of updating, expanding, and optimizing the UNT item bank, which may improve the efficiency of developing test materials and accelerate their renewal.

Keywords: artificial intelligence, generative language models, automated item cloning, UNT, educational measurement, test validation.