THE USE OF GENERATIVE ARTIFICIAL INTELLIGENCE IN HEURISTIC USABILITY EVALUATION

capabilities and limitations

Abstract

This study analyzes the use of Generative Artificial Intelligence as a support tool for evaluating the usability of a web page based on Nielsen’s heuristics. A qualitative-quantitative, descriptive, and exploratory approach was adopted, using a literature review and the implementation of an empirical test in which an evaluative prompt was developed to conduct a heuristic evaluation of a digital repository’s homepage. The prompt was submitted to the ChatGPT, Copilot, and Gemini tools, and the convergences and divergences among the results obtained were analyzed. The findings show that all GenAIs could perform heuristic evaluations of the interface; however, they differed in the descriptive detail and precision of the evaluations, in whether or not they considered the user profile described in the prompt, and in the assignment of severity ratings for the identified problems. ChatGPT stood out for providing more complete, contextualized analyses that were closer to a specialized human evaluation. Gemini produced results with less detail and contextualization than ChatGPT, focusing mainly on the structural and visual aspects of the interface. Copilot, in turn, demonstrated interpretative inconsistencies and evaluation errors, presenting the analysis in a more objective manner and with fewer explanations. It is concluded that GenAIs have potential as a pre-diagnostic tool in heuristic usability evaluations, since they are used critically and under supervision.