Skip to Main Content

AI in the Classroom Faculty Guide

Major Limitations of AI Chatbots

The capabilities of ChatGPT, Google's Gemeni, and other chatbots capabilities are definitely considerable, but they are also undeniably constrained by a few core weaknesses:

  1. Despite what some may believe based off the term "artificial intelligence," these chatbots do not actually "think" when they reply to user prompts. They generate responses using probability to determine what word is most likely to come next based on the vast amount of data it has been trained on. In other words, chatbots are not consciously aware of the answers they provides - they just predicts and imitates. They have no sense of context of when they provide answers. 
  2. The chatbots also tend to suffer from issues of incompleteness - it's often not clear when the cutoff date of the information that they access it, or what resources they scraped their information from. 

As a result: 

  • Chatbots often provides false information in their responses.
  • In particular, they often provides citations for resources that don't exists when asked to provide resources for citations for a specific research topic - an issue that has already been experienced by faculty and librarians alike here at SHU.  

You can check out the articles linked below to read more in-depth about the subject/ 

Articles That Discuss and Highlight Chatbot Limitations

Scholarly Articles About Major Chatbot Limitations

Accessibility