NU 602 Library Research Module

Limitations of AI Chatbots in Academic Research

AI chatbots like ChatGPT and Google Gemini have become popular tools for many tasks. However, when it comes to academic research, these tools have significant limitations:

1.) Hallucinations 

• AI chatbots will frequently provide incorrect information in their responses. This phenomenon is called a hallucination. 

• In academic research, hallucinations most commonly appear as citations to academic articles that don't exist.

• These fake citations can seem convincing, since they often include real author names, actual peer-reviewed journal titles, and plausible-sounding article titles. 

• Hallucinations are not a temporary problem that will eventually be fixed over time. Recent research from OpenAI (the organization behind ChatGPT) has found that hallucinations will occur as an inevitable feature due to how chatbots are currently trained. You can view that study at this link. 

2.) Information Quality 

• Even when chatbots cite real articles, their relevance and quality aren't guaranteed. Critical evaluation is always needed. A chatbot-provided resource will not necessarily be the best resource. It may not even be a good or even a relevant resource. 

• AI chatbots often operate with significant unknowns:

  • What sources are included in their training data?

  • How current is their information? (Most have knowledge cutoffs)

  • What criteria determine which sources they prioritize?

  • Are they accessing paywalled databases or only free web content?

3.) Refining Searches 

• Unlike in academic databases, you cannot easily filter, sort, or refine chatbot results using standardized search options (like date ranges or peer-review status)

• Also, chatbot responses are not reproducible. The same prompt, even if copy and pasted word-for-word, will generate similar but not identical results each time. This makes it hard to revisit earlier search results. 

For these reasons, the library's academic databases remain the more reliable and efficient tools for finding scholarly resources. 

When Can AI Be Included In Academic Research?

While AI chatbots are not very effective for finding academic resources, they can be helpful with preliminary research tasks:

• Brainstorming ideas, research topics, or keywords

• Helping summarize complex articles that you've already found and verified through library databases 

• Getting quick overviews of unfamiliar concepts, theories, or terminology

Overall, generative AI is most reliable when embedded within library databases as specialized research tools. These database-integrated AI features are designed for specific research functions, and their narrow focus reduces the risk of hallucinations and improves accuracy. 

To learn about the AI-powered research tools available through the SHU Library databases and strategies for using them effectively, visit the AI Tools in the Library page of the library's AI & Academic Research guide. 

Important Reminder: Always check your instructor's AI policy before using any AI tools in your coursework. When in doubt, be sure to ask!