Skip to Main Content

NU 602 Library Research Module

Limitations of AI for Finding Academic Sources

Using generative AI chatbots such as ChatGPT, Google Gemini, etc. to find academic sources can be an ineffective strategy due to several key limitations:

1.) Hallucinations 

  • AI chatbots frequently provide incorrect information - called "hallucinations"

  • In academic research, hallucinations most commonly appear as citations to articles that don't exist

  • These fake citations can seem convincing because they often include:

    • Real authors' names

    • Actual journal titles

    • Plausible-sounding article titles

  • Hallucinations are not a temporary bug that will eventually be fixed. Recent research from OpenAI (the organization behind ChatGPT) has found that hallucinations are an inevitable feature of current chatbot training methods

 

2.) Information Quality 

  • Even when chatbots cite real articles, their relevance and quality aren't guaranteed. Just because an AI tool provides an answer does not mean it will be good or relevant. Critical evaluation is always necessary

  • AI chatbots operate with significant unknowns:

    • What sources are in their training data?

    • How current is their information? (Most have knowledge cutoffs)

    • What criteria determine which sources they prioritize?

    • Are they accessing paywalled databases or only free web content?

 

3.) Limited Search Control 

  • Unlike library databases, you cannot easily filter or refine chatbot results using:

    • Date ranges

    • Peer-review status

    • Subject headings

    • Other standardized search filters

  • Chatbot responses are not reproducible - using the same exact prompt will generate similar but not identical results each time, making it difficult to revisit earlier searches

For these reasons, the library's academic databases remain the more reliable and efficient tools for finding scholarly resources. 

When Can AI Be Included in Academic Research? 

While not very effective for finding academic sources, AI chatbots can be helpful with preliminary research tasks:

  • Brainstorming research topics, ideas, or keywords

  • Getting quick overviews of unfamiliar concepts or terminology

  • Summarizing complex articles you've already found and verified through library databases

Better approach: Use AI tools embedded within library databases. These specialized tools:

  • Are designed for specific research functions

  • Have narrower focus that reduces hallucination risk

  • Provide more accurate, reliable results

Library AI Tools

The library provides AI-powered research tools that are more reliable than general chatbots because they work with verified academic content:

  • Natural Language Search (QuickSearch) - Search using natural language questions

  • "More Like This" (databases & QuickSearch) - Find similar articles automatically

  • Scopus AI - Generate summaries from peer-reviewed literature

These tools are designed specifically for academic research and have lower risk of hallucinations.

Learn how to use these tools: See the AI Tools in the SHU Library page in the AI & Academic Research Guide for detailed instructions. 

An Important Reminder:

Always check your instructor's AI policy before using any AI tools in your coursework. When in doubt, be sure to ask! 

Accessibility