Organizing Academic Research Papers: Evaluating Sources

Evaluating the authority, usefulness, and reliability of resources is a crucial step in developing a literature review that effectively covers pertinent research as well as demonstrating to the reader that you know what you're talking about. The process of evaluating scholarly research also enhances your general skills and ability to:

1. Seek out alternate points of view and differing perspectives,
2. Identify possible bias in the work of others,
3. Distinguish between fact, fiction, and opinion,
4. Develop and strengthen your ability to distinguish between relevant and irrelevant content,
5. Draw cogent, well thought out conclusions, and
6. Synthesize information, extracting meaning through interpretation and analysis.

The act of thinking critically about the validity and reliability of a research resource generally involves asking yourself a series of questions about the quality of both the item and the content of that item.

Evaluating the Source

Inquiring about the Author
What are the author's credentials--institutional affiliation (where he or she works), educational background, past writings, or experience? Is the book or article written on a topic in the author's area of expertise? Has your instructor mentioned this author? Have you seen the author's name cited in other sources or bibliographies? Is the author associated with a reputable institution or organization? What are the basic values or goals of the organization or institution?

Inquiring about the Date of Publication
When was the source published? Is the source current or out-of-date for your topic?

Inquiring about the Edition or Revision
Is this a first edition of this publication or not? Further editions indicate a source has been revised and updated to reflect changes in knowledge, include omissions, and harmonize with the intended needs of its readers. If you are using a Web source, do the pages indicate revision dates?

Inquiring about the Publisher
Note the publisher. If the source is published by a university press, it is likely to be scholarly. Although the fact that the publisher is reputable does not necessarily guarantee quality, it does show that the publisher may have high regard for the source being published.

Inquiring about the Title of Journal
Is this a scholarly or a popular journal? This distinction is important because it indicates different levels of complexity in conveying ideas.

Evaluating the Content

Intended Audience
What type of audience is the author addressing? Is the publication aimed at a specialized or a general audience? Is this source too elementary, too technical, too advanced, or just right for your needs?

Objectivity
Is the information covered fact, opinion, or propaganda? It is not always easy to separate fact from opinion. Facts can usually be verified; opinions, though they may be based on factual information, evolve from the interpretation of facts. Does the information appear to be valid and well-researched, or is it questionable and unsupported by evidence? Note errors or omissions. Are the ideas and arguments advanced more or less in line with other works you have read on the same topic?

Coverage
Does the work update other sources, substantiate other materials you have read, or add new information? Does it extensively or marginally cover your topic? You should explore enough sources to obtain a variety of viewpoints.

Writing Style
Is the publication organized logically? Are the main points clearly presented? Do you find the text easy to read, or is it stilted or choppy? Is the author's argument repetitive?

Evaluative Reviews
In the case of books, locate critical reviews of work in a database. Is the review positive? Is the book under review considered a valuable contribution to the field? Do the various reviewers agree on the value or attributes of the book or has are there strong differences of opinion? Does the reviewer mention other books that might be better? If so, locate these sources for more information on your topic.

Critical Thinking. Academic Skills Centre. University of Canberra; Evaluating Sources. Lakeland Library Research Guides. Lakeland Community College; Evaluating Sources. Writing@CSU. Colorado State University; Evaluating Print Sources. The Writing Center. University of North Carolina; Evaluation During Reading. The Writing Lab and The OWL. Purdue University.
 

Web Content Requires Additional Methods of Evaluation

The principles that guide your of evaluation books, journal articles, reports, and other print materials also applies to web resources. However, the interactive and multimedia dynamics of web-based content increases the level of assessment you must apply in order to ensure that you are viewing a valid source of information.

Additional things to look for when considering using a web-based resource include:

• Source of the content is stated -- whether original or borrowed, quoted, or imported from elsewhere. Note that content imported from another source via RSS feed can be difficult to identify, as this material can blend in with other content on the page without being appropriately labeled.
• Don't be fooled by an attractive, professional-looking presentation -- just because a site looks professional doesn't mean that it is. However, poorly organized or written web page designs are easy to recognize and can be a signal that you should carefully scrutinize the material.
• Site is currently being maintained -- check for posting or editing dates.
• Links are relevant and appropriate, and are in working order -- a site with a lot of broken links is an indication of neglect and out-of-date content.
• The site includes contact information -- if a site is produced anonymously, you cannot verify the legitimacy of its creator.
• Domain location in the site address (URL) is relevant to the focus of the material [e.g., .edu for educational or research materials; .org for profit or non-profit organizations; .gov for government sites]. Note that the domain is not necessarily a primary indicator of site content. For example, some authors post their content on blog or wiki platforms hosted by companies with .com addresses. Note as well that the tilde (~) usually indicates a personal page.

Evaluating Internet Information. Online Library Learning Center. University of Georgia; Evaluating Internet Sources: A Library Resource Guide. Olsen Library. Northern Michigan University; Evaluating Sources. Writing@CSU. Colorado State University; Writing from Sources: Evaluating Web Sources. The Reading/Writing Center. Hunter College.

Bias, whether done intentionally or not, is when a statement reflects a partiality, preference, or prejudice for or against an object, person, place, or idea. Listed below are issues to look for when determining if the source is biased in some way.

1. Distortion or Stretching the Facts -- making issues, problems, or arguments appear more extreme by using misinformation or exaggerated and/or imprecise language to describe research outcomes [e.g., “Everyone agreed the policy was a complete disaster.” Who's everyone? And, how does one specifically define something as a disaster? Is there sufficient evidence to support such a broad statement?].
2. Flawed Research Design -- bias can enter the narrative as a result of a poorly designed study; this may include a claim or generalization about the findings based upon too small a sample, manipulating statistics, or failing to report contrary conclusions from other studies.
3. Lack of Citations -- refers to broad, declarative statements or information presented as fact that does not include proper citation to a source or to sources that support the researcher's position, or that such statements are not explicitly framed as the author's opinion.
4. Misquoting a Source -- this is when an author rewords, paraphrases, or manipulates a statement, the information about a source is incomplete, or a quote is presented in such a way that it misleads or conveys a false impression.
5. Persuasive or Inflammatory Language -- using words and phrases intended to elicit a positive or negative response from the reader or that leads the reader to arrive at a specific conclusion [e.g., referring to one side in a conflict as “terrorists” and the other side as “peace-loving”].
6. Selective Facts -- taking information out of context or selectively data-picking only information that supports the argument while leaving the rest out.

NOTE:  The act of determining bias in scholarly research is also an act of constant self-reflection. Everyone has biases. Therefore, it's important that you minimize the influence of your own biases by approaching the assessment of another person's research introspectively and with a degree of self-awareness.

Evaluating Sources. Lakeland Library Research Guides. Lakeland Community College; Stereotypes and Biased Language. The Writing Lab and The OWL. Purdue University.