Political science professors at our institution are often frustrated by the quality of student research, particularly in terms of the quality of sources used in research papers. Students increasingly rely on the Internet as their primary method of gathering information (Barberio Reference Barberio2004; Robinson and Schlegl Reference Robinson and Schlegl2005). Although it is possible to use the Internet to identify high-quality, scholarly sources, the Internet can also be dangerous territory for students. In particular, students often have difficulty distinguishing between scholarly sources, such as academic journals or primary source data, and nonscholarly sources. For many professors, the response to this problem has been the traditional “one-shot” visit to the college library or a classroom visit from a librarian, to review the resources and materials relevant to student research on politics and government.
Is a single visit enough to turn students into good researchers? Research in the field of library science suggests that more is better when it comes to time in the library (Gandhi Reference Gandhi2004). A growing body of research points to the benefits of a focus on information literacy. According to the Association of College and Research Libraries (ACRL) Information Literacy Competency Standards for Higher Education (2000), information literacy involves the ability to “determine the extent of information needed … access the needed information … and evaluate information and its sources critically.” Although the use of this terminology in political science is relatively new (Marfleet and Dille Reference Marfleet and Dille2005; Williams, Goodson, and Howard Reference Williams, Goodson and Howard2006; Stevens and Campbell Reference Stevens and Campbell2008; Williams and Evans Reference Williams and Evans2008), certainly the concept is not.
This article examines the development and assessment of an integrated information literacy component designed to enhance student learning in political science. Our decision to implement an information literacy program grew out of the overlapping goals of political science and library faculty at Gustavus Adolphus College, a private, liberal arts college located in the midwest with an average enrollment of 2,300 undergraduates. The campus library is a teaching library, meaning that its mission is to educate students about research. Six library faculty members are responsible for delivering library instruction, which consists primarily of single sessions arranged by classroom faculty in collaboration with library faculty, to all on campus. The political science department has eight full-time faculty, serving 100 majors and enrolling approximately 900 students annually in courses covering all major subfields in political science. Political science faculty sought ways to enhance student research skills beyond an introductory level. Library faculty wanted to expand efforts to partner with academic departments in teaching information literacy skills among undergraduate students. Both library and political science faculty also wished to understand more precisely the effect of new teaching strategies on student learning.
We hypothesize that adding an integrated library component to an undergraduate research methods course leads students to select higher quality sources, demonstrate a broader and deeper use of library resources, and report increased levels of confidence in library research—skills that are hallmarks of information literacy. Our findings indicate that students who receive multiple instruction sessions in the library do demonstrate enhanced research skills, and increasing the amount of library instructional time further improves student research skills.
INFORMATION LITERACY IN THE CLASSROOM
Many political science writing assignments require students to identify and use high-quality sources, whether in traditional research papers or shorter response essays. Recent research suggests that incorporating information literacy into college courses can improve students' ability to do this effectively. Consequently, ACRL guidelines call for the development of information literacy goals within the context of discipline-based research and encourage collaboration between faculty and librarians (LPSS 2008). Discipline-based approaches to information literacy have been endorsed by political scientists as well (Grafstein Reference Grafstein2002; Hutchins Reference Hutchins and Avery2003; Norelli Reference Norelli, Cook and Cooper2006; Stevens and Campbell Reference Stevens and Campbell2008; Williams and Evans Reference Williams and Evans2008).
Existing research on the impact of one or two library instruction sessions on student learning outcomes indicates mixed results. In other disciplines, some researchers find little change (Emmons and Martin Reference Emmons and Martin2002), whereas others find significant improvement, particularly in the research confidence of students (Zoellner, Samson, and Hines Reference Zoellner, Samson and Hines2008). In political science, Marfleet and Dille (Reference Marfleet and Dille2005) experiment with students in one section of a methods course and one section of a nonmethods course. These students who are exposed to an information literacy component featuring a session with a reference librarian perform better on an assessment instrument measuring their information literacy competencies, compared to students in a similar section of a methods course and nonmethods course who are not exposed to the information literacy component.
Research focused on multisession library instruction models, however, is more conclusive and points to the positive effects of incorporating information literacy. Gandhi (Reference Gandhi2004) assesses learning outcomes for a five-session library instruction sequence and finds that students in the experimental group demonstrate higher levels of learning than students who receive the traditional one-shot session. Wang (Reference Wang2006) assesses the long-term effects of a semester-long library credit-bearing course on student information literacy skills. In comparing research papers written by students who had taken an elective library research course with students who had not, Wang finds that students in the elective course use significantly more scholarly resources in their papers, have fewer citation errors, and earn higher grades. Williams, Goodson, and Howard (Reference Williams, Goodson and Howard2006) find that adding a multisession integrated library component to an undergraduate comparative politics course improves students' information literacy and that “information literacy skills correlate significantly with student performance in written work” (518). Goebel, Neff, and Mandeville (Reference Goebel, Neff and Mandeville2007) show that students completing a semester-long information literacy course demonstrate improvement in specific research skills such as knowing how to use electronic periodical databases, as well as more ability to locate and evaluate relevant literature and avoid plagiarism. Finally, Stevens and Campbell (Reference Stevens, Campbell, Jacobson and Mackey2007) outline a collaborative approach to incorporate information literacy into courses at various levels within a political science department using multiple library sessions. They find that students at all levels improve, with the greatest improvement in lower-level and midlevel courses; interestingly, sophomores and juniors make greater gains than first-year or senior students.
Although this research certainly suggests that political science students would benefit from incorporating information literacy into the entire curriculum, we argue that methodology courses are a particularly appropriate place for the development of such a focus. As Hubbell (Reference Hubbell1994) argues, undergraduate research methodology courses “should primarily train students to be intelligent consumers of research” (60). Whereas other courses are heavily focused on content, methodology courses are primarily concerned with process. Thus, teaching the process of finding and evaluating appropriate research resources complements the overall focus of methodology courses. Furthermore, about two-thirds of US college political science departments offer research methods courses (Thies and Hogan Reference Thies and Hogan2005); Stevens and Campbell's research suggests that upper-level students—the largest audience for such courses as part of political science major programs—will particularly benefit from a focus on information literacy.
METHODOLOGY
To study the effectiveness of multiple instruction sessions on the information literacy skills of political science students, we developed a semester-long library lab component for the political science department's research methods course, Analyzing Politics. The course is typically taken by students in their sophomore year and is required for all majors. Due to its focus on research within the discipline, Analyzing Politics proved to be a natural place to incorporate a more comprehensive approach for developing students' information literacy skills. Targeting sophomore students also equips them to conduct more sophisticated research throughout the rest of their college careers.
The library lab component originated as a four-session instruction arc taught by J. Gilbert (library faculty) from spring semester 2007 to spring 2008 (fall 2006 students received no library instruction sessions and thus serve as a control group). Initial assessment data revealed that students who had been exposed to the four-session arc exhibited stronger research skills than control group students. Political science faculty (Knutson and C. Gilbert), in consultation with J. Gilbert, then expanded from four sessions to a semester-long library lab component, beginning in fall 2008. Lab students, as we will refer to them, meet with a librarian once each week through the entire semester (one-quarter of the scheduled course instruction time) to explore the diverse information sources available for political science research including government documents and statistical sources. Lab students also complete assignments designed to reinforce research skills discussed in class; the librarian grades all such assignments. The library lab component has several goals: help students investigate information sources in political science, including how to locate and use them; equip students with the skills to conduct effective research; prod students to discuss ethical and practical questions related to research; and refine student research skills as necessary to undertake a major research paper in their senior year (a requirement of the political science major).
We developed multiple assessment tools to analyze the effectiveness of the lab component and generate ideas for course modifications in future semesters. These tools include an end-of-semester survey of research skills and perceptions, writing prompts that include a final narrative self-assessment of research abilities, and a citation analysis of annotated bibliography papers. The survey asks students several questions about their research process: what sources of assistance they used during the semester, how much time they spent searching for annotated bibliography sources, and their perceptions of confidence in conducting research after completing the course. The narrative self-assessment allows students to reflect on their learning in more detail in two ways: first, by discussing the degree to which the lab component helped them with research in both Analyzing Politics and other classes and second, by identifying what they still find confusing about the library or research in general. To provide assessment data independent from student perceptions, we also analyze the sources students use for their annotated bibliography papers, which are assigned nearly every semester in Analyzing Politics. We count total sources and types of sources, focusing in particular on high-quality sources, which we define as peer-reviewed academic journal articles, books from university presses, articles from leading newspapers such as the New York Times and elite periodicals such as The Economist; and Internet sites sponsored by leading political organizations or government agencies.Footnote 1
LIBRARY LAB
The librarian designs every lab session and assignment to help students better complete the research required for Analyzing Politics course assignments. Several lab assignments in particular support the annotated bibliography paper. For example, early in the semester we discuss the differences between scholarly and nonscholarly sources. For the graded in-class assignment, students examine a variety of sources both in print and electronic formats at different stations throughout the classroom and determine whether or not the sources are scholarly. This exercise is beneficial because it puts sources directly into students' hands (many students have never touched a paper journal before) and helps them to understand the nature of scholarly publishing.
After students identify possible research topics during week three of the semester, another lab assignment asks them to investigate three different databases and write about what they find about their topics in each database. In addition to prodding students to use databases they have typically not used before (such as Social Sciences Citation Index), the assignment helps students determine the shape of research within their chosen topics. After students have chosen one topic for the annotated bibliography and other course papers, a related assignment toward the middle of the semester asks students to determine the extent to which the library can support research on their chosen topic. This assignment walks students through the types of resources available (such as reference books, other scholarly books, articles, and Internet sources) and asks them to describe how specific sources contribute to their understanding of their research topic. The assignment also requires students to speak with a librarian at the reference desk, which broadens their base of research assistance (especially because our students traditionally underutilize the reference desk). Students also outline a research plan, including a timeline for their research and an awareness of the degree to which they may need to request additional resources via interlibrary loan.
The lab component also teaches students about bibliographic traces, an assignment that has proven to be invaluable (see Appendix A for assignment prompt). During this library session, we discuss how to do bibliographic traces, including exploring unique database features that facilitate tracing. We also discuss why we do bibliographic traces, a conversation that feeds directly into the course goal of helping students conduct sophisticated, effective research. Students then spend the rest of class conducting a bibliographic trace on a scholarly article they are already planning to use for their annotated bibliography. At the conclusion of the class we discuss the successes and missteps students encounter while doing the trace. In their narrative self-assessments, students uniformly comment positively on the bibliographic trace assignment; they often remark that they had never thought to search in such a way and are often pleased by the depth of research they are able to collect using this method. The trace also gives them a better understanding of the scope of the literature in their field. Several students from previous semesters have also spontaneously approached J. Gilbert to describe how they have successfully used this technique in other classes.
Other lab assignments expose students to unique forms of information used in political science research, such as public opinion data, statistical sources, and government documents. Assignments for those classes generally involve in-class exercises that ask students to explore their topics through various information sources. Although the structure of the lab changes from semester to semester, depending on the course instructor's focus and the constraints of the academic calendar, the lab generally covers the topics in the order described in table 1.
Library Lab Weekly Overview
FINDINGS
Our research design allows us to assess research skills and perceptions for three sets of students: the control group with no library instruction sessions (fall 2006), students who received the four-session instruction arc (three semesters, spring 2007 through spring 2008), and students whose course included the full semester lab component (four semesters, fall 2008 through spring 2010). The narrative self-assessments described previously were only assigned to lab students; all other assessment tools discussed below are virtually identical across the semesters of this study.
As noted previously, we hypothesize that students who experience more lab sessions should have a broader and deeper understanding of the resources they could potentially use for research, should choose higher quality sources more often, and should demonstrate greater confidence in their research skills. Lab students should reflect these improvements and greater confidence even more than students who had four library sessions.
Table 2 presents evidence to support these initial hypotheses. The first set of data in the table gives a broad sense of what resources students use in completing course papers. Patterns differ across the three groups, revealing the effects of having any library sessions—three times more utilization of the reference librarians and the unique effects of the full-semester lab component: lab students are more likely to use an Internet search engine, the course website, and the course professor as resources; two-thirds of lab students use at least four resources (out of seven possible), nearly doubling the percentages of the other two groups; and lab students on average use more total resources (3.84 resources, versus less than 3.4 for the other two sets of students).
Research Behavior of Analyzing Politics Students (Percentages unless otherwise indicated)
Note: Significance tests used are chi-square (percentages) or difference of means (means);
* = p < .10;
** = p < .05;
*** = p < .01.
One byproduct of greater student exposure to research resources should be greater awareness of what resources exist to help students in their research process. Table 2 indicates that students with multiple library sessions use more forms of assistance than students with no library sessions, and far fewer multiple-session students report using no forms of assistance. Most strikingly, visiting the reference librarian for help is much more common among multiple-session students, who are 2.5 times more likely to seek help at the reference desk, with lab students the most likely to do so. Multiple-session students also ask the course professor for assistance more often. Greater resource use in general, plus greater use of assistance in the research process, probably accounts for the fact that lab students spend more time on average searching for annotated bibliography sources. The data also suggest that the control group students, without having any specific library instruction, require more time to find what they needed; four-session students spend less time overall than any other group, and lab students seek a greater range of resources, thus spending the most time of any group.
Finally, table 2 shows that multiple instruction sessions, whether for four weeks or 12, have little effect on some aspects of student research behavior. The survey asked students to identify the first place they began when searching for annotated bibliography sources. Although we might have expected lab students to use Internet search engines less often and library databases more often as starting points, the differences are modest at best and none are statistically significant.
Table 3 presents our analysis of annotated bibliography source quality across the three sets of students. The assignment prompt required a minimum of six sources, with six to 10 as a suggested range. Multiple-session students use more total sources on average, with lab students using the most total sources. Most important, the total number of high-quality sources is far higher among lab students—97% of all sources used by lab students are high quality, well above the other two groups. This table offers powerful confirmation that multiple lab sessions improve the quality of student research resources, and the semester-long lab component enhances use of high-quality sources even more.
Annotated Bibliography Source Usage, Analyzing Politics Students
Note: Significance tests used are chi-square (percentages) or difference of means (means);
* = p < .10;
** = p < .05;
*** = p < .01. HQ = high-quality source
Table 4 addresses our final hypothesis, that lab students would demonstrate greater confidence in their research skills. Table 4 indicates clearly that students in all groups have a high degree of confidence in their abilities by the end of Analyzing Politics. A beginning-of-semester assessment given to a subset of multiple-session students reveals that at least three-quarters consider themselves to be at least confident when they begin the course, with 15% to 20% feeling “not very confident” (pretest results not reported in table 4); hence high confidence at the end of the course is no surprise. Lab students are more likely than other students to place themselves at the top of this scale, “very confident” in their research skills. We might have hoped for a different outcome—perhaps greater exposure to more types of resources for political science research would lead lab students to realize that they could not hope to master all of this in one semester. Instead, more exposure boosts confidence to extremely high levels.
End-of-Semester Perceptions of Confidence in Research Skills, Analyzing Politics Students (Percentages unless otherwise indicated)
Note: Significance test is chi-square, p < .214.
IMPLEMENTING A LIBRARY LAB
Adding multiple instruction sessions to a research methods course correlates with, and probably helps lead to, positive changes in student research practices and student perceptions of their research abilities. The fact that lab students show even more positive changes than students who received four sessions indicates clearly that the number of sessions matters: more sessions do make a difference by giving students more exposure to more forms of library and Internet research materials, as well as more practice in accessing and evaluating resources. Devoting one-fourth of scheduled course instruction time to the library lab component is a significant commitment that pays off for students.
We also believe, based on the assessment data and our own perceptions in teaching this course, that integration matters, too. The more closely individual lab sessions are connected with course learning objectives and assignments, the more likely students are to take the lab work seriously. This level of integration has led to positive changes in student behaviors with two different political science professors teaching the course, which indicates that multiple professor-librarian pairings can be successful in using this approach. Integration also extends to classroom pedagogy; active learning is emphasized in all nonlibrary class periods of Analyzing Politics, hence students experience consistent pedagogical approaches when working on the active learning exercises in the lab sessions.
Successfully integrating the course objectives of two different faculty members is never a given, of course. Library and political science faculty responsible for teaching the course meet prior to the start of each semester to outline course goals and assignments, including how the library lab component will support the larger course learning outcomes. We also meet at the end of the semester to debrief, evaluate assessment findings, and plan for the following semester. One significant change resulting from this process involves the in-class library exercises. Students have commented that although they understand the purpose of the assignments, they find them to be repetitive and often feel like busywork. We have addressed these challenges by having both the librarian and the course instructor grade library assignments, with the librarian commenting primarily on student research processes. We have also streamlined some library assignments; for example, we combine two or three smaller assignments into a larger one and ask students to complete research logs describing steps they take while researching topics.
Effective courses are always works in progress. Analyzing Politics has improved with each iteration of the library lab component, and the course and lab will evolve as we refine and expand our understanding of the best practices that meet our students' information literacy needs. Our experience with the integrated lab component provides a model that could readily be adapted to the unique needs of students at other institutions. Assessment data show a consistent enhancement of student skills, and one that promises to improve student learning far beyond the single semester covered by this course.
Appendix A:
Bibliographic Trace Assignment
Bibliographic traces are an important part of the literature review because they help you determine the significant research in the discipline and become acquainted with related literature.
• Use the article you brought to class to complete this assignment. I will collect your assignments at the end of class and grade them.
1. Provide a complete citation for your article using APSA format (use the Cite Your Sources link on the homepage for help):
2. Let's find out who else has used your article in their research. Do a cited reference search in Academic Search Premier. (HINT: Look for the Cited References search option in the Advanced Search tab.)
How many times was your article cited? (If it wasn't cited at all, try a source from your article's references list until you find one that has been cited.)
Does the number of times an article was cited have any bearing on its significance to the literature? Why or why not? What else might affect the number of times an article has been cited?
3. Now repeat your search in Social Sciences Citation Index. Click the Web of Science tab and use the Cited References Search. (HINT: Try a cited author search, being careful to enter the name of the author as shown by the example. Enter the date of the article in the Cited Year(s) search box to limit the search.)
Select the records you want to examine by checking the boxes on the left-hand side. Be sure the Cited Work column matches the journal from your article citation. Click Finish Search.
Did you get different results from your previous search using Academic Search Premier? If so, why do you think this happened?
4. Repeat your search in Google Scholar using the same article. (HINT: Search for the article itself to see if anyone's cited it.)
Did you find citing articles? Are they different from results you found in other databases? Why might that be?
5. Return to your original article. Identify the three or four most significant sources that your article cites. (Jot down the article title and author.) Why did you determine that these were the most significant sources? Provide two or three reasons.
6. Describe how doing this initial bibliographic trace has (or has not) shaped your research for your topic.
