Editors' Commentary

This chapter illuminates some of the research design issues discussed in Chapter 3. It shows how the authors developed and piloted a novel intervention, visual cues, during one semester and fully implemented and assessed it in another. The methodology involved two sections of the same course, taught by the same instructor, one as an experimental group and the other as control group. Because using visual cues to assist with some computational skills was specific and limited in scope, many of the concerns mentioned about using control groups in Chapter 3 did not arise. The study involved similar interventions with visual cues in two different settings (first in remedial math, then in calculus). For calculus, the authors discovered that they had to alter their rubric to capture the information they wanted.

Introduction

The large number of students who must take remedial mathematics courses is a problem for many colleges and universities. The courses must be staffed, and students who place into remedial courses are less likely to successfully complete a degree (Attewell, Lavin, Domina, & Levey, 2006). Remediation has been a problem for decades (see, for instance, Handel & Williams, 2011; Patterson & Sallee, 1986; Weiss & Nguyen, 1998). In many lower division mathematics courses the focus of instruction is often on procedural skills. Students should view these courses as an opportunity to improve their skills, but they often see them as hurdles that do not count towards degree completion. The material in remedial courses is often familiar to students, which can make it more difficult for students to expend the time and effort required to correct previous misconceptions. Conversations about our frustrations with these courses led us to consider what we could do to be more effective in teaching procedural skills. The resulting investigation took the form of a What works? study.

We began with an intermediate algebra course, as it involves procedural skills that students have encountered previously. In investigating a What works? question, there are many possible approaches (Patterson & Sallee, 1986).

Editors' Commentary

In this chapter Mike Axtell and William Turner describe how they went about undertaking, as novices, a literature review in mathematics education. Their experience revealed to them the critical role that the literature review can play in refining a SoTL research question and how it can aid in designing a study. Readers may want to contrast their study of reading questions with that written by Derek Bruff in the preceding chapter.

The Backstory

We begin by providing the background of our investigation. We describe what motivated us to use pre-class reading assignments and how over time we developed a system that involved not just assigned readings but reading questions (RQs) as well. We explain why we decided to investigate their use and what it was that we wanted to know, at first.

The process of reading a mathematical text and understanding it is complicated and difficult (Konior, 1993; Reiter, 1998). It is an acquired skill that few, if any, come by naturally. However, it is one that mathematicians must learn. Mathematics conferences and pedagogical publications often contain ideas on how to get, or teach, students to read mathematics (Amick, 1997; Gold, 1998; King, 2001; Ratliff, 1998; Reiter, 1998; Taalman, 1998), and there are hundreds of references on how to help students improve their critical reading skills (Bratina & Lipkin, 2003). The goal is for students to become independent learners capable of teaching themselves, perhaps the ultimate goal of any liberal education program.

If we as teachers are convinced of the need to get students to read a mathematical text, we should then be concerned with determining what our students are gaining, and not gaining, from this task. If our strategies are not leading to desired outcomes, then we should rethink them. This observation led us to collaborate on a SoTL project during the 2005-2006 academic year that focused on student reading.

Introduction

SoTL involves the systematic investigation of a question we have about student learning and we look for answers in evidence generated by students. After framing a researchable question, we have to gather and analyze evidence. So this chapter examines some basic considerations of research design, such as whether, and how, to gather quantitative data, qualitative data, or both. It is likely that one or more of the types of evidence discussed in this chapter will be unfamiliar to mathematicians. Many of them were new to Curtis Bennett and me as well, when we began doing SoTL. In this chapter I write about these methods from our experience in learning to use them.

Triangulating Data

A SoTL researcher should develop a plan for systematically collecting multiple types of evidence. A diversity of evidence can help the researcher to form a convincing picture of student learning (Wiggins, 1998). This approach is called triangulation of the data. According to Webb, Campbell, Schwartz, and Sechrest (as cited in Shavelson & Towne, 2002, p. 64): “When a hypothesis can survive the confrontation of a series of complementary methods of testing, it contains a degree of validity unattainable by one tested within the more constricted framework of a single method.” In other words, claims or explanations supported by several types of evidence-for example, student work samples, interviews, and retention rates-are considered to be more accurate. This will be an asset if the work is submitted for publication in a peer-reviewed journal. We turn now to a discussion of the difficulties in approaching SoTL as standard educational research.

Challenges that Educational Research Design Presents for SoTL

Designing an educational research study can pose a number of challenges to faculty members interested in SoTL. Many of the same challenges confront mathematics education research at all levels. In response to a query from the NSF, and with its funding, the American Statistical Association held a series of workshops for statisticians and mathematics education researchers to discuss whether the statistics community could “offer any contributions to improving the quality of mathematics education research” (Working Group on Statistics in Mathematics Education Research, 2007, p. 1).

Introduction

This chapter, which closes Part I, offers additional resources and advice for completing a SoTL project. These include the need to obtain human subjects clearance in order to publish the results of the study, why it is a good idea to find collaborators for doing SoTL, and where to find them. Suggestions are offered for other sources of support and possible venues for dissemination. As in the previous chapters, the work of the authors in Part II provides examples.

Human Subjects Considerations

At the outset of a SoTL investigation, if the goal is to publish the results, then human subjects issues will arise, which will be unfamiliar to many mathematicians. According to United States Federal Guidelines, a human subject is a person about whom an investigator (whether professional or student) conducting research obtains data through intervention or interaction with the individual or identifiable private information (32 CFR 219.102.f). Because of past abuses of human subjects in medical trials in populations such as prisoners or minorities in the armed forces, the federal government has developed procedures requiring informed consent for human subjects research (U.S. Department of Health and Human Services, n.d.). Special rules for obtaining informed consent apply to any subject under the age of 18, a situation that can be encountered in SoTL studies of first-year college courses.

Because SoTL publications may involve making the work of our students public, we must follow institutional guidelines for working with human subjects. Most colleges and universities have a committee or group, often called an Institutional Review Board (IRB) or a Human Subjects Review Board, charged with ensuring that the federal guidelines are observed.

Human subjects researchers are expected to inform their subjects of the risks of their involvement in the study and obtain written consent for their participation. Studies that involve little or no physical or emotional risk to the subject, and will not reveal anything about the subject's behavior that would be damaging if it became public, may be exempted from obtaining written consent.

Editors' Commentary

In this chapter Rann Bar-On, Jack Bookman, Benjamin Cooke, Donna Hall, and Sarah Schott describe how one faculty member's attempt to improve student success in a special freshmen calculus sequence for underprepared students evolved into scholarship of teaching and learning. Key to this progression was collaboration with academic support professionals and non-tenure track faculty. Thoughtful discussions, a few trial interventions, and examining the research literature enabled the group to move from reflection and experimentation to scholarly teaching and then to the scholarship of teaching and learning. After several years of collaborative effort, a grant application to further develop, study, and share the results of this work was submitted to the NSF.

Introduction

Through the efforts of the Carnegie Foundation and others, many faculty are introduced to SoTL through conferences, workshops at their institution, colleagues, or journal articles devoted to SoTL (Hutchings, 2010). As discussed in this volume, these activities have struck a chord with, and given voice to, the scholarly and intellectual interests of many faculty in higher education. The growth of SoTL has provided validation and motivation for faculty to develop SoTL projects. In this chapter, we describe a different introduction to SoTL, one that is more unintentional and less self-conscious. We will discuss how a small group of faculty faced with an instructional problem gradually adopted an increasingly scholarly approach to addressing it. Instead of scholars creating solutions to problems, in this case a problem created scholars.

We were all exposed to SoTL work early in our careers. We mostly set it aside as we pursued our teaching. We rediscovered it when the need arose, and we wanted to stand on the shoulders of those who had attempted to address the problems we encountered. We believe that our experience can be instructive for those who are new to SoTL and for those who have had some involvement with it.

Editors' Commentary

In this chapter Cindy Kaus discusses a SoTL project that grew out of her involvement with a national initiative to incorporate civic engagement into the teaching of science and mathematics. She called upon SoTL to provide assessment for the effectiveness of her course redesign. The chapter considers a common problem in doing SoTL, namely encountering difficulties in getting comparison data from control groups taught by other faculty members even when they are willing to assist. The author also describes the professional connections and benefits that accrued to her from employing SoTL to investigate student learning.

Introduction

The relationship between successful mathematics course completion and degree attainment in higher education is significant (Adelman, 2006, 2009). Hence, the low graduation rates in higher education institutions (Callan, 2008) indicate a need for a more effective and engaging mathematics curriculum. In addition, as more mathematically underprepared students enter higher education (Parsad & Lewis, 2003), engaging students and increasing retention rates in mathematics courses become greater challenges. Low completion rates in general education mathematics courses at Metropolitan State University, a comprehensive public university in St. Paul, Minnesota, led to an investigation of how civic engagement could be used in a statistics course to increase retention and students' interest and confidence in their ability to do mathematics.

Metropolitan State University is an urban institution serving the Twin Cities of Minnesota. The student population is the most diverse in the higher education system in Minnesota. Students of color make up 34% of its student body. The university was founded on principles of connecting higher education with surrounding communities. The idea to incorporate community-based projects into an introductory statistics course emerged as a result of the university's founding principles and also from attending the 2006 SENCER Summer Institute. SENCER, which stands for Science Education for New Civic Engagements and Responsibilities, is an NSF-funded curriculum and dissemination project. It aims to improve collegiate instruction in STEM fields by promoting the teaching of science and mathematics through complex, real world problems.

The scholarship of teaching and learning is a growing field of inquiry in which faculty members bring disciplinary knowledge to investigate questions of teaching and learning and systematically gather evidence to support their conclusions. Submitting the results to peer review and making them public for others to build on have generally become expected components of the scholarship of teaching and learning, or SoTL. Individual faculty members, their students, departments, and institutions can all benefit from this work. As one of the contributors to this volume (Edwin Herman, Chapter 8) observes: “The process of doing SoTL research can be even more important than the results obtained. Framing and researching a question and designing a project encourages the researcher to experiment within the classroom, much as a painter experiments with styles on a canvas. As the project progresses, the question (or questions) become more refined, more interesting, and the answers can both inform and improve the way you teach” (p. 83).

This Notes volume is written for collegiate mathematics instructors who want to know more about conducting scholarly investigations into their teaching and their students' learning. Faculty members in related disciplines, such as engineering, computer science, or the sciences should also find the book of interest, as should high school mathematics teachers. Conceived and edited by two mathematics faculty members, the volume serves as a how-to guide for doing SoTL in mathematics. It contains information and resources for undertaking scholarly investigations into teaching and learning, and includes many examples.

SoTL is a topic of increasing interest in the mathematics community. Well-attended MAA minicourses on how to get started in SoTL have been presented at the 2006, 2007, and 2008 Joint Mathematics Meetings and at the 2009 MathFest. Successful SoTL paper sessions have been offered at the joint mathematics meetings annually since 2007. Project NExT fellows have shown interest in the topic by inviting speakers to address SoTL on panels they organized for the 2009, 2011, 2012, and 2013 Joint Mathematics Meetings.

Editors' Commentary

In this chapter, Rikki Wagstrom describes how she applied SoTL processes to aid in the development and evaluation of a new curriculum that integrated civic issues into a prerequisite course for college algebra. Her experience illustrates how it can take a long time to identify and frame an appropriate research question. She describes searching the literature and tells how it led her to a useful model, one that prompted her to change the site of her investigation and revise her research question. She provides insights into the problems that can arise in finding faculty members to teach experimental and control sections, and the tough decisions that have to be made about how much data to collect.

Introduction

My SoTL journey originated with the 2006 Summer Institute sponsored by SENCER (Science Education for New Civic Engagements and Responsibilities, see sencer.net). SENCER, a National Science Foundation funded program, supports mathematics and science faculty in creating or modifying curriculum, pedagogies, and courses to integrate civic issues. When I attended the institute, the courses already developed through SENCER were mostly in the sciences; only a few were in mathematics. I returned to Metropolitan State University in St. Paul, Minnesota, inspired to design a SENCER mathematics course and curious about how using the SENCER approach would affect introductory-level courses such as developmental mathematics and college algebra, where low retention rates are a problem.

When I returned for the 2007 SENCER Summer Institute, I participated in a pre-institute workshop on the Scholarship of Teaching and Learning. Workshop participants began to develop a SoTL research question to investigate during the coming year and to report on the following summer. I started with many questions: Would teaching college algebra through a civic issue generate interest in mathematics? Would it improve students' ability to think about and use algebra sensibly? Would it better prepare them for future mathematics courses? To launch my SoTL project, I settled on the question: How does teaching algebra through civic issues affect students' abilities to apply appropriate mathematical arguments or tools to mathematics-related problems arising in their lives?

Editors' Commentary

In this chapter a group at Illinois State University describe how they used the scholarship of teaching and learning to investigate whether having preservice teachers participate in a mathematical research experience for undergraduates (REU) program influenced their beliefs about teaching and learning mathematics. Unable to find an appropriate survey instrument, they developed their own. They explain the organization and content of the survey, tell how they piloted and tested it for reliability, and describe how the results are being used to improve the REU program.

Introduction

The Scholarship of Teaching and Learning (SoTL) can serve as a critical tool for improving instruction at the post-secondary level. If we, mathematicians and mathematics educators, can systematically explore, share, and reflect on our understanding of quality collegiate instruction, improvement in overall course quality and student learning can follow. The multifaceted nature of student learning presents a challenge for any educational study. One such facet is student beliefs. For example, an effective instructional strategy for teaching an honors calculus course for mathematics majors may not translate to a calculus course designed for business majors because of the different attitudes and beliefs of these two groups. Differences in students' views of the nature of mathematics, the importance of building mathematical understanding, the beauty of mathematics, or the standards for reasoning and proof, all can influence the success of instruction. In this chapter, we describe a SoTL study designed to inform and improve a mathematics education program that engages future secondary teachers in authentic mathematics research experiences. One of the goals of the program, the REU Site: Mathematics Research Experience for Pre-service and In-service Teachers (REU), is to change teachers' beliefs about mathematics and about the teaching and learning of mathematics and thus to influence the way they teach.

Impact of Beliefs on Secondary Mathematics Teachers

Numerous researchers have cited the impact of beliefs on teaching and learning (Ernest, 1989; Philipp, 2007; Schoenfeld, 1985; Silver, 1985; Thompson, 1992).

Editors' Commentary

The primary goal for this volume is to provide guidance for mathematics faculty members interested in undertaking a scholarly study of their teaching practice, but a secondary goal is to promote a greater understanding of this work and its value to the mathematics community. In this chapter we reflect on the value of SoTL, generally, and take stock of the outcomes and benefits that accrued to the 25 contributing authors as a result of their scholarly inquiries into teaching and learning.

Introduction

In 1999, Lee Shulman, then President of the Carnegie Foundation for the Advancement of Teaching, wrote entertainingly and perceptively about what it looks like when learning does not go well (Shulman, 1999). He coined a “taxonomy of pedago-pathology” consisting of amnesia, fantasia, and inertia. According to Shulman, amnesia refers to students forgetting, as a matter of course, what they learned. He joked that sometimes they even forget that they attended some classes. Fantasia denotes persistent misconceptions, where students are unaware that they misunderstand. Finally, inertia signifies that students are unable to use what they learned.

We suggest that teaching possesses similar pathologies. Amnesia is a good label for the many things about our teaching we forget from one semester to the next, things that went well and things that didn't, even when we are teaching the same course. In fact, Shulman called this “pedagogical amnesia” (as cited in Hutchings, 1998, p.17). Without observing and collecting data, we have little evidence or direction for improving and are unlikely to learn from our mistakes. The remainder of the taxonomy also transfers. Fantasia refers to our misconceptions about what students bring to class, think, learn, find difficult, or don't understand. Inertia signifies that we continue to teach as we have in the past independent of whether or not students are learning all that we want them to learn.

The chapters in Part II showed how the scholarship of teaching and learning helps instructors escape these pathologies.