Evaluating Impact: A Comparison of Learning Experiences and Outcomes of Students Completing A Traditional Versus Multidisciplinary, Project-Based Introductory Statistics Course
Dierker, Lisa1*
Flaming, Kristin2
Cooper, Jennifer L.3
Singer-Freeman, Karen4
Germano, Kaori5
Rose, Jennifer6
1,3,6Wesleyan University, USA |
AbstractThe teaching of statistics is limited by numerous challenges that are not easily overcome with traditional pedagogical approaches. To evaluate the potential impact of a multidisciplinary, project-based course in introductory statistics, this article examines differences in course experiences, confidence, and interest in future experiences with data analysis and applied statistics of students enrolling in a traditional introductory statistics course compared to a multidisciplinary, project-based course. Results demonstrated that students enrolled in the project-based course had more positive course experiences, showed a greater likelihood of increases in confidence in managing data (e.g. setting aside missing data, creating scales and/or dichotomizing variables), choosing the correct statistical test, and writing syntax or code to run statistical analyses. They also showed greater interest in pursuing additional course-work in statistics and related topics. Findings were not better accounted for by differences in background characteristics of the students enrolled in the traditional vs. project-based course. |
Licensed: |
|
Keywords: |
|
| (* Corresponding Author) |
1. Introduction
A central challenge of introductory statistics courses is the development of a curriculum that not only serves diverse students, but also sparks communication, reasoning, and collaboration that crosses traditional disciplinary boundaries. Many students express negative feelings about introductory statistics courses and emerge from these classes with few useful skills (Gal & Ginsburg, 1994). Petocz and Reid (2005) interviewed undergraduate students who had recently completed an introductory statistics class. They found that students who described the class as focusing on statistical techniques did not believe that statistical analyses were a useful way to evaluate information and planned to avoid the use of statistics in their professional lives. In contrast, students who described the class as focusing on analyzing and interpreting data were more likely to believe that statistical techniques could be used to understand real issues in a wide range of areas. Bailey, Spence, and Sinn (2013) suggest that developing this type of statistical thinking can best be achieved through inquiry-based projects. Inquiry-based projects allow students to, “decompose their topic, identify key components; abstract and formulate different strategies for addressing their research question; connect the original question to the statistical framework; choose and apply methods; reconcile the limitations of the solution; and communicate findings” (Nolan & Temple, 2009).
This type of project-based learning is most commonly defined as an instructional approach based on authentic, real-world activities that are aimed at engaging student interest and enthusiasm (Buck Institute for Education (BIE), 2012; Krajcik & Blumenfeld, 2006). Designed to answer a question or solve a problem, project-based learning allows students to face challenges that lead to answers, reflect on ideas, and make decisions that affect project outcomes (Aditomo, Goodyear, Bliuc, & Ellis, 2013). There is an emerging literature demonstrating that project-based learning is more effective in promoting deep thinking— the ability to apply knowledge, communication, and reasoning skills— when compared to traditional didactic approaches (e.g. (Harada & Yoshina, 2004; Hickey, Kindfield, Horwitz, & Christie, 1999; Hickey, Wolfe, & Kindfield, 2000; Hmelo-Silver, Duncan, & Chinn, 2007; Kirschner, Sweller, & Clark, 2006; Langer, 2001; Lynch, Kuipers, Pyke, & Szesze, 2005; Walker & Leary, 2009)).
In previous publications, we described the development of a multidisciplinary, project-based introductory statistics course aimed at engaging students in applied statistical projects across both divisional and departmental boundaries (Dierker, Kaparakis, Rose, Selya, & Beveridge, 2012). Funded by the National Science Foundation and first introduced into the curriculum at a selective liberal arts college, the project-based course follows each of the recommendations outlined in the Guidelines for Assessment and Instruction in Statistics Education (GAISE) report (Aliaga et al., 2005; American Statistical Association, 2014; Carver et al., 2016). For example, the course teaches statistical thinking (GAISE recommendation #1) by asking students to choose their own research project giving them time to think critically about statistical issues (Chance, 2002; Wild & Pfannkuch, 1999) recognize the usefulness of data for answering questions of interest to them and to society (Neumann, Hood, & Neumann, 2013) tackle complicated real-world questions that involve more than one or two variables (De Veaux, 2015) and emphasize practical problem-solving skills that are necessary to answer statistical questions (Garfield, Delmas, & Zieffler, 2012).
Further, by supporting students to make decisions about the most appropriate ways to visualize, explore, and, ultimately, analyze data the course also emphasizes conceptual understanding over hand calculations (GAISE recommendation #2). Rather than focusing on rules associated with traditional lists of statistical tools (e.g., z-test, one sample t-test, two sample t-test, paired t-test, etc.), we have organized the course to focus on the decisions and skills involved in statistical inquiry (e.g. selection and interpretation of tests). Basic themes such as measurement and descriptive and graphical representation are covered, as well as more specific inferential methods needed to test hypotheses and/or explore the empirical structure of data (Cobb, 2007; Gould, 2010; Horton, 2015). All, however, are introduced as the students’ scientific questions dictate. In this way, students are provided with opportunities to learn to select the tools that are most appropriate to address their research question(s) and to engage in statistical decision making. Using leading statistical software including SAS, R, Stata, Python or SPSS, (GAISE recommendation #5), students are exposed to a wide variety of statistical methods and learn to choose and use them flexibly as they are needed.
The project-based course uses a flipped classroom approach, which is an effective means of freeing class time for active problem solving (GAISE recommendation #4) because students view lecture videos outside of class (Bishop & Verleger, 2013; DeLozier & Rhodes, 2016; O'Flaherty & Phillips, 2015; Wilson, 2013; Winquist & Carlson, 2014; Yarbo, Arfstrom, McKnight, & McKnight, 2014). In our flipped project-based class, the majority of each class session is devoted to students actively working with instructors and peer mentors on their analyses and thinking about their data (Freeman et al., 2014). In the first week, students develop their own research question after an introduction to a number of large data sets representing different disciplines such as ecology, psychology, economics, and planetary science (GAISE recommendation #3). The students’ research questions evolve as they continue through the course and apply newly learned statistical techniques. All statistical analyses are done within the context of the students’ research question culminating with in a poster presentation. At this concluding event, students have the opportunity to describe to peers, instructors, and external experts their process of inquiry, including the different decisions made along the way and their premises, conclusions and any barriers that they faced. The project-based course is designed to take advantage of students' natural curiosity and provide a common language for approaching questions across numerous disciplines (Cobb, 2007).
In previous work, we have demonstrated that the project-based course has enrolled higher numbers of under-represented minority (URM) students compared to a traditional introductory statistics course(Dierker, Cooper, Selya, Alexander, & Rose, 2015). Further, because of our focus on programming in the context of data analysis (i.e. R, SAS, Stata, Python, etc.), we have compared enrollment of the project-based course to traditional introductory programming experiences, revealing higher rates of female and URM enrollment compared to both a general introductory programming course and an introductory course representing a gate to the computer science major (Cooper & Dierker, 2017). Overall, when compared to both traditional introductory statistics and introductory programming courses, this project-based course attracted students from a much wider range of academic backgrounds as measured by their Math SAT scores (Cooper & Dierker, 2017; Dierker et al., 2015). Despite these promising findings with regard to course access, we have yet to evaluate the possible impact of the project-based course on increased confidence in applied statistical skills, improved attitudes and interest in future exposure to statistics and related coursework. Students who feel negatively about their experience studying statistics are less likely to gravitate towards work in fields that require statistical literacy and are less likely to make use of statistical information to support evidence-based practice (Petocz & Reid, 2005).
The present paper compares background characteristics, learning experiences and course outcomes for students completing the project-based course compared to those completing a traditional introductory statistics course offered through the math department. Specifically, we hypothesized that students enrolled in the project-based course would have more positive course experiences than those enrolled in the traditional course including being able to better understand information presented, engaging in greater preparation for class sessions, increased effort, finding the course more interesting and useful and gauging its reward and feelings of accomplishment as higher than students enrolled in the traditional course. We also hypothesized that project-based students would be more likely to show increases in confidence in a variety of data analysis and statistical skills and that interest in future data analysis and applied statistics experiences would be more likely to increase. We also set out to establish that differences in course experiences, increased confidence and future interest in data analysis and applied statistics would remain after controlling for differences in the background characteristics of students enrolling in each course.
2. Method
2.1. Sampling Design
The sample was drawn from students enrolled in a traditional statistics course and those enrolled in the multidisciplinary, project-based statistics course. Both were semester-long courses meeting 2 to 3 times a week and were open to all students with no prerequisites for enrollment. The project-based course could be used as one option to fulfill a major requirement for biology, earth and environmental science, government, neuroscience and behavior, or sociology. Both project-based and traditional courses counted toward the major requirements for psychology, but only as alternatives to an introductory statistics course offered directly through the psychology department. Both courses could also be applied to the natural sciences and mathematics general education recommendations. Importantly, neither course was specifically required of any student or represented a single option for fulfilling requirements for any major within the university. The traditional introductory statistics course was selected as a comparison group as, like the project-based course, it was open to all students and not designed for students from only a single major.
2.2. Treatment
The multidisciplinary, project-based introductory statistics course was offered through the Quantitative Analysis Center, a collaborative effort of academic and administrative departments that supports quantitative analysis across the curriculum and provides an institutional framework for collaboration across departments and disciplines in the area of statistics and data analysis. Titled Applied Data Analysis, the course was described in the university’s on-line catalogue as a “project-based course, [in which] you will have the opportunity to answer questions that you feel passionately about through independent research based on existing data. Students will have the opportunity to develop skills in generating testable hypotheses, conducting a literature review, preparing data for analysis, conducting descriptive and inferential statistical analyses, and presenting research findings. The course offers unlimited one-on-one support, ample opportunities to work with other students, and training in the skills required to complete a project of your own design. These skills will prepare you to work in many different research labs across the University that collect empirical data. It is also an opportunity to fulfill an important requirement in several different majors.” The project-based course utilized a free eText providing integration of visuals and text, including appropriate narration in multimedia settings. This sort of text has been shown to promote enhanced learning compared to a text-only presentation (Mayer, 2009).
The traditional introductory statistics course was offered through the math department. Titled Elementary Statistics, it was described in the university’s online course catalogue as “covering the topics of organizing data, central measures, measures of variation, distributions, sampling, estimation, conditional probability (Bayes' theorem), hypothesis testing, simple regression and correlation, and analysis of variation.” The course used a traditional text (Mendenhall, Beaver, & Beaver, 2012) with 75% of in class time devoted to lecture and the remaining 25% spent on group activities often including generating a class data set (e.g. Each student counts the number of heads from 10 coin flips) and then analyzing that data set. Statistical software was not used.
2.3. Instruments
Data were drawn from administrative records and student surveys completed before and after each course. The pre course survey was completed prior to the end of the second week of classes and the post course survey was completed during the last weeks of the semester. Each survey took approximately 10-15 minutes to complete. The final sample, which included students completing both the pre and post course surveys, included 77 students enrolled in the traditional statistics course and 295 students enrolled in the multidisciplinary, project-based course. All students taking the project-based course between fall 2014 and spring 2016 were administered a pre/post survey (12 course sections). For the traditional course, all students enrolled in sections in which the instructor provided permission for their students to participate were also administered the pre/post survey (3 sections). The average enrollment per class section in the multidisciplinary, project-based statistics course was 27.8 students (SD = 5.65, range: 15 to 50) compared to an average of 45 students (SD = 1.7, range = 44 to 47) in the participating sections of the traditional statistics course.
Background Characteristics. Class year was dichotomized into 1st and 2nd year vs. upper classmen and graduate students. A variable categorizing gender was also available from administrative records and course surveys. Reports of race/ethnicity, including White, Black, Hispanic, Asian and Other, were not mutually exclusive. Students also self-reported whether or not they were the first generation of their family to attend college and whether or not they were eligible for free or reduced lunch in high school. Students’ high school backgrounds included whether they attended a public vs. non-public high school. The non-public category included private schools, religious schools, and home-schools. College financial aid status was characterized as students with demonstrated need receiving grants and/or self-help financial aid vs. those enrolled in the university without financial assistance.
Academic background was assessed by questions about previous statistics, mathematics, and programming courses in the pre survey. Students were asked whether they had previously taken a statistics course (advanced placement or non-advanced placement statistics in high school or a college level statistics course). Self-perceived skills in mathematics was measured by the question, “How good at mathematics are you?” on a scale from 1 (very poor) to 5 (very good) and Math SAT scores were drawn from administrative data. Any prior experience with general programming and/or code-based statistical software was evaluated in the pre course survey (e.g., R, SAS, Stata, Matlab or Python, C++, Java, HTML, etc.).
Experiences with the Course. Based on the post course survey, students rated how well they understood the information presented in each type of course resource (i.e., instructor or video lectures, textbook, one-on-one support from instructor and peer mentor) from 1 (none of it) to 5 (all of it). Students’ perceptions of their engagement in the course were measured by questions about their preparation ("How frequently did you prepare by completing assigned materials before class?" on a scale from 1 = never to 5 = always), and their effort ("How hard did you work in this course?" on a scale from 1 = not at all hard to 5 = extremely hard). Students’ perceptions of rigor were measured with the questions “How challenging did you find this course?" (from 1 = not at all to 5 = the most challenging) and “Was this course more challenging, less challenging or similarly challenging compared to other college courses you have taken?” Overall impressions of the success of the course were measured on a scale of 1 (not at all) to 5 (extremely rewarding) with the questions “How rewarding did you find this course?”, “Did you accomplish more than you expected, less than you expected or about the same as you expected?”, “Did you find the course more interesting, less interesting or the same compared to other college courses you have taken?” and “Did you find this course more useful, less useful or similarly useful compared to other college courses you have taken?”. Students were also asked about the amount of individualized support and how useful they perceived that individualized support to be relative to their experiences in other courses. All questions comparing experiences in the statistics course to other college courses were dichotomized as more vs. less or the same.
Increases in Confidence. Increases in perceived confidence on specific data analysis and statistical skills were evaluated based on changes in student ratings from the pre to post course survey, using a scale of 1 (not at all confident) to 4 (very confident). These skills were developing a research question, choosing the correct test, managing data (e.g. setting aside missing data, creating scales and/or dichotomizing variables), checking assumptions, calculating a test statistic by hand, writing syntax to run a statistical analysis, interpreting results, graphing, effectively presenting results, and the more general category of conducting a statistical analysis of data. For students not rating themselves at the maximum confidence level in the pre-survey (leaving 72.3% - 97.1% of the sample), individual dichotomous variables were created for each skill indicating whether or not the student’s confidence increased between the pre and post course surveys. In addition, the total number of these skills on which students increased from pre to post was calculated. An overall confidence question, "How confident are you that you mastered introductory statistics material?” rated from 1 (not at all) to 5 (extremely confident) was also asked in the post survey. Post-test ratings of confidence, from 1 (not at all confident) to 5 (extremely confident), in abilities to learn more statistics, analyze data, answer questions with data and master introductory material were also examined.
Attitudes and Interest in Future Experiences. An increase in students’ interest in conducting researchwas measured by comparing the pre survey and the post survey responses, each measured on a scale of 1 (not at all interested) to 5 (extremely interested). Increases in interest in using statistics were measured by a series of questions on student expectations about using statistics in future employment, interest in pursuing advanced statistics coursework, intention to take more statistics courses in the future, and their expectation of using statistics in the remainder of their degree program. Each of these questions was rated on a 5-point Likert scale in the pre and post surveys with higher values indicating increased interest in using statistics. The question “In the field in which you hope to be employed when you finish school, how much will you use statistics?” was also included, with ratings from 1 (not at all) to 5 (a great deal). For students not rating themselves at the maximum interest level in the pre survey (76% and 96%), dichotomous variables were created for each question indicating whether or not the students’ interest increased between the pre and post surveys.
A final outcome measure considered if students would like to take one or more courses as a follow-up to their statistics course. Options included a course in advanced statistical tools, programming, data set construction, data visualization, science writing, or other. Individual courses were examined separately and an aggregate variable based on post survey data was constructed indicating whether or not a student endorsed an interest in taking at least one follow-up course.
2.4. Analyses
Chi-Square Tests of Independence and ANOVA were used to examine the association between types of course for categorical and continuous variables, respectively, measuring student’s background characteristics, course experiences, increased confidence and future interest in data analysis and applied statistics. As an estimate of effect sizes, odds ratios were calculated for percentage differences based on Chi-Square Tests of Independence and Cohen’s d was calculated for mean differences based on ANOVA. Multivariate linear or logistic regression models, for continuous and binary outcome variables respectively, were then run to evaluate whether differences between courses persisted when controlling for student background characteristics.
3. Results
3.1. Student Characteristics by Course Type
A comparison of demographic and other student background characteristics in the multidisciplinary, project-based course vs. the traditional introductory statistics course is presented in Table 1. A significant association between course type and class status showed higher rates of upperclassman (i.e. juniors, seniors and graduate students) enrolled in the project-based course compared to the traditional introductory statistics course. When examining demographic characteristics, both courses enrolled similar rates of female students; White, Asian and Hispanic students; students from public high schools; those receiving free or reduced cost lunches in high school and those currently receiving financial aid. The project-based course however, enrolled higher rates of Black and first generation college students compared to the traditional statistics course.
While average SAT scores for Math were statistically similar for students in both courses, those students enrolled in the traditional math statistics course rated themselves as better in math than those enrolled in the project-based course. When considering previous experience with statistics courses at the high school or college level, students in the traditional and project-based courses reported these experiences at similar rates. Students in the project-based course were significantly more likely, however, to report some previous experience with programming compared to students in the traditional introductory statistics course.
3.2. Student Experiences by Course Type
Based on post survey responses, a comparison of self-reported experiences in each course is presented in Table 2. Project-based students reported preparing for class by completing assigned materials more frequently (1 = never, 5 = always) than students in the traditional course, though ratings for “How hard did you work in this class?” were found to be statistically similar. Further, while students showed statistically similar ratings of their understanding of the video (project-based students) or lectures (traditional students) and traditional vs. project-based textbook, those students in the project-based course reported understanding significantly more of the information presented through one-on-one support from the instructor and peer mentors than students completing the traditional introductory statistics course. Students in the project-based course also reported receiving significantly more individualized support compared to other courses they had taken, and those in the project-based course were significantly more likely to report that the individualized support was more useful than what they had received in other college courses.
Though students in the project-based course were no more likely than those in the traditional course to find the experience more interesting or challenging than other courses they had taken, project-based students were significantly more likely than traditional students to rate their experience in their statistics course as more useful than other courses. Project-based students were also significantly more likely than traditional students to rate their course as more rewarding than other courses. Finally, project-based students were significantly more likely than traditional students to report accomplishing more than expected in their course.
When controlling individually for student background characteristics that were found to differ by course type (i.e., class year, race/ethnicity, first generation college student status, self-reported ability in math and prior experience with programming), all differences in course experiences between students enrolled in the project-based vs. traditional statistics course remained significant.
3.3. Increased Confidence in Basic and Applied Statistics Skills by Course Type
A comparison of ratings of increased confidence in basic and applied statistics skills are presented in Table 3. Based on responses to both the pre and post course surveys, students completing the project-based and traditional statistics courses reported increased confidence between pre and post course surveys at similar rates when asked about developing a research question, checking whether the assumptions for a statistical test are met, calculating a test statistic by hand, interpreting results, graphing, effectively presenting research results and conducting statistical analysis of data. However, students taking the project-based courses were significantly more likely than students in the traditional course to report an increase in confidence with regard to managing data, choosing the correct statistical test, and writing syntax or code to run statistical analyses. In addition, students in the project-based course had increases in confidence in a larger number of skills than students in the traditional course. No differences were found between project-based and traditional students in terms of their post course ratings of confidence in their ability to learn more statistics, analyze data, or answer questions with data through interpretation of results. Students in both courses also showed statistically similar ratings of their confidence that they had mastered introductory statistics material.
When controlling individually for student background characteristics that were found to differ by course type (i.e., class year, race/ethnicity, first generation college student status, self-reported ability in math and prior experience with programming), all the differences in increased confidence remained significant.3.4. Improved Attitudes and Interest in Future Exposure to Statistics by Course Type
A comparison of improved attitudes and interest in future courses in presented in Table 4. When examining attitudes and interest in future exposure to statistics and related course work, though students completing the project-based and traditional course were no more likely to show increases in expectations of using statistics in future employment, conducting research or a general intention of taking an additional statistics course in the future (though this was marginally significant in favor of project-based students), the project-based course did produce a significantly higher rate of students who increased their interest in pursuing advanced course-work in statistics Figure 1. Though not statistically significant in individual models, more students in the project-based course reported wanting to take each of the related courses that were presented (i.e., advanced statistical tools, constructing data sets, graphing and data visualization and programming), with the exception of a course in science writing. When aggregating across types of potential follow-up courses, those students completing the project-based course were significantly more likely to have reported interest in one or more follow-up courses than those completing the traditional introductory statistics course.
When controlling individually for student background characteristics that were found to differ by course type (i.e., class year, race/ethnicity, first generation college student status, self-reported ability in math and prior experience with programming), differences found in interest in pursuing advanced statistics course work and being interested in at least one follow-up course between students enrolled in the project-based vs. traditional statistics course remained significant.
4. Discussion
4.1. Context
Statistical analysis is a critical component of research across the natural and social sciences and is arguably the most salient point of intersection between diverse disciplines; information on varied topics is constantly communicated through the common language of statistics. Despite its central importance however, the teaching of statistics is limited by numerous challenges that are not easily overcome with traditional pedagogical approaches. Students may enroll in a statistics course believing that it will be a negative experience (Onwuegbuzie, 1997) which in turn may influence their levels of engagement in the course. Similarly, the traditional lecture-style statistics course inhibits instructors’ abilities to work one-on-one with students who may need extra attention to feel engaged in the course. Finding pedagogical approaches to statistics education that engage and educate students is critical (Cobb, 2007). In previous reports we have described the development of this project-based, introductory statistics course funded by the National Science Foundation and its success in attracting higher rates of under-represented students and students from a wide range of academic backgrounds compared to both a traditional statistics curriculum and introductory programming courses (Cobb, 2007; Dierker et al., 2015).
The goal of the present study was to evaluate the success of a project-based statistics course in increasing confidence in applied skills and fostering positive attitudes toward future learning. To that end, survey and administrative data from students enrolled in a project-based course and students enrolled in a traditional statistics course were utilized to evaluate differences in learning experiences and course outcomes. We hypothesized that students completing the project-based course would be more likely than students completing the traditional course to have positive experiences with the course, increase their confidence in applied statistical skills and express greater interest in pursuing additional data analysis and applied statistics experiences in the future.
4.2. Experiences in the Course
Though students in the project-based and traditional courses rated their experiences equally in terms of how hard they worked and how interesting and how challenging they found the course, project-based students showed some signs of increased engagement based on their reports of more frequently preparing for class by completing assigned materials. This is a relevant addition to the literature on both statistics education (Horton, 2015) and flipped classrooms, where one of the reported challenges is having students recognize the need to come to class prepared (McLean, Attardi, Faden, & Goldszmidt, 2016). Our results are similar to those reported by Moraros, Islam, Yu, Banow, and Schindelka (2015) who found that masters-level students in a flipped epidemiology course reported that they studied more frequently for the flipped course than traditional courses. Moraroset al. (2015) hypothesized that the flipped course might evoke higher levels of preparation because students were required to show their level of understanding regularly through quizzes and in-class practice problems. In our project-based course, we allowed unprepared students to watch the video lessons during class time. This ensured that all students viewed the lessons and provided prepared students with increased opportunities to collaborate with their peers, peer mentors, and instructors during class time. We believe that this class policy may have helped to shape prepared behavior without instructor leveled penalties or a reduction in active class time for those students who had prepared.
Notably, though lectures were delivered through different means within the project-based and traditional course sections (i.e. video lectures in the project-based course and instructor led lectures in the traditional course), students showed similar ratings of their understanding of the lecture material, confirming previous literature reviewing different lecture formats available in support of flipped classrooms (DeLozier & Rhodes, 2016). The same was true of ratings of the course textbooks (a traditional text (Mendenhall et al., 2012) vs. a project-based eText) which were not found to promote differences in overall ratings of understanding. Where differences were seen was in the ratings of the amount and usefulness of individualized support. Project-based students were more likely than traditional students to report having received more individualized support compared to their other college courses and that the individualized support was more useful than support received in other college courses. These results are similar to those reported by McGivney-Burelle and Xue (2013) who found that students in a flipped calculus class responded positively to the increased availability of the instructor for individualized help compared to students in a traditional calculus class. Importantly, much of the individual support in the current study was provided by peer mentors. These peer mentors can serve as important role models for students and might influence their interest in pursuing more advanced statistical training. Studies involving diverse groups of undergraduate students have found that mentoring is associated with increases in academic performance and persistence (Crisp & Cruz, 2009).
Importantly, students in the project-based course reported higher levels of understanding of the information they received from individualized support than did students in the traditional course. In a study of statistics anxiety, Williams (2010) found that the physical and psychological availability of an instructor significantly improved students’ feelings about the course. Perhaps, the increased availability of instructors in the project-based course reduced students’ anxiety thereby increasing the extent to which they could benefit from the individualized support that was offered.
Perhaps students’ greater ability to benefit from individualized support can help to explain why project-based students were more likely than traditional students to report accomplishing more than they had expected and that the course was more useful and more rewarding than the other college courses. These results are similar to those found by others who have examined the effects of transforming introductory classes through the inclusion of hands-on, project-based learning. For example, Armbruster, Patel, Johnson, and Weiss (2009) found that students in a biology class with active problem-based learning viewed the class more positively than students in a traditional biology class. Similarly, Wilson (2013) found that students in a flipped section of an undergraduate statistics course received higher grades and expressed higher levels of course satisfaction than those who had completed the traditional version of the class during the previous semester.
Garfield and Ben-Zvi (2007) stress the importance of active student involvement in the construction of statistical knowledge. Students learn information by integrating new types of knowledge with existing knowledge. Students will learn more deeply and integrate new knowledge more thoroughly if they experience and resolve conflict between the new and existing knowledge (Mvududu, 2005). Through the resolution of conflict, the new knowledge becomes firmly integrated with the existing knowledge and the existing knowledge is modified to incorporate the new ideas. Students benefit from struggling to solve conceptually difficult problems and associating these problems with other areas of relevant knowledge. The productive resolution of temporary intellectual roadblocks is a common occurrence in the project-based course. We hypothesize that the successful resolution of these moments of intellectual struggle may have helped students to experience a stronger sense of accomplishment about their work in the class than was experienced by students in the traditional class.
4.3. Increase in Confidence
Given the different amounts of time dedicated to the practice of applied and theoretical skills in the project-based and traditional courses, we were somewhat surprised to find that students reported similar increases in their confidence with both theoretical (checking whether the assumptions for a statistical test are met or calculating a test statistic by hand) and applied skills (developing a research question, graphing, conducting statistical analyses, interpreting results, and effectively presenting results) regardless of their course experience. These findings highlight possible differences between perceived confidence, measured in the present study, and actual achievement in each of these areas. We are currently collecting data that will address this through responses to open-ended questions that ask students to develop a research question based on a given set of variables and then to identify the steps they would take to address this research question. Notably, tests of statistical understanding often fail to address these active, functional skills that are inherent in the intense decision making process of empirical research.
Students’ active engagement in solving a genuine research question may have also boosted their confidence in their practical skills. Increased confidence was more commonly seen among project-based students with regard to some of the more applied skills including managing data (e.g. setting aside missing data, creating scales and/or dichotomizing variables), choosing the correct statistical test and writing syntax or code to run statistical analyses. Though use of statistical software in introductory statistics courses has become far more common over the past decade, these experiences can fall into the category of canned exercises that do not provide students with opportunities to create testable hypotheses, formulate strategies to address them, interpret the meaning of the results in the context of the original question, and communicate the findings and their limitations (Nolan & Temple, 2009).
The project-based interdisciplinary course evaluated in this study was designed in alignment with GAISE guidelines. Students in the project-based course engage in active learning in which they integrate real data with a context and purpose to address a real-world question by using technology to employ statistical thinking. The use of real data is important for the development of a sense of practical confidence because it brings the statistical problem solving that takes place in the classroom into closer alignment with the types of statistical analyses students are likely to encounter in other contexts. The use of real data that concerns real world problems can also support affective engagement by demonstrating the role of statistical analysis in effectively responding to important issues. In addition to reaffirming the six original recommendations for undergraduate statistics education that were articulated in the 2005 report, the 2016 GAISE guidelines add two new areas of emphasis for statistics education: 1) teach statistics as an investigative process of problem-solving and decision making; and 2) give students experience with multivariable thinking. The project-based course provides students with rich opportunities to investigate their own research question using multivariable thinking.
4.4. Future Interest
When examining attitudes and interest in future exposure to statistics and related course work, students completing the project-based and traditional course were no more likely to show increases in expectations of using statistics in future employment, conducting research or their general intention of taking an additional statistics course in the future. However, students in the project-based course were significantly more likely than students in the traditional course to report increased interest in pursuing advanced course-work in statistics or completing one or more follow-up courses. Interest in follow-up courses is a particularly important outcome given the importance of developing deep statistical literacy in order to successfully conduct or comprehend research in the natural and social sciences (Horton, 2015). If students’ intentions accurately predict their future behavior, then participation in the project-based course might lead to higher levels of statistical expertise among students who took the project-based course than students who took the traditional course.
4.5. Limitations
The present results should be interpreted in the context of several limitations. First, identifying appropriate comparison groups in this type of research can be challenging given vast differences across classroom contexts and the fact that random assignment of students to different curricular experiences is often infeasible. Because of this, the present sample is a self-selected, incidental sample of students composing a non-equivalent quasi-experimental design and there was no content or specific methods controlled in these courses. Thus differences may have been influenced by one or more of the many differences between courses (e.g. project-based approach, flipped classroom, use of statistical software, etc.). Furthermore, the measures used in the present analyses represented single items rather than scales or subscales. Given that single items are often less reliable, these findings will need to be replicated within the context of expanded data collection efforts. Finally, while outcomes in self-reported confidence and interest in future data analysis and applied statistics experiences were examined, no outcome measure of achievement was available. We are currently collecting data based on the Assessment Resource Tools for Improving Statistical Thinking (ARTIST) survey at a Midwestern regional university campus and hope to be able to present these findings in a future publication.
5. Conclusions
As expected, students enrolled in the project-based course had more positive course experiences than those enrolled in the traditional course including being able to better understand information presented through one-on-one support from the instructor and peer mentors, engaging in greater preparation for class sessions, finding the course more useful and gauging its reward and feelings of accomplishment more highly. Students enrolled in the project-based course also showed more increases in confidence with regard to managing data, choosing the correct statistical test, and writing syntax or code to run statistical analyses, and they showed greater interest in pursuing additional course-work in statistics and related topics. Importantly, student background characteristics did not explain these differences.
Previous research has recognized the general challenge in teaching courses in statistics at both the undergraduate and graduate level. There is no typical statistics student; instead, students come into statistics courses with differing backgrounds, experiences, learning styles and levels of preparation. The project-based course described in this paper provokes students to encounter (and struggle with) the central concepts and principles not only within the discipline of statistics, but also with the discipline of their research question. Taken together, these findings are especially relevant as the most beneficial effects of project-based learning are often on measures that consider the application, as opposed to the memorization, of knowledge (Dochy, Segers, Van den Bossche, & Gijbels, 2003) and it is potentially these additional application outcomes that are most important to continued persistence in statistics. Given that no single course will adequately prepare students for either the amount or complexity of data they will encounter as professionals and as citizens (Cobb, 2007; Collins & Halverson, 2010; Horton, 2015) modern courses need to focus on imparting a deep interest among students and desire to continue learning statistics and disciplines focused on data and computation. Our approach is currently being used within varied educational settings (i.e. state and regional universities, small liberal-arts colleges, community colleges, high school and youth enrichment programs). We are happy to share our course materials with others and encourage faculty to consider integrating project-based course content (http://passiondrivenstatistics.com).
Project-based Statistics |
Traditional Statistics |
Statistics |
Odds Ratio |
|
N = 295 |
N = 77 |
|||
| Demographics | n (%) |
n (%) |
||
| Junior, Senior, or Graduate student | 180 (61.4%) |
25 (33.8%) |
c2(1) = 17.21, p < .001 |
3.26 |
| First generation student | 50 (17.1%) |
5 (6.7%) |
c2(1) = 4.33, p = .037 |
2.94 |
| Free or reduced lunch in high school | 48 (17.7%) |
11 (16.9%) |
ns |
1.16 |
| Financial aid in college | 141 (48.8%) |
38 (51.4%) |
ns |
0.94 |
| Public high school | 147 (51.9%) |
43 (58.1%) |
ns |
0.79 |
| Gender (% female) | 170 (57.6%) |
48 (62.33%) |
ns |
0.82 |
| White | 179 (60.5%) |
55 (71.4%) |
ns |
0.62 |
| Asian | 66 (22.3%) |
15 (19.5%) |
ns |
1.19 |
| Black | 37 (12.5%) |
3 (3.9%) |
c2(1) = 3.90, p = .048 |
3.54 |
| Hispanic | 26 (8.8%) |
9 (11.7%) |
ns |
0.73 |
| Other | 12 (4.1%) |
1 (1.3%) |
ns |
1.12 |
| Academic Background | ||||
| Previous statistics course | 112 (38.0%) |
26 (34.2%) |
ns |
0.67 |
| Any programming experience | 135 (45.8%) |
22 (28.6%) |
c2(1) = 6.71, p = .01 |
2.11 |
M (SD) |
M (SD) |
Cohen’s d effect size |
||
| Math SAT (n = 180, n = 41) | 718.4 (65.6) |
716.3 (67.3) |
ns |
0.04 |
| How good at mathematics are you? (1 = very poor to 5 = very good) | 3.54 (0.84) |
4.00 (0.63) |
F(1,369) = 19.9, p < .001 |
0.62 |
Note: Percentages are based on the number of respondents completing each item. |
Project-based Statistics |
Traditional Statistics |
Statistics |
Effect Size* |
|
N = 295 |
N = 77 |
|||
| Understanding of Course Resources (1 = none of it, 5 = all of it) |
M (SD) |
M (SD) |
||
| instructor and/or video lectures | 3.68 (0.87) |
3.70 (0.86) |
ns |
0.42 |
| Textbook | 3.98 (0.94) |
3.73 (0.80) |
ns |
0.29 |
| one-on-one help from instructor | 4.32 (0.78) |
3.72 (1.16) |
F(1,354) = 26.0, p < .001 |
0.61 |
| one-on-one help from peer mentor | 4.17 (0.85) |
3.33 (1.34) |
F(1,353) = 41.5, p < .001 |
0.75 |
| Engagement | ||||
| How frequently did you prepare by completing assigned materials before class? (1 = Never, 5 = Always) | 3.85 (0.96) |
3.36 (1.36) |
F(1,356) = 12.2, p < .001 |
0.42 |
| How hard did you work in this class? (1 = not at all, 5 = extremely hard) | 2.75 (0.91) |
3.00 (0.80) |
ns |
0.29 |
| Rigor | ||||
| How challenging did you find this course? (1 = not at all to 5 = the most challenging) | 2.64 (0.8) |
2.77 (0.9) |
ns |
0.15 |
| course was more challenging than other courses, n (%) | 29 (10.0%) |
12 (17.4%) |
ns |
0.59 |
| Overall Impressions | ||||
| How rewarding did you find this course? (1 = not at all, 5 = extremely rewarding), M(SD) | 3.46 (0.95) |
3.10 (1.06) |
F(1,356) = 7.69, p = .006 |
1.01 |
| accomplished more than I expected | 86 (29.8%) |
9 (13.0%) |
c2(1) = 7.15, p = .008 |
3.18 |
| Compared to other college courses | ||||
| course was more interesting | 56 (19.4%) |
10 (14.5%) |
ns |
1.57 |
| course was more useful | 129 (44.6%) |
20 (29.0%) |
c2(1) = 4.99, p = .025 |
2.21 |
| received more individualized support | 190 (65.7%) |
13 (18.8%) |
c2 (1) = 48.02, p < .001 |
8.91 |
| individualized support was more useful | 137 (47.4%) |
12 (17.4%) |
c2(1) = 19.43, p < .001 |
4.70 |
Note: Percentages are based on the number of respondents completing each item. |
Project-based Statistics |
Traditional Statistics |
Statistics |
Effect Size* |
|
| Increased Confidence on Skills | n (%) |
n (%) |
||
| developing a research question | 82 (71.3%) |
16 (57.1%) |
ns |
1.86 |
| managing data | 105 (73.4%) |
14 (46.7%) |
c2(1) = 7.07, p = .008 |
3.16 |
| choosing the correct statistical test | 114 (78.1%) |
18 (54.5%) |
c2(1) = 6.53, p = .01 |
2.97 |
| checking whether the assumptions for a statistical test are met | 96 (66.2%) |
18 (54.5%) |
ns |
1.63 |
| calculating a test statistic such as r (the correlation coefficient) or F (found in ANOVA) by hand | 64 (43.8%) |
18 (56.2%) |
ns |
0.61 |
| writing syntax or code to run a statistical analysis | 120 (80.5%) |
6 (18.8%) |
c2(1) = 44.67, p < .001 |
17.93 |
| interpreting results | 61 (46.2%) |
15 (53.6%) |
ns |
0.74 |
| graphing | 74 (54.8%) |
13 (52.0%) |
ns |
1.12 |
| effectively presenting research results | 57 (50.9%) |
12 (42.9%) |
ns |
1.38 |
| conducting a statistical analysis of data | 104 (72.7%) |
20 (62.5%) |
ns |
1.60 |
| Number of Skills with Increases, M (SD) (N=189) | 5.66 (2.78) |
4.41 (3.36) |
F(1, 187) = 5.19, p = .024 |
0.41 |
| Confidence Levels (N=289) (1 = not at all confident, 5 = extremely confident) |
M (SD) |
M (SD) |
||
| How confident are you of your ability to learn more statistics? | 3.59 (0.93) |
3.41 (1.01) |
ns |
0.19 |
| How confident are you of your ability to analyze data? | 3.53 (0.87) |
3.45 (0.90) |
ns |
0.09 |
| How confident are you of your ability to answer questions with data? | 3.56 (0.87) |
3.49 (0.87) |
ns |
0.08 |
| How confident are you that you mastered introductory statistics material? | 3.49 (0.94) |
3.54 (0.98) |
ns |
0.05 |
*Odds ratios were calculated for percentage differences and Cohen’s d was calculated for mean differences. Ns for Increased Confidence on skills ranged from N=140 (effectively presenting research results) to N=181 (writing syntax or code to run a statistical analysis). |
Project-based Statistics |
Traditional Statistics |
Statistics |
Odds Ratio |
|
N = 289 |
N = 69 |
|||
| Interest in Using Statistics | n (%) |
n (%) |
||
| Increased interest in conducting research |
67 (30.6%) |
22 (40.7%) |
ns |
0.79 |
| Increased expectation of using statistics in future employment | 88 (33.3%) |
19 (32.2%) |
ns |
1.11 |
| increased interest in pursuing advanced statistics coursework | 115 (41.5%) |
17 (26.6%) |
c2(1) = 4.29, p = .038 |
1.61 |
| increased intention to take additional statistics courses in the future | 98 (35.9%) |
15 (23.4%) |
ns |
1.56 |
| Interest in Future Courses | ||||
| advanced statistical tools | 122 (41.4%) |
18 (23.4%) |
ns |
1.62 |
| constructing data sets | 78 (26.4%) |
15 (19.5%) |
ns |
1.24 |
| graphing and data visualization | 122 (41.4%) |
21 (27.3%) |
ns |
1.39 |
| computer programming | 130 (44.1%) |
23 (29.9%) |
ns |
1.35 |
| scientific writing | 67 (22.7%) |
22 (28.6%) |
ns |
0.73 |
| Any | 240 (83.0%) |
47 (68.1%) |
c2(1) = 6.90, p = .009 |
1.22 |
*Odds ratios were calculated for percentage differences. |
Figure-1. Differences in increased interest in pursuing advanced coursework in statistics by course type.
References
Aditomo, A., Goodyear, P., Bliuc, A. M., & Ellis, R. A. (2013). Inquiry-based learning in higher education: Principal forms, educational objectives, and disciplinary variations. Studies in Higher Education, 38(9), 1239-1258.
Aliaga, M., Cuff, C., Garfield, J., Lock, R., Utts, J., & Witmer, J. (2005). Guidelines for assessment and instruction in statistics education (GAISE): College report. American Statistical Association.
American Statistical Association. (2014). Curriculum guidelines for undergraduate programs in statistical science. Retrieved from http://www.amstat.org/education/curriculumguidelines.cfm .
Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered pedagogy improve student attitudes and performance in introductory biology. CBE Life Sciences Education, 8, 203-213. Available at: 10.1187/cbe.09-03-0025.
Bailey, B., Spence, D. J., & Sinn, R. (2013). Implementation of discovery projects in statistics. Journal of Statistics Education, 21(3), 1-24.
Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. Paper presented at the ASEE [American Society for Engineering Education] National Conference Proceedings, Atlanta, GA.
Buck Institute for Education (BIE). (2012). Boise state university, department of educational technology. Retrieved from http://pbl-online.org .
Carver, R., Everson, M., Gabrosek, J., Rowell, G. H., Norton, N., Lock, R., & Wood, B. (2016). Draft: Guidelines for assessment and instruction in statistics education (GAISE) college Report. Retrieved from http://www.amstat.org/education/gaise/collegeupdate/GAISE2016_DRAFT.pdf .
Chance, B. (2002). Components of statistical thinking and implications for instruction and assessment. Journal of Statistics Education, 10(3), 1-14.
Cobb, G. W. (2007). The introductory statistics course: A ptolemaic curriculum? Technology Innovations in Statistics Education, 1(1), 1-15.
Collins, A., & Halverson, R. (2010). The second educational revolution: Rethinking education in the age of technology. Journal of Computer Assisted Learning, 26(1), 18-27.
Cooper, J. L., & Dierker, L. (2017). Increasing exposure to programming: A comparison of demographic characteristics of students enrolled in introductory computer science programming courses vs. A multidisciplinary data analysis course. International Research in Higher Education, 2(1), 92-100.
Crisp, G., & Cruz, I. (2009). Mentoring college students: A critical review of the literature between 1990 and 2007. Research in Higher Education, 50, 525-545. Available at: 10.1007/s11162-009-9130-2.
De Veaux, R. (2015). What’s wrong with Stat 101? Presentation Given at the United States Conference on Teaching Statistics (USCOTS).
DeLozier, S. J., & Rhodes, M. G. (2016). Flipped classrooms: A review of key ideas and recommendations for practice. Educational Psychology Review, 1-11. Available at: 10.1007/s10648-01509356-9.
Dierker, L., Cooper, J., Selya, A., Alexander, J., & Rose, J. (2015). Evaluating access: A comparison of demographic and disciplinary characteristics of students enrolled in a traditional introductory statistics course vs. A multidisciplinary, project-based course. Interdisciplinary Studies of Education 4(1), 22-37.
Dierker, L., Kaparakis, E., Rose, J., Selya, A., & Beveridge, D. (2012). Strength in numbers: A multidisciplinary, project-based approach to introductory statistics education. Journal of Effective Teaching, 12(2), 4-14.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based learning: A meta-analysis. Learning and Instruction, 13(5), 533-568.
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(26), 8410–8415.
Gal, I., & Ginsburg, L. (1994). The role of beliefs and attitudes in learning statistics: Towards and assessment framework. Journal of Statistics Education, 2(2), 3.
Garfield, J., & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistics Review, 75(3), 372-396.
Garfield, J., Delmas, R., & Zieffler, A. (2012). Developing statistical modelers and thinkers in an introductory, tertiary-level statistics course. ZDM: The International Journal on Mathematics Education, 44(7), 883-898.
Gould, R. (2010). Statistics and the modern student. International Statistical Review, 78(2), 297– 315.
Harada, V. H., & Yoshina, J. M. (2004). Moving from rote to inquiry: Creating learning that counts. Library Media Connection, 23(2), 22-24.
Hickey, D. T., Kindfield, A. C., Horwitz, P., & Christie, M. A. (1999). Advancing educational theory by enhancing practice in a technology-supported genetics learning environment. Journal of Education, 181(2), 25-55.
Hickey, D. T., Wolfe, E. W., & Kindfield, A. C. (2000). Assessing learning in a technology-supported genetics environment: Evidential and systemic validity issues. Educational Assessment, 6(3), 155-196.
Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark. Educational Psychologist(42), 99–107. Available at: 10.1080/00461520701263368.
Horton, N. J. (2015). Challenges and opportunities for statistics and statistics education: Looking back, looking forward. The American Statistician, 69(2), 138-145.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.
Krajcik, J. S., & Blumenfeld, P. (2006). Project-based learning. In Sawyer, R. K. (Ed.), the Cambridge Handbook of the Learning Sciences (pp. 317-333.). New York: Cambridge University Press.
Langer, J. A. (2001). Beating the odds: Teaching middle and high school students to read and write well. American Educational Research Journal, 38(4), 837−880.
Lynch, S., Kuipers, J., Pyke, C., & Szesze, M. (2005). Examining the effects of a highly rated science curriculum unit on diverse students: Results from a planning grant. Journal of Research in Science Teaching, 42(8), 912-946.
Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.
McGivney-Burelle, J., & Xue, F. (2013). Flipping calculus. Primus. 23, 477–486. Available at: 10.1080/10511970.2012.757571.
McLean, S., Attardi, S. M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning: Not just surface gains. Advances in Physiology Education, 40(1), 47-55.
Mendenhall, W., Beaver, R. J., & Beaver, B. M. (2012). Introduction to probability and statistics (14th ed.). Belmont, CA: Brooks/Cole: Cengage Learning.
Moraros, J., Islam, A., Yu, S., Banow, R., & Schindelka, B. (2015). Flipping for success: Evaluating the effectiveness of a novel teaching approach in a graduate level setting. BMC Medical Education, 15(27). Available at: http://doi.org/10.1186/s12909-015-0317-2 .
Mvududu, N. (2005). Constructivism in the statistics classroom: From theory to practice. Teaching Statistics, 27(2), 49-54.
Neumann, D., Hood, M., & Neumann, M. (2013). Using real-life data when teaching statistics: Student perceptions of this strategy in an introductory statistics course. Statistics Education Research Journal, 12(2), 59-70.
Nolan, D., & Temple, L. D. (2009). Approaches to broadening the statistics curricula. In M. C. Shelley, L. D. Yore & B. Hand (Eds.), Quality Research in Literacy and Science Education (pp. 357-381). London, UK: Springer.
O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25(C), 85-95.
Onwuegbuzie, A. J. (1997). Writing a research proposal: The role of library anxiety, statistics anxiety, and composition anxiety. Library and Information Science Research, 19(1), 5-33.
Petocz, P., & Reid, A. (2005). Something strange and useless: Service students’ conceptions of statistics, learning statistics and using statistics in their future profession. International Journal of Mathematical Education in Science and Technology, 36(7), 789-800.
Walker, A., & Leary, H. (2009). A problem based learning meta-analysis: Differences across problem types, implementation types, disciplines, and assessment levels. Interdisciplinary Journal of Problem-Based Learning(3), 6–28. Available at: 10.7771/1541-5015.106.
Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67(3), 223-265.
Williams, A. (2010). Statistics anxiety and instructor immediacy. Journal of Statistics Education, 18(2), 1-18.
Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology, 40(3), 193-199.
Winquist, J. R., & Carlson, K. A. (2014). Flipped statistics class results: Better performance than lecture over one year later. Journal of Statistics Education, 22(3), 1-10.
Yarbo, J., Arfstrom, K. M., McKnight, K., & McKnight, P. (2014). Extension of a review of flipped learning. Flipped Learning Network. Retrieved from http://flippedlearning.org/domain/41 .