Abstract:
This work in progress explores 21st Century engagement as an indicator of online instructional quality. How do you move from having a policy of providing quality online learning to actually determining the magnitude, amount, or volume of quality learning experiences provided in a particular course?
Like many institutions providing online learning, we are engaged in a process of exploring the most effective way of improving the quality of courses. Online learning is steeped in the tradition of providing a typical type of experience for learners, with common elements that have become the status quo for over the past 20 years.
The idea is that high quality instruction contains specific elements that will improve the potential for teaching to engage students through socio-emotional, behavioral and cognitive means. These elements have been well researched, mainly in the context of face-to-face primary and secondary education. This study stems from the belief that these elements are just as important to the quality of teaching and learning in an online environment.
Many of us are experienced at offering online learning, and we recognize the need for quality, but are we prepared to assess our courses to see if all the pieces are in place for a 21st Century engaged learning environment? As seen in numerous scholarly publications focusing on contemporary higher education, there are a growing number of academics who are wondering if the quality of online instruction has kept up with its explosive growth.
This work seeks to create a process for evaluating the potential of online course design to engage students through designing for socio/emotional engagement (interactions and reactions), cognitive investment (intellectual), and behavioral engagement (participation).
Keywords: Engaged Learning, 21st Century, Online Learning, Quality, Instructional Design, and Indicators
Introduction
A transformative goal of instructional design would be to create an environment that supports and promotes intrinsic motivation in order to create the optimal learning experience. This study is an opportunity to revisit research-based indicators of socio-emotional, cognitive, and behavioral engagement, as seen in online courses and assignments, in order to evaluate the potential level of student engagement.
Research identifies indicators that can be categorized as Instructional Approach, Teaching, and Learning with observable indicators that are tied to the elements of Socio-emotional, cognitive and behavioral engagement.
Cognitive engagement is described as being thoughtful with an example being when instruction challenges students to draw upon basic skills to engage in higher level thinking and requires students to interact with the curriculum in a deep and thoughtful manner. Socio-emotional engagement is described as being affective, including interactions and reactions with an example being when students have strong ties to the community of learners. Behavioral engagement is described having a high level of class participation and interactions beyond course requirements.
Course design can also be examined for varying degrees of engaged learning as demonstrated through Csikszentmihalyi’s FLOW Model (1990).
Flow refers to the intrinsic motivation to learn where the learner experiences a “state of dynamic equilibrium”. When in the Flow the learner’s perceptions of challenges to perform or to produce are aligned and in balance with the learner’s capacity to succeed. The learner with skills lower than needed to succeed or who does not feel challenged becomes apathetic. If skills are much lower than need to be successful, the learner becomes anxious or becomes negative. Low challenges presented to highly skilled learners lead to complacency and then boredom.
The validity of these observations has been confirmed in numerous studies of interest and motivation in the fields of art, science, sport and technology. It has been assimilated by psychology and learning theory.
Developing the Framework
This framework was built upon an earlier version developed by Barbara Means of SRI International and adapted for the publication “Pugging-In: Choosing and Using Educational Technology” (http://files.eric.ed.gov/fulltext/ED415837.pdf)
Means identified seven variables that, when present in the classroom, indicate that effective teaching and learning are occurring.
These classroom variables are:
- students are engaged in authentic and multidisciplinary tasks
- assessments are based on students’ performance of real tasks
- students participate in interactive modes of instruction
- students work collaboratively
- students are grouped heterogeneously
- the teacher is a facilitator in learning
- students learn through exploration
We took these variables along with the framework published in “Pugging-In: Choosing and Using Educational Technology” and reorganized them into a set of three fundamental elements of learning and instruction: Instructional Approach, Teaching, and Learning. These elements contain nine categories: vision of learning, technology, instructor role, tasks, grouping, instructional model, assessment, learning context, and student roles. We then expanded the definitions of variables with information from recent research on learning and online instruction and added many new variables. In all, there are 30 variables or 30 indicators of engaged learning in online courses. Each indicator is further identified as being related to cognitive investment, socio-emotional engagement, and/or behavioral engagement.
Applying the Framework to Assess Course Quality
As a pilot study, six online courses were reviewed for 21st Century engagement based on the Indicators of Engaged Learning Online framework. These courses were assessed through the evaluation of course design artifacts including the syllabus, assessments, activities, assignments, instructional resources and promotional materials. For each course, design artifacts were reviewed and scored for engaged learning based on the framework developed. For each course, the sum of scores was used to show the level of engaged learning that was seen through the course design artifacts. Course design scores were then plotted on a continuum from passive learning to engaged learning, from 0-90.
The number refers to features that are present in the design of the course, as stated in formal descriptions such as syllabus, assignments, and promotional materials. Place this number in the Design column for each indicator in each chart.
Scoring
Design Scores for Learning and Technology Engagement
0 = Not in place at this time/not applicable
1 = Design element in place but feature in the course falls short of potential stated in the definition (e.g., COLLABORATIVE – there is a collaborative activity but is useless)
2 = Design element in place and corresponds clearly to one or more features in the course (e.g., COLLABORATIVE – there is a collaborative activity but it could be better)
3 = Design element in place and is a major appeal of the course (e.g., COLLABORATIVE – there is a collaborative activity in place and it is designed with engaged learning elements in mind).
Plotting Graph
Plot the grand total for engaged learning on the horizontal learning axis by drawing a vertical line indication where the course design falls on the continuum of Passive learning to Engaged learning.
Potential Impact
More and more students are attracted to online learning as an option for higher education. According to “Grade Change: Tracking Online Education in the United States, 2013” (Allen, I.E. & Seaman, J. 2014):
- 7.1 million of higher education students are taking at least one online course.
- The percent of academic leaders rating the learning outcomes in online education as the same or superior to those as in face-to-face instruction grew from 57% in 2003 to 74% in 2013.
The eleventh annual report on the state of online learning in U.S. higher education from the Babson Survey Research Group, Pearson and the Sloan Consortium includes data responses from more than 2,800 colleges and universities.
Quality issues can become evident through observations of teaching effectiveness, faculty-to-student ratios, attrition rates, student satisfaction, and institutional resources invested in online delivery. With the world at their doorstep, there are so many options available to online students that institutions cannot compete solely from technical innovation. Instead, we need to attract more learners by providing them with better education quality than they can get from the competition.
Outcomes
As a result of this study, we will be able to:
- Develop a framework to measure engaged learning online.
- Assess the quality of online courses using 21st Century engagement indicators as aligned with elements of socio-emotional, cognitive and behavioral elements.
- Utilize a framework that will encourage and support an engaged learning environment throughout course design, development, and revision.
References
Allen, I. E., & Seaman, J. (2014). Grade change: Tracking online education in the United States. Babson Survey Research Group. Retrieved from:
http://www.onlinelearningsurvey.com/reports/gradechange.pdf
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: Book 1, Cognitive domain. New York: Longman.
Bransford, J. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY: Harper & Row Publishers, Inc.
Csikszentmihaly, M. (1996). Flow: The Psychology of Optimal Experience.
Dewey, J. (1938). Experience and education. New York: Simon & Schuster.
Fredricks, J., Blumenfeld, P., and Paris, A. (2004). School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research. Vol. 74, No. 1, pp. 59
Johnson, D. & Johnson, R. (2004). The Three Cs of Promoting Social and Emotional Learning. Chapter 3 from the book by Zins, et.al. (2004). Building Academic Success on Social and Emotional Learning.
Jones, B. F., Valdez, G., Nowakowski, J., & Rasmussen, C. (1995). Plugging in: Choosing and using educational technology. Washington, DC: Council for Educational Research and Development, and North Central Regional Educational Laboratory. Available online: http://www.ncrel.org/sdrs/edtalk/toc.htm
Ledward, B.C. and D. Hirata. (2011). An overview of 21st century skills. Summary of 21st Century Skils for Students and Teachers, by Pacific Policy Research Center. Honolulu: Kamehameha Schools – Research & Evaluation
Lombardi, Marilyn M. and Oblinger, Diana G. (2007). Authentic Learning for the 21st Century: An Overview. ELI Paper 1:2007
Moore, Janet. (2010). A Quality Scorecard for the Administration of Online Education. The Sloan Consortium. Rubrics: Constantly Seeking Quality. http://sloanconsortium.org/node/18236
North Central Regional Educational Laboratory & the Metiri Group. (2003). enGuage 21st century skills: Literacy in the digital age. Naperville, IL: Author
National Survey of Student Engagement. (2012). Promoting Student Learning and Institutional Improvement: Lessons from NSSE at 13. Bloomington, IN:Indiana University Center for Postsecondary Research.
Newmann, F., Wehlage, G. G., & Lamborn, S. D. (1992). The significance and sources of student engagement. In F. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 11–39). New York: Teachers College Press.
Shernoff, David J.; Csikszentmihalyi, Mihaly; Shneider, Barbara; Shernoff, Elisa Steele: (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, Vol 18(2), 158-176.