Reprinted from Computers & Education 39 (2002) 261-269

Evaluating ADAPT: A Hybrid Instructional Model Combining Web-Based and Classroom Components

Bruce W. Tuckman*

Academic Learning Lab, The Ohio State University, Columbus OH, USA

* Corresponding author. Tel.: 614-688-8284; Fax: 614-688-3912; E-mail: tuckman.5@osu.edu

Abstract

The purpose of this evaluative study was to determine the effectiveness of a hybrid instructional model, called ADAPT (Active Discovery And Participation thru Technology) that combines the important features of traditional classroom instruction (classroom, instructor, textbook) with those of computer-mediated instruction (learning by performing rather than listening, frequent assessment and feedback). In combination, the model is distinguished from either distance or traditional instruction, and can be employed in campus computer labs. Both the ADAPT model and the traditional approach were used to teach a 10-week study skills course, the objective of which was to improve students’ academic performance, as measured by grade point averages. Results of using the two approaches and comparing them to one another and to a matched control group experiencing neither yielded an overall significant difference as well as significant differences between each condition. Students taught using ADAPT achieved the highest GPAs, relative to past performance, while those not taught achieved the lowest, with conventionally taught students in between. The hybridity of the ADAPT model seemed to provide students both structure and opportunity for involvement in the learning process.

Keywords: Computer-mediated instruction, pedagogical issues, post-secondary education 

1. Introduction

The critical element in technology-enhanced instruction is the nature of the instructional model, in particular, the extent to which it incorporates the unique advantages of the technology rather than simply mirroring that of conventionally-taught courses. The instructional model that was the subject of this evaluation study is called ADAPT (Active Discovery And Participation thru Technology; Tuckman, 2001). This model for teaching a web-based course in a campus-based computer classroom is a form of  “hybrid.”  It combines important features of traditional classroom instruction (i.e., required attendance, a printed textbook, presence of an instructor) and those unique to  computer-based instruction (i.e., class time spent doing computer-mediated activities rather than listening to lectures, a large number of performance activities rather than just two or three exams, self-pacing with milestones rather than a lockstep pattern). These features are blended together throughout the entire period of instruction.

While technology has played a major role in distance education, its role in campus-based instruction has been minor by comparison.  Lecturing remains the primary instructional method in postsecondary education. In Autumn 1998, 83 percent of faculty and staff with instructional responsibilities reported using the lecture format as their primary instructional method, despite the fact that e-mail, the internet, and web sites are rapidly becoming core components of postsecondary life (U.S. Department of Education, 2001). In those instances where the internet was employed for instructional purposes, it was almost exclusively for distance education. The distance use, however, has not been embraced by traditional faculty members who prefer personal contact with students and the “traditional rhythms of classroom-based instruction” (Carr-Chellman, 2000).  There is no reason why web-based instruction should not become more mainstream in campus courses, not merely as an occasional alternative for convenience purposes, but as an integral part of the instructional design. The ADAPT model represents a way that this can be accomplished, and the effectiveness of this model was tested in this study.

2. Nature of the ADAPT instructional model

The instructional or pedagogical design utilized in the ADAPT model is basically a blend of the objectivist and constructivist approaches, what Passerini & Granger (2000) recommend as the ideal paradigm for internet course design. Mayer (1999)  contends that “one does not need discovery learning to have constructivist learning; i. e., learners can construct meaning from well-designed direct instruction” (p.143), thereby further blending the distinction between the objectivist and constructivist models. At the same time, Hannafin, Land, & Oliver (1999) identify authenticity, anchoring, problem-based manipulation of objects and ideas as the key pedagogical features of technology-enhanced learning.

The ADAPT model is an attempt to combine both approaches by providing well-designed direct instruction about the content of the course with problem-based, manipulative or active learning in the form of computer-mediated performances (such as those described below). These computer-mediated instructional activities were designed to incorporate modeling, coaching, and scaffolding, as recommended by Jonassen (1999) as ways to support learning in constructivist learning environments. Examples, guided practice, and problem solving with feedback are extensively employed for this purpose.

Integrated into the ADAPT instructional model is an assessment approach that departs from the typical classroom experience.  In contrast to typical undergraduate student assessment based on a mid-term, final, and perhaps project, the computer-mediated model used for this type of hybrid course enables students to be involved in assessment on a daily basis. Indeed, in the ADAPT model, computer-based learning activities serve as both opportunities to construct knowledge (as mentioned above) and assessments of progress.

This process of tightly integrating instruction and assessment serves to make the latter a seamless part of the former, and one that has relevance to the student far beyond earning a grade (Siedel & Walters, 1994; Simmons, 1995).   As such, motivation is generated from genuine involvement, commitment, and emotional response, all prerequisites to genuine understanding (Perkins & Unger, 1999).

The classroom features of instruction incorporated into the model were done to maximize the structure and discipline typically lacking in a distance learning course. Regular class attendance under the watchful eye of an instructor increases the likelihood that work will be done in a timely manner (Tuckman, 1999a ). The use of a printed textbook was predicated on findings that show that printed information is more easily remembered than information conveyed by computer (Murphy, 1999).

3. The course and the ADAPT model

The ADAPT model was used to teach Individual Learning and Motivation: Strategies for Success in College, a 5-credit course on “study skills.” The objective of the course is to improve students’ study skills, thereby enabling them to improve their overall academic performance. While the ADAPT model appears to have broader applicability, its evaluation in this study was limited to its use for this course. However, the use of the model is aptly illustrated by its particular application in this course.  

The course met for two 2 ¼ hour periods a week, for 10 weeks. Students spent the majority of each class period engaged in a variety of online performance activities that enabled them to apply the concepts and strategies they were learning in a variety of contexts, while simultaneously demonstrating their understanding of those concepts and strategies.  Over the course of the 10-week term, students engaged in 200 computer-mediated performance activities relating to the specific subject-matter being taught, in this case the study strategies and skills that are critical to success in college. (See Tuckman, 2001 for a description of these.) The performance activities required students to contextualize, discover the value of, and apply the strategies that were presented to them.  They also served an assessment function. Assessment of performance is considered an important feedback tool for students as well as an evaluative mechanism (Simmons, 1994). A variety of forms of feedback were used throughout instruction allowing students to improve their work as they developed their understanding of and facility with the concepts and skills in the course.

The performance activities were designed to engage students in self-reflection, skillful problem solving, and independent thinking.  Students also learned to work in teams - one form of assessment, called “applications,” was always done in a paired-student configuration.  Each module of instruction included the following types of computer-mediated performance activities:

1. Self-surveys. Questions or questionnaires that asked students to explore their beliefs about themselves and others, about their skills, and their attitudes.

2. Quickpractices. Activities that helped students practice the specific skill being introduced, and provided immediate feedback.

3. Assignments.  More complex applications of skills and motivational concepts that were contextualized within students’ own life experiences, and that served the dual purpose of discovery and evaluation.

4. Applications.  Assignments that were done in groups of two.

5. Spotquizzes.  Pairs of multiple-choice and short essay quizzes that tested basic content, that is, capability to perform the objectives of each module.

6. On-line discussions. Posting of and replying to messages relevant to pre-assigned discussion topics that were related to study skills and their use. Students were required to post two messages and two replies to each of 10 discussion topics.

In addition, two types of performances were submitted via email, or in traditional hard copy. These were:

1. Portfolios. Capstone performances of each module that required students to apply core skills and concepts of the module to their academics or lives.  These performances were more complex and required more effort on the part of students than the preceding types.

2. Papers written about the main character of a biography used as a supplementary reading. In these papers, students applied the concepts taught in the course to the life of the main character.

All performance activities appeared both in the printed textbook that was used in the course, and in a relational, web-based database that also served as an online gradebook. All performance activities except the portfolios and papers were submitted online through the front-end of the database, were read and graded by instructors through the back-end of the database, and all grades and instructors’ comments appeared in the front-end of the database. All performance activities also had “windows” of time after which they would not be accepted. Closing dates appeared in the database and were also projected on a screen during class periods. Instructors monitored student progress via the database and met with students during class periods.

4. Purpose of the study

The purpose of the study was to compare the academic performance for the term in which the course was taken of students taking the traditional classroom version of the course to those taking the computer-mediated ADAPT version to determine the effectiveness of the ADAPT instructional model used to teach the course. Improving students’ overall academic performance was the primary objective of the course. 

The course was originally taught conventionally, that is in classrooms, without the use of computers. While the curriculum was the same, the instructional model differed in that lectures were used, performance activities were done as homework, and were then submitted in written form. The same textbook, content, and performance activities were used in both the ADAPT instructional model and conventional model for teaching the course.

Effectiveness of the ADAPT instructional model was determined on the basis of students’ grade point averages. Since the primary objective of teaching study skills was to enable students to improve their academic performance, it seemed most appropriate to judge instructional effectiveness on the basis of degree of improvement in grade point average.

5. Methods

5.1 Subjects

The number of students involved in the evaluation study was 452. Of this number 74 were enrolled in the classroom version of the course during each of two academic quarters, 189 were enrolled in the computer-mediated version employing ADAPT during the corresponding two quarters the following academic year, and 189 did not take the course in either format.  The distribution of students in the sample by gender was 49% male and 51% female; by ethnicity, 33% minority and 67% non-minority; by year in college, 25% freshmen, 40% sophomores, 24% juniors, and 11% seniors. The distribution across all three parameters was equivalent for the three groups.

Students enrolled in the course, for either version of instruction, primarily on the basis of recommendations by their advisors. The group not taking the course was identified for comparison on the basis of matching course takers as closely as possible on gender, ethnicity, year in school, and prior cumulative grade point average.

5.2 Independent Variable

Instructional condition served as the independent variable, with three levels: (a) taking the version of the course taught using ADAPT (the hybrid, computer-mediated/classroom version of the course, (b) taking the version of the course taught using conventional classroom instruction, and (c) not taking the course.

(a) The ADAPT version of the course, as described above, was taught in one half of a 4,600 square foot on-campus computer lab housing 63 personal computers (outfitted with Pentium III processors) connected over a 100 megabit network to two Dell Poweredge 4350 servers running IIS 5.0 with Windows 2000 Server. The lab is dedicated to providing students with learning assistance; the other half of the lab is used to provide individualized assistance to students not taking the course, or to enable students taking the course to spend time out of class time working on the course. Electronic submissions to the course database could only be made from computers housed in the lab.

The class met in the lab for four hours and 36 minutes (in two sessions) a week for 10 weeks. Students were expected to be present during these periods. Attendance was monitored by instructors who were present at all times. The textbook used was Learning and Motivation Strategies: Your Guide to Success (Tuckman, Abry, and Smith, 2002). The course contained 215 performance activities, of which 200 were computer-mediated (web-based) with display, data capture, and threaded discussion functions powered by Microsoft Frontpage and Access. All performance activities had windows or deadlines with penalties for late submission. Students spent most of their class time working on computers completing and submitting performance activities. Some time was spent accessing and examining computer-mediated feedback, and some was spent in individual consultation with instructors.

(b) The conventional classroom version of the course was taught in exactly the same time frame as the ADAPT version, using the same textbook, all of the same performance activities, and a subset of the same instructors. The differences between the two versions were that 1. the conventional classroom version was taught in a regular classroom, rather than a computer lab; 2. students spent class time listening to lectures and engaging in activities; 3. performance activities were done primarily after class hours as homework (although some students worked on them during class time rather than attending to what was going on); 4.  instructors spent virtually all their time in class lecturing and leading activities, leaving little time for the kinds of individualized contact used in the ADAPT version. The conventional classroom version also included discussions during class time, but these were done “live” rather than electronically as in the ADAPT version.

(c) In the control condition, students received no formal instruction of any sort in study skills.

5.3 Dependent Variable

The criterion measure was the grade point average students earned during the quarter in which they took the course. The grading system includes the grades A, B, C, D, F with plusses for all grades except A and F and minuses for all grades except F. These grades were converted to numbers and averaged across all of the courses being taken that quarter. GPAs were adjusted on the basis of prior cumulative grade point average to provide some degree of control for past academic capability. 

5.4 Data Analysis

The design of the study represents summative evaluation (Tuckman, 1985, 1999b). It allows researchers to make systematic comparisons leading to inferences about cause and effect.

To compare the effect of the three conditions on quarter grade point average, an analysis of covariance using SPSS’ general linear model was run on quarter grade point average with prior cumulative grade point average as the covariate. The between-subjects factor was condition, with three levels. Where appropriate, post hoc comparisons were made using the least significant difference method (LSD).

6. Results

To establish the equivalence of level of achievement of the students in the three instructional conditions prior to taking (or not taking) the course, an analysis of variance was run on prior cumulative grade point average. The resulting nonsignificant F (df=2/448) of 1.395 indicated that three groups were initially equivalent on GPA. The means and standard deviations on prior cumulative GPA for the three groups were as follows: ADAPT= 2.40 (sd=.55), Conventional= 2.35 (sd=.61), Control= 2.41 (sd=.56).

The results of the analysis of covariance of quarter GPA are shown in Table 1. They reveal a highly significant main effect for instructional condition (F=30.435, df=2/448, p<.001). Post hoc analysis revealed a significantly higher quarter GPA for students taking the ADAPT version of the course than the conventional classroom version (mean difference=.24, p<.02). Students taking the ADAPT version of the course earned an average quarter GPA of 3.00 (sd=.62) compared to 2.76 (sd=.77) for students taking the conventional classroom version. Both course versions were significantly more effective than no version (M=2.52, sd=.73), indicating that the course content as well as the instructional model contributed to quarter GPA. Mean comparisons are displayed in Figure 1.

To additionally demonstrate that the differences were a result of the instructional treatment and not prior achievement, GPA-gain scores were calculated by subtracting individual prior GPA values from quarter GPA values (essentially a post-minus-pre score), and subjecting these scores to analysis of variance. The resulting significant F ratio for instructional condition (F= 29.73, df=2/448, p<.001) reflected greater gains in the ADAPT condition (.55) than either the conventional condition (.31) or the control condition (.05).

To test whether the grade earned in the course, a measure based partly on instructor judgment, might have had a biasing effect on quarter GPA differences between the ADAPT and conventional classroom versions, an analysis of covariance was run on course grade with two levels of instructional condition as the between-subjects factor, and prior cumulative GPA as the covariate. The no course condition was excluded since students not taking the course could not have earned a grade in it. The main effect of condition for this second analysis was not significant (F=1.455, df=1/260, p=.30), indicating that grades earned in the course did not differ as a function of instructional model, and hence did not affect the GPA differences described above.

7. Conclusions

The results of this evaluation study showed that students taught study skills using the combined classroom/ computer-mediated ADAPT instructional model improved significantly more in academic performance than students taught the same material by the conventional classroom approach.  Moreover, this performance difference was not simply a function of either prior differences in achievement or grade in the course. In addition, both instructional models affected academic performance more favorably than no instruction at all.  

One can hypothesize that the features of the training likely to have been responsible for the observed improvement in academic performance were (1) the structure and discipline provided by classroom meetings and instructors that distinguish the ADAPT model from distance instruction but not from conventional classroom instruction, and (2) the opportunities for practice, assessment, and feedback provided by 200 computer-mediated performance activities that distinguish the ADAPT model from conventional classroom instruction but not from distance instruction. The structure of the learning environment was likely to help students manage their time and remain on task . The frequent performance activities served as a way of causing students to be actively involved in the learning process throughout the entire training period, to facilitate transfer, and to stimulate the development of new, functional behaviors.

 The combination of the two features, designed to help students contextualize what they were learning by applying it to other courses and other situations, may have contributed to the transfer required to increase grades in other courses. Further research will be required to determine the specific contribution of these and other features of the ADAPT instructional model to its overall effectivenesss.

One final note on the concept of “hybridity” as a category of technology-enhanced instruction. The term hybrid in any context refers to a composite, composed of disparate or incongruous elements. In terms of instruction, the term is most commonly applied to the use of conventional classroom instruction part of the instructional time and online, distance instruction the remainder of the time (Lago, M.E., 2000). Thus described, it is not a true composite, because the two forms of instruction are not combined in terms of either time or place, each occurring at different times and in different places. While the course may be a hybrid, the instruction itself is not.

In the ADAPT model, on the other hand, true hybridity exists, because both forms of instruction are joined together in both time and place. The results of the evaluation conducted in this study suggest that such true hybridity, while perhaps not maximizing student convenience, does maximize the likelihood of students achieving the objectives of instruction. It remains for further research to compare these two forms of hybridity directly to better understand their possible differences in effectiveness.

References

Carr-Chellman, A.A. (2000). Distance education: A reflective action research project and its systemic implications for higher education. Systemic Practice and Action Research, 13, 587-612.

Hannafin, M., Land, S. & Oliver, K. (1999). Open learning environments: Foundations, methods, and models. . In C. M. Reigeluth (Ed.), Instructional-Design theories and models: Vol. 2.  A new paradigm of instructional theory (pp. 115-140). Mahwah, NJ: Lawrence Erlbaum Associates.

Jonassen, D. (1999). Designing constructivist learning environments. . In C. M. Reigeluth (Ed.), Instructional-Design theories and models: Vol. 2.  A new paradigm of instructional theory (pp. 215-240). Mahwah, NJ: Lawrence Erlbaum Associates.

Lago, M.E. (2000). The hybrid experience: How sweet it is! Converge, 3 (11), 32-34, 36.

Mayer, R. (1999). Designing instruction for constructivist learning. . In C. M. Reigeluth (Ed.), Instructional-Design theories and models: Vol. 2.  A new paradigm of instructional theory (pp. 141-160). Mahwah, NJ: Lawrence Erlbaum Associates.

Murphy, P.K. (1999). Modeling the multifaceted persuasion process: Learner and text interactions. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.

Passerini, K. & Granger, M.J. (2000). A developmental model for distance learning using the internet. Computers & Education, 34, 1-15.

Perkins, D. N., & Unger, C. (1999). Teaching and learning for understanding. In C. M. Reigeluth (Ed.), Instructional-Design theories and models: Vol. 2.  A new paradigm of instructional theory (pp. 91-114). Mahwah, NJ: Lawrence Erlbaum Associates.

Seidel, S, & Walters, J. (1994). The “things” children make in school: Disposable or indispensable. Harvard Graduate School of Education Bulletin, 39 (1), 18-20.

Simmons, R. (1994). Dusting the television: Teaching for understanding. Harvard Graduate School of Education Bulletin, 39 (1), 13-16.

Simmons, R. (1995). The horse before the cart: Assessing for understanding. Educational Leadership, 51 (5), 22-26.

Tuckman, B.W. (1985). Evaluating instructional programs (2^nd ed.). Boston: Allyn and Bacon.

Tuckman, B.W. (1999a). A tripartite model of motivation for achievement: Attitude/Drive/Strategy. Paper given at the annual meeting of the American Psychological Association, Boston, MA.

Tuckman, B.W. (1999b). Conducting educational research (5^th ed.). Fort Worth, TX: Harcourt Brace Publishers.

Tuckman, B.W. (2001). The effectiveness of the strategies-for-achievement approach for teaching “study skills.” Paper given at the annual meeting of the American Educational Research Association, Seattle, WA.

Tuckman, B. W., Abry, D., & Smith, D. R. (2002). Learning and motivation: Your guide to success. Englewood Cliffs, NJ: Prentice Hall.

U.S. Department of Education (2001). The condition of Education 2001. National Center for Educational Statistics (2001-072). Wash, DC: Government Printing Office.

Figure Captions

Fig. 1. Mean Quarter GPAs for the Three Instructional Conditions

Table 1

Analysis of Covariance of Quarter GPA by Instructional Condition (CNDITION )With Prior Cumulative GPA (PRCUM) as the Covariate

R Squared = .277 (Adjusted R Squared = .272)