Observing Online Classes

May
2005

So you're on a tenure committee and you have to observe a class being taught online. The question is, how does one "observe" a class that's out there on the Web? What do you look for? When do you observe it, and for how long? What if your own web skills are not particularly stellar; will you know what you're seeing when you see it?

With the explosive growth of online courses, many campuses are facing these and many more questions about how to effectively observe and evaluate online teaching. Ultimately, every school must set its own observation protocols, but fortunately much good work has been done in this area, and hopefully no one will have to completely reinvent the wheel. This article assembles some of the best ideas and thoughts on this issue and will raise important considerations. In addition to addressing the question of peer observations of online courses, it also looks at the tricky issue of student assessment of online teaching. All articles, peer observation forms, and other resources are available for your perusal by visiting the web address at the end of this article.

WHAT IS "GOOD ONLINE TEACHING"?

Achieving an institutional consensus on what constitutes good online teaching is a crucial first step to observe online instruction effectively. At the web resource page (listed below) you will find documents developed by our colleges (Shasta College and Mt. San Jacinto College respectively) to aid instructors as they prepare to observe their peers' online courses. Colleges approach the development of observation criteria in a variety of ways, but the local senates should collaborate with their association counterparts to determine what works best for them.

In addition to helping in the course assessment process, these documents also serve a formative assessment function for those teaching online. Online instructors can use them as a guideline to design and conduct their online classes.

HOW DOES AN OBSERVER "VISIT" AN ONLINE CLASS?

Unlike visiting a face-to-face class, you can't just walk in the door! How does one visit an online class when the class exists only in cyberspace? Generally they are organized by units or by time blocks. For example, Frank Nigro's critical thinking class at Shasta College is organized into eighteen one-week units. Students have a set number of tasks to complete that week, including a weekly quiz, a weekly class discussion, and an online lecture. An observer could simply choose one of these unit-weeks to observe, and, over the course of the week, examine how the instructor delivers course content and interacts with the students. The observer may also ask to see any email communications with students for that week and evaluate a) whether communication is indeed taking place, and b) if the instructor is responding to student emails in a timely manner.

As part of the observation, the course observer should review the instructor's policies on communication. If an instructor waits 24 hours to respond to a student's email, or if the instructor completely neglects to do so over the weekend, it's not necessarily a bad thing. However, the observer should make sure the instructor has established this as a policy somewhere in the course. In an online course, the information typically found in a syllabus, including course organizational material, may appear in one place, or it may be distributed at key spots throughout the course. The observer should be aware of how the instructor has provided this information and may want to review any start-up or welcome instructions for the course. If you, as an observer, can't figure out where the lecture materials, assignments and learning resources are in the course, it is likely the students can't either!

IN AN ONLINE DISCUSSION BOARD, WHAT CONSTITUTES MEANINGFUL STUDENT/TEACHER CONTACT?

Effective student/teacher contact isn't only one of the key principles of online teaching, but it's required by Title 5 55211. The notion of "regular 4 effective contact", while evident in a face-to-face course, is very important to identify in an online course. Key to any successful online course is the degree to which it is made interactive. That connection may happen in one-to-one email exchanges between students and instructor, but equally as important is the interaction between the students and among students and instructor through class discussion.

Some online instructors use synchronous discussion tools known popularly as "chat." Chat requires everyone to be in the chat room at the same time. However, since a chat requirement defeats the purpose of taking online classes (most students take online classes because they cannot commit to meeting at a set time or place), most instructors now use asynchronous discussion boards (also known as bulletin boards, threaded discussions, or discussion forums). With discussion boards, students may check in at their own convenience, as long as they do it within the prescribed time frame.

In an article entitled "Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses," Graham Cagiltay, Lim, Craner & Duffy (2001; ) define some features of good online participation through threaded discussion boards (Retrieved May 20, 2005 http://distance.wsu.edu/facultyresources/savedfromweb/7principles.htm). They recommend that participation be both mandatory and graded, that discussion groups be kept small and focused on a task, that threaded discussions always produce some sort of "product" or outcome, and that students be given consistent feedback on their discussions. As with their communication preferences, instructors should post their expectations for discussion and grade the discussion on the quality of the students' participation rather than the number and length of their postings. The instructor's role in the discussion is to read all the students' responses, answer any questions directed at the instructor, and in general moderate discussion as he or she would in a face-to-face class, without dominating the discussion. Class visitors should go to the online discussion board as part of the observation and see if these general guidelines are being followed.

YOU'RE OBSERVING AN ONLINE CLASS BUT YOU'RE JUST NOT THAT INTO COMPUTERS

Colleges should consider offering short, introductory workshops specifically for online class observers. However, in the absence of this, the observer should contact a colleague who is experienced in online instruction and get a quick orientation on how the course management system (CMS) works. Important here is to separate the instructors from the technology. If the instructors are teaching a course online for the first time, they may be less adept at utilizing some of the tools at their disposal than more experienced online instructors. This does not necessarily mean, however, that the students are not enjoying a useful learning experience. Instructors generally adapt courses over a few semesters, incorporating technological tools as they make sense for delivering particular course content.

FORMATIVE ASSESSMENT OF ONLINE COURSES

At most colleges, tenure-track instructors are observed every semester for several years or more. Instructors who already have tenure are observed much less frequently, perhaps every three years or so, and even then, they are often able to stipulate what classes may be visited and by whom. Observations of part-time instructors vary widely in their frequency and nature. So how often should online courses be observed? Or, to put it bluntly, what if a perfectly abysmal online course is taught for years on end and no one observes it?

Of course, these questions can be applied to face-to-face teaching as well as online teaching situations, and in both cases, the best solution is to provide formative assessment guidelines for online classes as they're being developed.

Summative assessment is generally done as part of the tenure process. It provides end evaluation for an instructor by his peers/administrators as well as by the students. It is intended to evaluate and becomes a condition of continued employment. Formative assessment is assessment provided before and during the course. It aims to help the instructor teach better. It does not become a condition of continued employment, and it is generally not required. However, formative assessment may be especially needed for online classes given their relative newness and the technological skills needed to teach them well.

Colleges may require that faculty receive a certain amount of training before teaching an online course, or at least demonstrate competency in the online environment. Certainly training on whatever CMS will be used is necessary, but training in web design and how to make web pages Section 508 compliant should also be included. Often such training is available . . . online! Cerro Coso Community College, for example, offers an online certificate for potential online teachers; the California Virtual Campus and the System's @ONE program also provide training opportunities at little or no cost to the college or instructor. Your college may choose to require such training for those who wish to teach online. Although the Distance Education Guidelines and Title 5 tell us that our approved minimum qualifications must be used in choosing online instructors, additional standards for online instructors can be developed locally and are becoming more prevalent as online programs grow.

Other formative steps include setting up an online course development timeline to specify how online courses can be put together in a timely fashion. This will help ensure that instructors do not commit to teaching an online course a week before the semester begins. Part of the development process could include requiring that the instructor show portions of the developed course to an advisory team of experienced online instructors or even to an online teaching mentor at some point prior to the course's initial delivery. All of these steps can better ensure the likelihood of a positive online learning and teaching experience.

Quality instructional programs begin with attention to planning details. If your college is just beginning to develop an online program, consider developing standards for courses, for instructor selection, for observation, and develop a strong curriculum review process that starts the course on its way to success.

EVALUATING ONLINE COURSE RESOURCES ON THE ACADEMIC SENATE WEBSITE

Distance Education Guidelines, Published by the System Office in March 2004

Curriculum Session:

PowerPoint Presentation, created by Pat James Hanz and Bob Grill.

Sample Distance Education Addendum in use by Mt. San Jacinto College

Excerpted items from the DE Guidelines, you may want to use this to explain the mandates to colleagues at your school.

General Information Resource Page Guidelines for Participating in Online Discussions from Bernie Fortenbaugh, CIAT. Copyright, 2001 Towson University

Peer Evaluation of Distance Education Session:

Senate Presentation Works Cited, Excellent resource articles compiled by Frank Nigro, Shasta College and Nasreen Rahim, Evergreen Valley College

How Interactive are your Distance Ed. Courses? Online article with rubric for interactivity.

Peer Observations:

Peer Observation Guidelines for Shasta College

Peer Observation Guidelines for Mt. San Jacinto College

Comments from Butte College

Student Evaluations:

Student Evaluation Tool from Shasta College

Student Evaluation Tool from Foothill College

Student Evaluation Tool from Mt. San Jacinto College

(word version of digital survey)

Student Evaluation from Lassen College