The Accountability Game - Stanford 9 in K-12, HMO's in Health Care, .MSLOs in CC's
The increasing demand for accountability, particularly in tax-supported institutions appears to be aimed primarily at the community colleges. There is nothing intrinsically wrong with expecting accountability, but, when an activity is held up to scrutiny, we should ask ourselves: What kind of accountability is being called for? Who is demanding the accountability? Why are they demanding it? And, are the methods used to scrutinize the activity valid?
In June 2002, the Accrediting Commission for Community and Junior Colleges (ACCJC) passed radical new standards by which to accredit community colleges, incorporating the idea of "continuous improvement" of "measurable student learning outcomes" (MSLOs) throughout. The ACCJC passed these new standards over the vociferous objections of respected faculty organizations. Nationally, the American Association of University Professors (AAUP) has come out against modifying accreditation standards this way, and in California, the Academic Senate for California Community Colleges along with the Community College Council of the California Federation of Teachers (CCC/CFT) have condemned this radical change by ACCJC. Why?
The whole concept of MSLOs as the latest fad in education is somewhat akin to the now discredited fad of the `90's, Total Quality Management, or TQM. Essentially, the ACCJC adopted MSLOs as the overarching basis for accrediting community colleges based on their faith in the theoretical treatises of a movement, just as advocates for the use of TQM in education (often called continuous quality improvement or CQI in educational circles) were part of an ideological movement. After repeated requests for research showing that such use of MSLOs is effective, none has been forthcoming from the ACCJC. Prior to large scale imposition of such a requirement at all institutions, research should be provided to establish that continuous monitoring of MSLOs has resulted in measurable improvements in student success at a given institution. No such research is forthcoming because there is none. If the "learning paradigm" is so superior as to justify its widespread adoption, then the research should clearly be compelling.
The new standards would require documentation and continuous improvement of learning outcome measures at the course, program, and certificate levels. This would require faculty and administration to measure outcomes that can be immediately documented, not long-term outcomes such as successful application of coursework in students' careers.
Also, as student learning outcomes are measures of knowledge or skills a student has attained as a result of a given college course or program, they do not include institutional measures such as course retention, completion or graduation. Community colleges in California already gather data on institutional outcomes, but this would require generation and tracking of a whole range of new measures.
Often, objections to MSLOs are met with, "But, you faculty will define what the outcomes to be measured are." This assumes that what faculty currently measure, via exams and grades are not adequate, and that faculty should spend their time generating new and much more specific skill based measures. However, no evidence has been presented establishing that the outcomes of our pedagogical efforts are not adequately measured by our current approaches, or that new measures would lead to greater student success.
In addition, much that is most beneficial in higher education is often difficult or impossible to measure-but certainly is not measurable at the course level. A business department might feel the most important outcome is that their students use what they learn in the classroom successfully in a career in the business world. But this is not a learning outcome that can be documented at the course or program level.
In the book, "A Beautiful Mind" about the genius, John Nash, (upon which the movie of the same name is based), the early chapters, 3,4,5,.describe how undergraduates learned at Princeton. It is illuminating in how casually many of these students acquired knowledge, and certainly not in a measurable way, but learning was effective, nevertheless. Tiger Woods has said that, even though he dropped out of Stanford in his sophomore year, he learned skills important to him in his professional golfing career, specifically in the area of time management skills. How would one measure that outcome?
It is interesting that there is suddenly this push to get "accountability" from the community colleges, the institutions that serve the most diverse and working class students, and not the same push to get it from the elite institutions, even though many universities are, like most community colleges, dependent upon taxes for their continued existence. The truth of the matter is that institutions like the Ivy League schools and top universities are not being threatened with MSLOs.
Furthermore, the MSLO movement utilizes a scorecard approach, in which you assess in percentage terms where your students are now in terms of a defined learning outcome, and how you would like to increase the percentage in 5 years, say. Then, you set as your goal the percentage improvement you want to make each semester! This requirement, that there be continuous improvement of learning outcomes, assumes that student achievement can be increasingly rationalized like a production process.
This push to document and improve student learning outcomes essentially creates pressure to focus one's course objectives on discrete, skill-based and hence most easily measured variables. Quantitative variables are more easily tracked than qualitative ones. This over time will yield to a "dumbing down" of the curriculum, as broad capacities and more long-term, qualitative changes in student behavior and perception will be relatively de-emphasized in the push to measure.
How about faculty members in art deciding that an outcome is that students have at least a rudimentary appreciation of great art and how to recognize it? How does one measure that? How does one measure a sociology department's desired outcome that their students have a more tolerant attitude towards other cultures and ethnic groups? You can probably think of more examples of the impossibility of measuring outcomes of what we do. Even if it were possible, is it realistic to expect a 2% (say) improvement per semester in any given outcome?
In the teaching and learning process, there is a two-way interaction, and there has to be cooperation and interest on both ends. Whether a student succeeds in a class is a function of not one but many factors. Some of these are: the intellectual level of the student's household, the quality of the preparation the student received in educational institutions attended before reaching ours, the priority that the student places on the class, the amount of effort a student is willing to apply outside of class, resistance to distractions from friends, family, and jobs. Many of these are beyond the instructor's or college's control. Yes, we can find new and better ways to present the material, and we can use tutors and workshops to help motivate students and to help them succeed, but those efforts alone might go for naught for some students.
Perhaps what irritates us most about the ACCJC's action, besides the fact that they chose to ignore the best advice of the practitioners in the field (the faculty), is that tying accreditation to MSLOs means that the faculty as a whole would have to spend precious time and effort to engage in measuring everything that moves on the campus, diverting our energy and efforts from interacting with students. Will our colleges receive additional funding for these efforts? We seriously doubt it! So, we are being asked to engage in what virtually amounts to a huge unfunded mandate.
What is the evidence that the institutionalization of testing like the Stanford 9 in K-12 education has really improved children's education? Too often, teachers find themselves "teaching to the test," rather than providing a well-rounded education. In health care, HMOs look over the shoulders of health practitioners to decide what procedures should or should not be pursued. Conscientious faculty members are constantly studying ways to improve their teaching and to get better results in the classroom. We do not need the MSLOs movement to second-guess the way we do our jobs.
Another argument that is advanced is that the ACCJC approved these new standards unanimously; the three faculty members sitting on the commission also voted for these new standards, allowing the commission staff to claim that there are faculty supporting the MSLOs movement. Of course it is possible to find individual faculty members to support almost any position that one can think of, but these faculty members were not appointed by any faculty organization, and their votes represented nobody but themselves. The Academic Senate plenary sessions passed resolution after resolution condemning the use of MSLOs as the basis of accrediting decisions, many being passed unanimously. These were votes of faculty members representing faculty at all 108 California community colleges, and represent the collective wisdom of California community college faculty. The faculty members sitting on the ACCJC are or have been active in local or state senates. It is unfortunate that they did not heed that collective wisdom and vote against the implementation of these new standards. Every fad that comes along will find a few adherents among the faculty, but when the opposition among our faculty is as strong as it is, it's clear that the faculty is not split on this issue.
Accountability is fine, but don't give us an untested, obviously bogus scheme with which to hold us accountable when there is not one institution in the country where it has been shown to be effective. Faculty leaders were not brought into the discussion to construct these new standards. Don't tell us, "Oh, but you establish the outcomes to measure," when you haven't asked us whether or not we want to even establish such outcomes in the first place. Let us have an open, frank discussion with representatives of all constituencies, about how to judge the effectiveness of a community college for the purposes of accreditation. In California, accreditation processes are an academic and professional matter: number 7 of the 10+1 items that require input from the academic senate is "Faculty roles and involvement in accreditation processes, including self-study and annual reports." It is a violation of California law that the Academic Senate was not brought into the discussion in the formation of these new standards.
Based on the resolutions passed overwhelmingly at its plenary sessions, the Academic Senate is studying ways to combat the institutionalization of MSLOs. The Academic Senate is working with both the AAUP and the CCC/CFT to consider our next steps, whether it is possible to delay implementation of these radical changes in the accreditation standards, as well as to explore alternatives to the ACCJC.
Please, community college faculty members, give us your ideas on how to resist this reactionary movement so akin to what is going on in health care. Do you really want community colleges to become the HMOs of higher education? If not, spread the word that we do not have to put up with this, and together we can nip this impending disaster in the bud.
The articles published in the Rostrum do not necessarily represent the adopted positions of the academic senate. For adopted positions and recommendations, please browse this website.