The Student Success and Support Program (SSSP) is intended to increase student access to and success in California Community Colleges. Goals include ensuring that all students complete courses, persist to the next academic term, and ultimately achieve their desired educational outcomes with the assistance of admissions, orientation, assessment and testing, and student follow-up.
Assessment is an important part of the SSSP process, as improper placement can increase the number of remedial courses students are required to take prior to college level, increase student frustration, and increase overall costs of attending college. Conversely, proper placement can decrease the length of remediation or number of remedial courses required, decrease student frustration by ensuring students are placed at a level appropriate to their knowledge and skills, and increase the likelihood that students achieve their desired education outcomes.
As explained in the Chancellor’s Office Assessment Q&A document published in March 2005, assessment is much more than testing: “According to Section 55502(c) of the Title 5 Regulations, assessment instruments, methods, and procedures ‘include, but are not limited to, interviews, standardized tests, holistic scoring processes, attitude surveys, vocational or career aptitude and interest inventories, high school or college transcripts, specialized certificates or licenses, educational histories and other measures of performance.’” But while many different methods and pieces of information can be used to place students into courses, assessment tests are common. Although placement can be done without a test, Title 5 §55521(a)(3) establishes that placement cannot be done with a test alone. Colleges typically use an approved assessment test in combination with other information to place students into courses. And, in order to receive Student Success and Support Program funding, colleges must use the common assessment instrument once it is available if an assessment test is used as part of a college’s placement methods (Title 5, §55518).
By now faculty should be aware that a common assessment test (CCCAssess) is being developed. The Common Assessment Initiative (CAI) has been working for nearly three years to create a custom assessment system for the California Community Colleges. A May 2016 RP Group evaluation of CAI processes, led by Senior Researcher for the RP Group and lead evaluator for CAI Tim Nguyen, included a survey of individuals participating in the Common Assessment Initiative. According to Nguyen, the survey found that 94 percent of respondents were satisfied or very satisfied with the decision to custom-develop CCCAssess rather than adopt or modify an established testing instrument, and 91 percent were satisfied or very satisfied with the progress made in the past year to develop CCCAssess.
Title 5 §55522(a) states that “the Chancellor’s Office shall establish and update, at least annually, a list of approved assessment tests for use in placing students in English, mathematics, or English as a Second Language (ESL) courses.” Any assessment test that a college uses for placing students into courses in these disciplines and others must be on the Chancellor’s Office approved list. All assessments tests must be evaluated to ensure they minimize or eliminate cultural or linguistic bias, are valid, and are reliable.
While CCCAssess is a CCC-developed assessment test, it still must meet the standards that all second party publishers and local campus developers must meet to earn a place on the Chancellor’s Office approved list of assessment tests. This process includes submission of a proposal to the Chancellor’s Office with all required information as explained in the Chancellor’s Office document Standards, Policies, and Procedures for the Evaluation of Assessment Instruments Used in the California Community Colleges, 4th edition (March 2001). The ultimate goal is to ensure that an assessment test is valid, meaning that it measures what it is intended to measure, does so consistently, and is free from bias.
Assessment test proposals are received by the Chancellor’s Office and evaluated by the Assessment Workgroup, a sub-committee of the SSSP Advisory Committee. The workgroup consists of community college staff representing all groups responsible for assessment at colleges: administrators including representatives of CEOs, CIOs, and CSSOs, faculty including English, reading, ESL, mathematics, and counseling, assessment coordinator, SSSP coordinators, and researchers. The Assessment Workgroup meets biannually, usually mid-spring and mid-fall, to review and evaluate proposals and make a recommendation for one of three levels of approval or for disapproval. The workgroup is aided by psychometric consultants who do an initial review of proposals, submit a preliminary recommendation and request for clarifying information to Chancellor’s Office and submitting colleges or publishers, produce a modified report based on clarifying information and any other supplemental material provided by submitters, and then work alongside the workgroup to generate a final recommendation to the Chancellor’s Office, which then notifies colleges and publishers.
Test submissions are evaluated on established standards for content validity, test bias, establishing and validating cut scores, disproportionate impact, and reliability. Submissions
must include content‐related descriptions of the test (e.g. test blueprint, information about item selection algorithm) and enough test items so local colleges can conduct a content
alignment study between items and placement course prerequisites. Submissions must also include criterion or consequential validity evidence from at least three community
colleges for probationary approval, at least four community colleges for provisional
approval, and at least six community colleges for full approval, with a majority of the colleges supplying data being from California for any level of approval (2001 CCC Assessment Standards). A bias review panel and empirical study on disproportionate impact must be included to indicate fairness across all groups. Finally, reliability evidence must be provided using test‐retest reliability methodology and reporting standard errors of measurement. Cut score validation and disproportionate impact are the responsibility of the college using the assessment, even in the case of publisher exams.
Depending on the information provided, the workgroup can make one of four recommendations:
- Full approval—submission meets standards in all areas.
- Provisional approval—submission meets most standards, including minimum required standards for validity and absence of bias.
- Probationary approval—submission meets the minimum required standards for validity and absence of bias.
- Disapproval—submission fails to meet the minimum required standards for validity and absence of bias.
Reviewing assessment test submissions is not an easy task, and the psychometric consultants as well as experienced assessment experts play a critical role in making certain that all workgroup members understand and apply the minimum standards as submissions are evaluated. Workgroup members often speak of the people at the center of it all – students – while reviewing and discussing submissions. Ultimately, the workgroup is responsible for ensuring that an approved assessment test does no harm to students by making sure tests are fair, reliable, and valid.
According to the CCCAssess Implementation Timeline, colleges are expecting to implement CCCAssess at pre-determined points between Fall 2016 and Fall 2018. Twelve of the colleges planning for Fall 2016 CCCAssess implementation were pilot colleges; the total number of colleges included in the Fall 2016 implementation was increased with the announcement that ACT’s Compass would sunset effective November 30, 2016, at which point the total number of colleges implementing CCCAssess increased to 37, including pilot, sister, and Compass colleges. With 37 total colleges expecting to implement CCCAssess in Fall 2016, the stakes were high and the pressure was on CAI to launch an assessment test that would consistently and effectively produce placement recommendations that, when used in combination with other methods, appropriately place students into courses.
However, recent developments have led to the conclusion that CCCAssess will not be an approved assessment test for Fall 2016. Expansions and improvements were already planned beyond the initial rollout and approval, and some of those planned expansions are pieces critical for implementation. While the launch delay causes challenges for the 37 colleges planning on the test’s availability in Fall 2016, those involved in the decisions are confident that the postponement will make CCCAssess a more robust product at approval and initial launch. The Common Assessment Initiative Steering Committee, management team, and workgroups are developing an assessment test that, once complete and approved by the Chancellor’s Office and considered along with other CAI efforts, accomplishes multiple goals of the Board of Governors and California Legislature, including developing a common assessment test, reducing the cost of assessment, and reducing the need for re-assessment by increasing the portability. System stakeholders remain positive that CAI efforts will continue to increase the effectiveness and accuracy of placement for students assessing at or below college level, lower unnecessary remediation rates for California Community College students, increase the initial placement level for qualified students, increase awareness of the importance of placement tests and improved student preparation, assist colleges with efforts to improve local assessment and placement practices, and leverage multiple measures data and research to improve placement.
Progress has already been made on many of these CAI goals, and as system stakeholders faculty should recognize the multi-faceted efforts of CAI rather than focusing solely on the delay of CCCAssess as a Chancellor’s Office approved assessment test. System stakeholders should also remember that the initial timeline for CCCAssess implementation only included twelve pilot colleges in Fall 2016. The development timeline was challenging, putting significant pressure on developers, and was then exacerbated further by an urgent need for an alternate assessment test due to the sunset of Compass. CAI is preparing to serve far more colleges with CCCAssess far earlier than planned, and while challenges still exist at the system and college levels, patience now will result in a stronger product for our students later.