Assessment Matters

The University of Nebraska announced their superior achievement in value-added learning based on students’ performance on the Collegiate Learning Assessment (CLA). The CLA is one of three performance-based measures (tests) of student learning approved for use on the College Portrait web template (Voluntary System of Accountability-VSA) to which UNR has signed on. The other tests are the Measure of Academic Proficiency and Progress (MAPP from ETS) and Collegiate Assessment of Academic Proficiency (CAAP from ACT). Participation in the VSA will require UNR to administer of one of these assessments (as well as a survey such as the National Survey of Student Engagement - NSSE) to freshmen and seniors over the next 4 years.

The use of a single test to assess general education achievement is controversial. Although standardized tests may seem efficient in their simplicity, there is a downside. Not only are there limitations (Trudy Banta), they bestow an illusion of precision to the assessment process that makes them susceptible to misuse, and not only by institutions, as this Inside Higher Education article Let the Assessment PR Wars Begin illustrates.

Our old friend, Cliff Adelman, adds a his choice comments:

If U.S. higher education really believed that the CLA (or any other test) was a valuable measure of what it expects every student to learn, was, in fact, the ultimate measure of what it does, then it should require all students—and not just unrepresentative samples—to take and pass the exam in question as a condition of graduation. Each school can set its own cut score (there are standard psychometric techniques for doing so), and give students 3 shots to pass.

At that point, the credibility of the enterprise will rise. Until then, what we are doing is just a hollow show. Besides, our students don’t get degrees in “critical thinking” or “making and breaking arguments.” They get degrees in engineering, nursing, anthropology, etc.—and that’s the way we organize our faculties and curriculum as well.

Read Cliff’s full commentary, the wide ranging opinions many others and get involved in the online discussion yourself by adding your views to the conversation.

More food for thought!
Check out the oped by Charles Murray (Yes of Bell Curve fame), For Most People, College Is a Waste of Time from the August 13, 2008 Wall Street Journal.

The Higher Education Act may finally be re-authorized following a 40-4 vote of the Conference Committee last night, July 29, 2008. One provision in the legislation, which influences how accrediting agencies (Northwest Commission on Colleges and Universities accredits UNR) go about reviewing campus-wide assessment of student learning, will:

“Bar the U.S. Education Department from issuing regulations (governing higher education accreditation) designed to ensure that colleges are measuring student learning outcomes, . . .”

Other provisions will mandate transparency in areas, such as tuition and financial aid, but only time will determine how the National Association of State Universities and Land Grant Colleges (NASULGC) & American Association of State Colleges and Universities (AASCU) initiative, the Voluntary System of Accountability’s College Portrait will evolve.

The major provisions are highlighted in today’s Doug Lederman’s Inside Higher Ed article,
7 Years, 1,158 Pages. . . and Almost Done.

A Chronicle of Higher Education article, House and Senate Negotiators Approve Compromise Higher Education Act, details increased federal oversight.

During a 2008 Council for Higher Education Accreditation’s (CHEA) Summer Workshop Panel discussion, the CHEA president [Judith Eaton], an Education Department official, a key Congressional aide, and two accrediting agency leaders debated Eaton’s vision and, more fundamentally, the current state of the tension-filled system by which the federal government, accrediting agencies, states and colleges and universities seek to ensure the quality of education provided to students.

Panelist discussed the divide between those who believe federal rules should govern accrediting agencies standards for student learning and those who believe, “Institutions should be able to define for themselves what participation in postsecondary education provides to an individual.”

Senator Lamar Alexander’s Congressional Aide, David Cleary, warned that just because Congress has stepped in to limit DOE regulation . . . does not mean college leaders should assume accountability is not forthcoming. Cleary said, “I think we’re talking about a five-year time frame.”

Read the full description of the Inside Higher Ed article.

DOE may still issue regulations for accrediting agencies’ oversight of colleges’ student learning outcomes. Secretary of Education, Margaret Spellings, under pressure from Congress had agreed not to issue such regulations until the re-authorization of the Higher Education Act. As the final vote on this Act is expected by the July 4th recess, concerns mount among higher education officials, lobbyists & Congressional aides that the Department will take a ‘Last Bite at the Apple.”

With the clock ticking on the Bush administration’s time in office, many higher education officials have wondered whether (and in some cases feared that) Spellings and her aides might make a final push to institute policies that might instigate change in higher education.

Read the full article in today’s Inside Higher Ed.

Read/Respond - Cliff Adelman Comments, “You can’t sit on your hands on this one: . . .The present prospects are utter folly.”

In today’s Inside Higher Ed, Doug Lederman details the status of a House & Senate Conference Committee’s work to reconcile the HE Act by Memorial Day. Work not only remains on major difference between the House and Senate version, but one Congressional aide says, in reference to an expanding number of lists intended to improve consumer information & institutional transparency, “It seems like it is going to serve to either confuse the hell out of people or give people a false confidence or a false sense of danger.”

Regarding DOE’s growing involvement in accreditation matters, “One other closely watched area in which the draft legislation offers a compromise is in the area of accreditation, where the circulated discussion draft would continue to bar the U.S. Education Department from promulgating regulations governing student achievement (and additional areas such as curriculum and admissions practices) but would allow the agency to regulate in realms such as faculty, facilities, and student support services.”

Read the full Article & don’t miss Adelman’s commentary.

Doug Lederman’s Inside Higher ED article (April 15th), “Margaret Spelling, Where Are You?“, describes the extensive as well as varied approaches being undertaken by faculty and institutions accredited by the North Central Association to address external demands for accountability.

“We try to not just be generating data because someone at the state level is asking for it . . . We try to process it in a way that is going to serve the institution…. And many of the things we have undergone internally to improve our own processes and student learning have helped us to be able to respond quickly and agilely to state mandates.” - Stephanie Booth, Kent State Assoc. Provost

Read the full article and Cliff Adelman’s interesting comment!

AAC&U Releases National Employer Survey Findinsgs: 65% of those surveyed believe that new graduates of 4-year colleges have most or all of the skills to succeed in entry-level positions, but only 40% believe that they have the skills to advance.

Assessment: Employers want intensive, personally evaluated projects, not more testing (Item #6):

• 50% - Faculty-evaluated internships or community-based experiences;
• 35% - Essay tests that measure students’ problem solving, writing and analytical-thinking skills;
• 32% - Electronic portfolios of students’ work, including examples of accomplishments in key skill areas and faculty assessment of them;
• 31% - Faculty-evaluated comprehensive senior projects demonstrating students’ depth of skill in major & advanced problem-solving, writing, and analytical reasoning skills;

while only

• 8% - Tests that show how a college compares to others in advancing students’ critical-thinking skills; and only
• 5% - Multiple-choice test of general content knowledge

Only 7% said it would be ‘very effective’ to have the results of multiple choice tests of general knowledge, and there was little interest in tools that would compare one colleges’ graduates to another on critical thinking.

Full article & reader comments from Inside HIGHER ED
Full Survey Report

Panelists at an American Anthropology Association session on 11/29/07 say most faculty view assessment as a threat from accreditors and administration rather than a threat from federal agencies, e.g. The Spelling Commission.

Professors who gathered for the session said it’s time for a change . . . Assessment works best when faculty members are involved and it’s not a top-down mandate. They need to be the ones asking questions of themselves, each other and their own students.

Read the full article with its numerous comments in the November 30, 2007 Inside Higher Ed.

While Congress debates how much accreditors should ask of colleges in the reauthorization of the Higher Education Act, the National Association of State Universities and Land Grant Colleges (NASULGC) unveiled its ‘Voluntary System of Accountability’ (the College Portrait web template) for members - UNR is a member - to publish student demographics, costs, persistence rates, and more.

A November 12, 2007 Inside Higher Ed article, Accountability System Launched, describes the release of the ‘College Portrait’ at NASULGU’s annual meeting.

Participating institutions will also be required to report results from:

One of four surveys of student experience and perceptions:

• the College Student Experiences Questionnaire;
• the College Senior Survey;
• the National Survey of Student Engagement; or
• the University of California Undergraduate Student Experience Survey.

One of three measures of cognitive skills:

• the Collegiate Assessment of Academic Proficiency (CAAP from ACT);
• the Measure of Academic Proficiency and Progress (MAPP from ETS); or
• the Collegiate Learning Assessment (CLA from RAND).

We should design assessment, first, to support worthwhile learning . . . Standards will be raised by improving student learning rather that by better measurements of limited learning.

. . . the press to assess with a test . . .
Trudy Banta of IUPUI recounts the recent concerns regarding accountability. Although she generally views standardized tests unfavorably, Dr. Banta suggests that if such instruments are going to be used to compare institutions it would be beneficial to learn more about them. Some states have already instituted standardized testing and the Department of Education’s Commission of the Future of Higher Education has made a number of such recommendations.

“Should we prepare ourselves to imagine a time when our students’ scores on a standardized test become an important component of judging our effectiveness for promotion, tenure, and raises?”

“Will the college curriculum then be narrowed, as it has been in grades K-12, to focus students’ attention on attaining the knowledge and skills defined by the content of these tests?”

Read the Banta Article