Is Our Students Learning?

Remarkably, one of the topic's of yesterday's blog post (and another I wrote two years ao)-- the limited learning taking place on many college campuses-- is the subject of a New York Times op-ed today. Titled, "Your So-Called Education," the piece argues that while 90% of graduates report being happy with their college experience, data suggests there's little to celebrate. I urge you read it and its companion op-ed "Major Delusions," which describes why college grads are delusional in their optimism about their future.

We don't regularly administer the Collegiate Learning Assessment at UW-Madison, the test that the authors of the first op-ed used to track changes in student learning over undergraduate careers. From talking with our vice provost for teaching and learning, Aaron Brower, I understand there are many good reasons for this. Among them are concerns that the test doesn't measure the learning we intend to transmit (for what it does measure, and how it measures it, see here), as well as concerns about the costs and heroics required to administer it well. In the meantime, Aaron is working on ways to introduce more high-impact learning practices, including freshmen interest groups and learning communities, and together with colleagues has written an assessment of students' self-reports of their learning (the Essential Learning Outcomes Questionnaire). We all have good reason to wish him well. For it's clear from what we do know about undergraduate learning on campus, we have work to do.

The reports contained in our most recent student engagement survey (the NSSE, administered in 2008) indicate the following:

1. Only 60% of seniors report that the quality of instruction in their lower division courses was good or excellent.

This is possibly linked to class size, since only 37% say that those classes are "ok" in size -- but (a) that isn't clear, since the % who says the classes are too large and the % that say they are too small are not reported, and (b) the question doesn't link class size to quality of instruction. As I've noted in prior posts, it's a popular proxy for quality but also one that is promoted by institutions since smaller classes equates with more resources (though high-quality instruction does not apparently equate with smaller classes nor high resources). There are other plausible explanations for the assessment of quality that the survey does not shed light on.

2. A substantial fraction of our students are not being asked to do the kind of challenging academic tasks associated with learning gains.

For example, 31% of seniors (and 40% of freshmen) report that they are not frequently asked to make "judgments about information, arguments, or methods, e.g., examining how others gathered/ interpreted data and assessing the soundness of their conclusions." (Sidebar-- interesting to think about how this has affected the debate over the NBP.) 28% of seniors say they are not frequently asked to synthesize and organize "ideas, information, or experiences into new, more complex interpretations and relationships." On the other hand, 63% of seniors and 76% of freshmen indicate that they are frequently asked to memorize facts and repeat them. And while there are some real positives-- such as the higher-than-average percent of students who feel the university emphasizes the need to spend time on academic work-- fully 45% of seniors surveyed did not agree that "most of the time, I have been challenged to do the very best I can."

3. As students get ready to graduate from Madison, many do not experience a rigorous academic year.

In their senior year, 55% of students did not write a paper or report of 20 pages or more, 75% read fewer than 5 books, 57% didn't make a class presentation, 51% didn't discuss their assignments or grades with their instructor, and 66% didn't discuss career plans with a faculty member or adviser. Nearly one-third admitted often coming to class unprepared. Less than one-third had a culminating experience such as a capstone course or thesis project.

4. The main benefit of being an undergraduate at a research university--getting to work on a professor's research project-- does not happen for the majority of students.

While 45% of freshmen say it is something they plan to do, only 32% of seniors say they've done it.

Yet overall, just as the Times reports, 91% of UW-Madison seniors say their "entire educational experience" was good or excellent.

Well-done. Now, let's do more.


Postscript: Since I've heard directly from readers seeking more resources on the topic of student learning, here are a few to get you started.

A new report just out indicates that college presidents are loathe to measure learning as a metric of college quality! Instead, they prefer to focus on labor market outcomes.

Measuring college learning responsibly: accountability in a new era by Richard J. Shavelson is a great companion to Academically Adrift. Shavelson was among the designers of the CLA and he responds to critics concerned with its value.

The Voluntary System of Accountability, embraced by public universities who hope to provide their own data rather than have a framework imposed on them. Here is Madison's report.

On the topic of students' own reports of their learning gains, Nick Bowman's research is particularly helpful. For example, in 2009 in the American Education Research Journal Bowman reported that that in a longitudinal study of 3,000 first year students, “across several cognitive and noncognitive outcomes, the correlations between self-reported and longitudinal gains are small or virtually zero, and regression analyses using these two forms of assessment yield divergent results.” In 2011, he reported in Educational Researcher that "although some significant differences by institutional type were identified, the findings do not support the use of self-reported gains as a proxy for longitudinal growth at any institution."

As for the NSSE data, such as what I cited above from UW-Madison, Ernie Pascarella and his colleagues report that these are decent at predicting educational outcomes. Specifically, “institution-level NSSE benchmark scores had a significant overall positive association with the seven liberal arts outcomes at the end of the first year of college, independent of differences across the 19 institutions in the average score of their entering student population on each outcome. The mean value of all the partial correlations…was .34, which had a very low probability (.001) of being due to chance."

Finally, you should also check out results from the Wabash study.

You have read this article academic standards / New Badger Partnership / University of Wisconsin-Madison with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2011/05/is-our-students-learning.html. Thanks!

Daily Drivel

It's hard to believe that the Wall Street Journal fancies itself a national newspaper while publishing this largely baseless, political clap-trap on its editorial page:

But national standards are no substitute for school choice and accountability, which are proving to be the most effective drivers of academic improvement.

First of all, to frame education reform as pitting national standards against choice/accountability is ridiculous on its face. It is a false choice. Plus, the Obama Administration's reform blueprint is so much more broad than that. About the only thing that the WSJ editorial gets right is in saying that national standards "won't magically boost learning" by themselves.

Secondly, the WSJ appears to be falling into the "silver bullet" mentality all too prevalent among simplistic education reformers. "Just run schools like a business!" Or, "[INSERT pet approach] is the answer." Yes, we've been down that road before .... small schools, merit pay, open classrooms. The WSJ apparently wants to contribute choice and accountability to the junkyard of spent shell casings.

Third, where is the research evidence to suggest that school choice and accountability should be in the driver's seat? The editorial offers no evidence. The presence of publicly funded vouchers is no panacea. Just look at Milwaukee's experience (here and here). At the recent meeting of the American Education Finance Association, the U.S. Department of Education's senior adviser Marshall 'Mike' Smith offered evidence that rates of gain in student test scores were lower after No Child Left Behind became law than before. We chided Margaret Spellings last year for touting the successes of NCLB on similar grounds. So much for bare-bones accountability.

Does the Wall Street Journal have any editorial standards? Or any shame?

UPDATE: Read Claus von Zastrow's take on this editorial on Public School Insights: "It doesn't pass the laugh test."

You have read this article academic standards / accountability / ESEA / NCLB / school choice / Wall Street Journal with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2010/03/daily-drivel.html. Thanks!

Fight The Power!

Dana Goldstein raises some serious questions and concerns in her American Prospect article ("Testing Testing") about the process of developing national academic standards. The process is dominated by three organizations--two (ACT and the College Board) with a proprietary interest in ensuring that assessments are a featured component of any final product.

The problem is that the initiative's co-signers aren't just state governments--they are also testing groups: Achieve, a nonprofit that advocates for more effective standardized tests; the College Board, maker of the SAT; and ACT, which administers a competing college-entrance exam. Right now, the College Board and ACT have little engagement with the K-12 education sector. They do, however, have ample experience creating and administering national exams. And there is little doubt that one goal of this national-standards process is to create standardized tests--not one single national test but perhaps two or three options from which states can choose.

As oligopolists, it makes total sense for the College Board and ACT to be eyeing, together, expansion into the immense K-12 assessment market. But given these testing companies' track records, it is worth asking if this is a wise idea. A number of studies have found SAT scores are far less effective than high school grades in predicting how well students will perform in college, and professors say standardized-test prep does little to teach students the research and critical thinking skills they will need at the college level. Because of these shortcomings, an increasing number of colleges--led by the giant University of California system--have made standardized test scores optional for admission.

It would be a shame if national education reform further cemented a system in which passing standardized tests is the goal of learning.

While others (including Dan Brown) have pointed out that only one classroom teacher has a seat at the table, Goldstein follows the money, so to speak. I am disappointed, although not surprised, that the national organizations leading this effort have basically turned it over to Testing, Inc. The corporate boards of both the Council of Chief State School Officers and the National Governors Association are littered with representatives of the student assessment industry--ETS, McGraw Hill, and Pearson in the case of CCSSO; and ACT, the College Board, ETS, and Pearson in the case of NGA. To their defense, both CCSSO and NGA list these organizations directly on their respective web sites. As a former employee at NGA, I also can honestly say that their funding did in no way impact the substantive advice provided to the nation's Governors when I worked there. But does it provide these companies ready access to Governors and their senior staff at regular meetings? Sure. Does it raise questions about their role in this standards-setting process and create the appearance of bias? Absolutely.

Of the 29 slots on the mathematics and English-language arts Work Groups, 15 are taken by employees or affiliates of ACT and The College Board. Another seven slots are occupied by Achieve, Inc. (Some individuals serve on both Work Groups.) Of the remaining seven slots, two are filled by America's Choice, two by Student Achievement Partners, and single seats by a communications firm, a consultant, and a professor. In addition, 37 individuals serve on twin Feedback Groups for both math and English/LA standards. They are overwhelmingly higher education faculty. Of the 19 members of the math feedback group, 15 represent higher education with a single k-12 teacher in the mix. Of the 18 members on the English/LA feedback group, 14 are professors and there is one "instructional performance coach" from a public charter school as well.

The decision to cut k-12 educators out of the standards development process contrasts sharply with the rhetoric of President Obama and Secretary Duncan about including educators in the development of education reforms. Indeed, it would "be a shame" if Testing Inc. rode this gravy train to the (hopefully not) inevitable conclusion suggested by Goldstein's article. Of course, in the end, it is the product rather than the process that really matters. In this case, one can hope that some of the participants' potentially parochial and proprietary interests don't define the outcome or the intent of the entire effort. The standards should be developed based on what is best for students and how such standards can best be utilized by educators -- not to ensure their ease in being converted into multiple-choice tests.

Hat tip to TWIE.

UPDATE: See Education Week story (7/30/2009).

You have read this article academic standards / ACT / American Prospect / Assessment / College Board / Council of Chief State School Officers / Dana Goldstein / National Governors Association / teacher with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2009/07/fight-power.html. Thanks!

Evolution Lives

On a tie vote, it appears that the creationists on the Texas State Board of Education failed in their attempt to prevent the teaching of evolution in science classes. However, they managed to make some mischief, requiring science teachers to evaluate critically a variety of scientific principles like the Big Bang, for example.

Here is additional background, as reported here on Wednesday ("Will Darwin Take It On The Chin In Texas?").

The New York Times ("Defeat And Some Success For Texas Evolution Foes") and the Dallas Morning News ("Split vote upholds Texas education board ruling to ax evolution 'strengths and weaknesses' rule") have the full story.

Board members deadlocked 7-7 on a motion to restore a longtime curriculum rule that "strengths and weaknesses" of all scientific theories – notably Charles Darwin's theory of evolution – be covered in science classes and textbooks for those subjects.

The tie vote upheld a preliminary decision by the board in January to delete the strengths-and-weaknesses rule in the new curriculum standards for science classes that will be in force for the next decade. That decision, if finalized in a last vote today, changes 20 years of Texas education policy.

Because the standards spell out what must be covered in textbooks, science educators and publishers have been monitoring the Texas debate closely. As one of the largest textbook purchasers in the nation, Texas influences what is sold in other states.

Now if they could only agree on how old the earth really is.

You have read this article academic standards / creationism / evolution / science / Texas with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2009/03/evolution-lives.html. Thanks!

Will Darwin Take It On The Chin in Texas?

The Washington Post editorial page ('Strengths and Weaknesses') weighs in on the creationism debate in the Lone Star State.

Starting today, the state's board of education will consider whether the phrase "strengths and weaknesses" should remain deleted from the state's science standards. Debating strengths and weaknesses of various scientific theories might sound reasonable until you learn that those are supportive buzzwords for people who doubt evolution and want creationism taught in the classroom.

The force behind restoring the "strengths and weaknesses" language, which was stripped from the science standards in January after two decades, is Don McLeroy. He's the chairman of the State Board of Education. He is also a "young earth creationist" who believes the Earth was created by God no more than 10,000 years ago. Never mind plenty of scientific evidence that the planet has been around for a few billion years. The scary thing is that what's happening in Texas is by no means isolated.

So, in a state of 24 million people, this Mr. McLeroy is the best candidate that Republican Governor Rick Perry could find for the job of chair of the State Board of Education? Scary. Purposefully scary, I'll bet.

For more background, see here (6/3/08), here (7/21/08), here (1/22/09), and here (1/23/09).

You have read this article academic standards / creationism / evolution / governor / Rick Perry / science / Texas / Washington Post with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2009/03/will-darwin-take-it-on-chin-in-texas.html. Thanks!

Obama Offers Details on Education Plan

The Chicago Tribune is reporting that President Obama will unveil his education plan today. Reportedly, it will address performance pay for teachers, higher academic standards, dropout prevention programs, and direct aid for college students. He will not propose any legislation nor will he address the reauthorization of the Elementary and Secondary Education Act (No Child Left Behind) at this time.
-------------------
UPDATE: From The New York Times Caucus Blog:

President Obama said Tuesday that the nation must overhaul its education system and dramatically decrease the drop-out rate among students to remain competitive in the global economy.

In an address to the Hispanic Chamber of Commerce, Mr. Obama issued a challenge to states to increase the quality of reading and math instruction to keep American students at pace with other countries....

The president challenged teachers unions, renewing his support for a merit-based system of payment.... “It means treating teachers like the professionals they are while also holding them more accountable,” Mr. Obama said. “New teachers will be mentored by experienced ones. Good teachers will be rewarded with more money for improved student achievement, and asked to accept more responsibilities for lifting up their schools.”

You have read this article academic standards / Barack Obama / dropout prevention / Education / financial aid / Performance Pay / President / teacher pay with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2009/03/obama-offers-details-on-education-plan.html. Thanks!

Creationism Redux -- We're Not in Kansas Anymore

Last month, I took aim at Florida and Kansas for prostelyzing through science standards.

Today the Chronicle of Higher Education reports that the standards front is not the only place where the battle between creationism and evolution is waged. It is also waged in the classroom. According to a recent Penn State survey, one in eight high school biology teachers report teaching creationism as "a valid, scientific alternative" to evolution. Further, one in six believe that "God created human beings pretty much in their present form at one time within the last 10,000 years or so."

Yikes. I wonder if evolution is taught in theology classes?

You have read this article academic standards / creationism / evolution / science with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2008/05/creationism-redux-we-not-in-kansas.html. Thanks!

AFT: Raising The Standards

Last week the American Federation of Teachers released a report on state academic standards. In Sizing Up State Standards 2008, the AFT analyzed state standards in English, math, science and social studies. States met the AFT criteria if their standards were clear, specific and content-focused. If 8 pages is too daunting, here's the press release.

Coming out on top -- with a perfect score -- in the AFT analysis was Virginia. Not bad for a right-to-work state! The laggards that "lacked clear criteria for any grade or subject" were Colorado, Illinois, Iowa, Montana, Nebraska, Pennsylvania and Wisconsin. So I'm guessing this group doesn't get an 'A' for effort?

More important than its ranking of the states are the report's recommendations for improving content standards and its recommendation that states provide instructional guidance and teacher resources to help teachers bring the standards into the classroom.

As I mentioned in this recent post ("Teaching To The Test"), one of the benefits of a coherent set of academic standards is that it enables teachers to prepare their students for assessments without resorting to explicitly teaching to the test, or "drill and kill," as the AFT calls it. Clear standards can also help teachers make informed decisions about their professional development and allow them to work together to address student needs. That sure beats Underwater Basket Weaving.

Maybe because this topic isn't as sexy as high school exit exams or performance pay, I haven't seen much coverage of this report. But academic standards should be the engine that drives the k-12 train.

You have read this article academic standards / AFT / Virginia with the title academic standards. You can bookmark this page URL http://apt3e.blogspot.com/2008/05/aft-raising-standards.html. Thanks!