Beware the New "Education Sector"

Over the years, Kevin Carey and I have had our tussles, most recently over whether some of his recent stances on education reform were too faithful to a business model, which I called "neoliberal."  But throughout it all, I have remained a fan of both Kevin and his shop, Education Sector, since both are known for asking hard, data-driven questions about whether higher education is meeting the needs of students from disadvantaged families.   So I am extremely disappointed to see that Education Sector has been hijacked by the conservative Right, and now clearly represents the interests of business elites, pushing free-market principles on all of education.  Kevin, to his credit, is getting the hell out of there, moving to the New America Foundation, accompanied by his talented colleagues Stephen Burd, Amy Laintinan, and Rachel Fishman.

Within a few days the change at Education Sector will be complete.  The leadership includes several consultants to the Romney campaign and members of the Hoover Institution, such as John Chubb, Macke Raymond, and Bill Hansen, who seem to believe that markets have magical powers, and that educating students is akin to making hamburgers or sauerkraut. Worse yet, Hansen is a former Bush appointee who lobbies for the Apollo Group, and has worked against every effort to contain corruption in for-profit schools.  He was president of Scantron, of the "fill in the bubble" testing industry, and has worked to advance the cause of student loan providers. And his jobs have been described as things like "creating a new education line of business...and  integrating the education services activities throughout the company into a strategic product portfolio." Stephen Burd's long been on to this guy- he is trouble.

No doubt about it, these folks will use Education Sector to advance an agenda aimed at ensuring the federal government stops helping students afford college.  They'll start by telling us that college isn't really necessary, and that financial aid is ineffective-- but they'll also do nothing to ensure public higher education becomes free. Instead, they will push free-market solutions -- mainly online education-- for other peoples' children, while probably sending their own kids to elite private schools.

So next time you see a report from Education Sector, give it a second look.  Theirs are no longer "Charts You Can Trust."  They are acts of political manipulation pushed by the hard Right.

Consider yourself warned.

Updated at 11:16 am CST. Gee, Google is so much fun.

You have read this article bill hansen / Diane Ravitch / Education Sector / for-profits / higher education / john chubb / Kevin Carey / macke raymond / mitt romney / online / privatization with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2012/06/beware-new-sector.html. Thanks!

Measuring Up? The Trouble with Debt to Degree

The following is a guest blog post by Robert Kelchen, graduate student in Educational Policy Studies at UW-Madison, and a frequent co-author of mine. --Sara

I was pleased to see the release of Education Sector’s report, “Debt to Degree: A New Way of Measuring College Success,” by Kevin Carey and Erin Dillon. They created a new measure, a “borrowing to credential ratio,” which divides the total amount of borrowing by the number of degrees or credentials awarded. Their focus on institutional productivity and dedication to methodological transparency (their data are made easily accessible on the Education Sector’s website) are certainly commendable.

That said, I have several concerns with their report. I will focus on two key points, both of which pertain to how this approach would affect the measurement of performance for 2-year and 4-year not-for-profit (public and private) colleges and universities. My comments are based on an analysis in which I merged IPEDS data with the Education Sector data to analyze additional measures; my final sample consists of 2,654 institutions.

Point 1: Use of the suggested "borrowing to credential" ratio has the potential to reduce college access for low-income students.

The authors rightly mention that flagship public and elite private institutions appear successful on this metric because they have a lower percentage of financially needy students and more institutional resources (thus reducing the incidence of borrowing). The high-performing institutions also enroll students who are easier to graduate (e.g. those with higher entering test scores, better academic preparation, etc) which increases the denominator in the borrowing to credential ratio.

Specifically, the correlations between the percentage of Pell Grant recipients (average of 2007-08 and 2008-09 academic years from IPEDS) and the borrowing to credential ratio is 0.455 for public 4-year and 0.479 for private 4-year institutions, compared to 0.158 for 2-year institutions. This means that the more Pell recipients an institution enrolls, the worse it performs on this ratio.

While even though Carey and Dillon focus on comparing similar institutions in their report (for example, Iowa State and Florida State), it is very likely that in real life (e.g. the policy world) the data will be used to compare dissimilar institutions. The expected unintended consequence is “cream skimming,” in which institutions have incentives to enroll either high-income students or low-income students with a very high likelihood of graduation. (Sara and I have previously raised concerns about “cream skimming” with Pell Grant recipients in other work.)

The graphs below further illustrate the relationship between the percentage of Pell recipients and the borrowing to credential ratio for each of the three sectors.

There is also a fairly strong relationship between a university’s endowment (per full-time equivalent student) and the average borrowing to credential ratio. Among public 4-year universities, the correlation between per-student endowment and the borrowing to credential ratio is -.134, suggesting that institutions with higher endowments tend to have lower borrowing to credential ratios. The relationship at private four-year universities is even stronger, with a correlation of -.346. For example, Princeton, Cooper Union, Caltech, Ponoma, and Harvard are all in the top 15 for lowest borrowing to credential ratios.

The relationship between borrowing to credential ratios and standardized test scores is even stronger. The correlations for four-year public and private universities are -.488 and -.589, respectively. This suggests that low borrowing to credential ratios are in part a function of student inputs, not just factors within an institution’s control. In other words, the metric does not solely measure college performance.

It is critical to note that the average borrowing to credential ratio should be lower at institutions with more financial resources and who enroll more students who can afford to attend college without borrowing. However, institutions who enroll a large percentage of Pell recipients should not be let off the hook for their borrowing to credential ratios. These two examples highlight the importance of input-adjusted comparisons, in which statistical adjustments are used so institutions can be compared based more than their value-added than their initial level of resources. The authors should be vigilant to make sure their work gets used in input-adjusted comparisons rather than unadjusted comparisons. Otherwise, institutions with fewer resources will be much more likely to be punished for their actions even if they are successfully graduating students with relatively low levels of debt.


Point 2: The IPEDS classification of two-year versus four-year institutions does not necessarily reflect a college’s primary mission.

IPEDS classifies a college as a 4-year institution if it offers at least one bachelor’s degree program, even if the vast majority of students are enrolled in 2-year programs. Think of Miami Dade College, where more than 97% of students are in 2-year programs but the institution is classified as a 4-year institution.

For the purposes of calculating a borrowing to credential ratio, the Carnegie basic classification system is more appropriate. Under that system an institution is classified as an associate’s college if bachelor’s degrees make up less than ten percent of all undergraduate credentials. The Education Sector report classifies 60 institutions as four-year colleges that are Carnegie associate’s institutions.

This classification decision has important ramifications for the borrowing to credential comparisons. The average borrowing to credential ratio by sector is as follows:

Two-year colleges, Carnegie associate’s: $6,579 (n=942)
Four-year colleges, Carnegie associate’s: $13,563 (n=60)
Four-year colleges, Carnegie bachelor’s or above: $23,166 (n=1,421)

Ten of the twelve and 20 of the top 40 four-year colleges with the lowest borrowing to credential ratios are classified as Carnegie associate’s institutions. For example, Madison Area Technical College is 54th on the Education Sector’s list of four-year colleges, but is 564th of 1,002 associate’s-granting institutions. These two-year institutions with a small number of bachelor’s degree offerings should either be placed with the other two-year institutions or in a separate category. Otherwise, anyone who wishes to rank institutions based on their classification would be comparing apples to oranges.

In conclusion: the effort in this report to measure institutional performance is a laudable one. But the development and use of metrics is challenging precisely because of their potential for misuse and unintended consequences. Refining the proposed metrics as described above may make them more useful.

You have read this article college outcomes / college performance / Education Sector / Kevin Carey / metrics / Miami Dade College with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2011/08/measuring-up-trouble-with-debt-to-degree.html. Thanks!

Federal Funding for Teacher Quality Innovation?

This is a follow-up to my post of two weeks ago about the use of Title II, Part A funding under NCLB. In these tight economic times, it is inevitable that the focus will move from spending more money on education to spending existing dollars more wisely. Currently, most school districts are not using these federal dollars in particularly innovative, let alone effective or impactful ways.

An article by Stephen Sawchuk ('Grants in NCLB to Aid Teaching Under Scrutiny') was published in this week's edition of Education Week. In part, it discusses the findings of a recent Education Sector report on this topic.

For those of you who aren't Ed Week subscribers and may not be able to access the story, here is a peek at the story:

The Teacher and Principal Training and Recruiting Fund—better known as Title II, Part A of NCLB—is the federal government’s second-largest K-12 investment, after the Title I grants for disadvantaged students. Ninety-five percent of the funds flow to school districts, and they come with few strings attached.

Although the fund has promoted some promising local practices, Title II, in general, “is not especially aligned with leading-edge [teacher-quality] efforts, and it’s the federal government’s big entry in this sweepstakes,” said Andrew J. Rotherham, the co-director of Education Sector, a Washington think tank, and the report’s author.

...

In his paper, Mr. Rotherham stakes out one conceptual approach that Mr. Obama and legislators could consider when they revise the program as part of the reauthorization of the NCLB law: to transform Title II into a fund for seeding innovations to the education human-capital continuum, and to disallow a handful of currently authorized activities, including class-size reduction.

...

Nationally representative U.S. Department of Education survey data show that districts in 2007-08 spent 6 percent of their Title II funds on professional-growth initiatives—such as mentoring programs or incentives for teachers to pursue certification from the National Board for Professional Teaching Standards—and 4 percent on recruitment, including performance-based pay and teacher loan-forgiveness programs.

More than three-quarters of districts’ Title II allocations subsidize professional development and smaller class sizes. In his paper, Mr. Rotherham deems those activities “low leverage” because they typically lack quality-control mechanisms and reinforce traditional human-capital structures, rather than altering them.

You have read this article Andy Rotherham / Education Sector / Education Week / ESEA / No Child Left Behind / teacher effectiveness / teacher quality / Title II with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2008/12/federal-funding-for-teacher-quality.html. Thanks!

Teacher Quality and Title II

Education Week published an incredibly important story this week by Stephen Sawchuk ("Spending On Federal Teacher-Quality Funds Questioned") and Education Sector issued an incredibly important report (Title 2.0: Revamping The Federal Role in Education Human Capital) on Title II, Part A dollars in No Child Left Behind. The notion of better utilizing existing resources is especially critical in light of the economic downturn and budgetary challenges which will make new resources harder to come by.

As I wrote in two recent posts ($29 Billion Buys You A New Education System? and Will The New President Support New Educators?), it is abundantly clear that Title II, Part A's $3 billion are not flowing toward the most impactful initiatives in schools and districts. Most are going to class-size reduction and professional development (of questionable quality).

In a nutshell, Education Sector recommends shifting "the federal government’s role from enabler of existing activities largely irrespective of quality to a driver of reform through strategic investments in new initiatives, institutions, and policy schemes to recruit, train, support, and evaluate and compensate teachers."

Easy stuff, right?

You have read this article budget / Education Sector / Education Week / ESEA / NCLB / No Child Left Behind / teacher quality / Title II with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2008/11/teacher-quality-and-title-ii.html. Thanks!

Teacher Mentoring and Student Achievement

UPDATE (6/28/2010): "Positive Effects of Comprehensive Teacher Induction"


Friday afternoon's blog post on Education Sector's The Quick and The Ed offers a surprisingly negative take on a new teacher mentoring study, but does raise some shrewd points about the importance of a shared vision around supporting new teachers.

As I recently discussed, last week the American Enterprise Institute featured an event at which Columbia University economist Jonah Rockoff spoke about his study of teacher mentoring in New York City. One of the principal findings in Rockoff's study is that students' math and reading achievement was higher in the classrooms of new teachers who received more hours of mentoring, supporting the notion that time spent working with a mentor does improve teaching skills.

Rockoff writes:

The magnitude of these effects are substantial, with an additional ten hours of mentoring expected to raise student achievement by 0.05 standard deviations in math (0.10 in the survey sample) and 0.04 standard deviations in reading (0.06 in the survey sample). If truly causal, these effects would lend considerable support for the notion that mentoring has an impact on student achievement.

In her blog post, Laura Guarino speaks of "slight increases" in student achievement and Rockoff's overall "tepid" results. I think that understates this finding in his study. However, I think some of Guarino's other counsel is wise and her perspective as a new teacher tremendously valuable.

Guarino writes:

If the purpose of mentoring is to provide support in order to keep good teachers and make them better, then the responsibility of a mentor must be clear to both parties involved. When there are numerous goals and assorted models of mentoring, it is clear that we need to find “a best practice” in carrying out these programs. Mentoring is one of those good ideas in theory, but is far more complicated than it seems.

As a first-year TFA teacher in Charlotte, it sounds like Guarino experienced some sporadic and haphazard mentoring. It’s an experience from which we can learn. She references four different mentors giving her advice with four different visions of what their roles were. Four mentors?!?! Egads! That might sound like an embarrassment of riches, but certainly it isn't if the mentors are operating at cross-purposes and if they haven't been trained for the role.

Guarino is correct in saying that "Mentoring is more complicated than it seems." That’s a lesson that policymakers and district leaders need to learn. It is not enough simply to require mentoring. It’s not enough merely to assign a mentor to every new teacher. There’s much more that goes into designing induction and mentoring programs to produce the desired impact on teaching and learning.

A well-designed induction program pairs a carefully selected, intensively trained mentor with each first- and second-year teacher. It provides protected time for a mentor to meet regularly with a new teacher for 1.25-2.5 hours per week. Induction is not an old-fashioned buddy system that makes everyone feel good but doesn’t provide the regular, structured, contextualized and substantive support and feedback to the beginning teacher. I cannot underestimate the importance of mentor training and time in this regard. A prepared mentor is the necessary engine to ensure that the mentoring relationship focuses on improving teaching practice rather than on providing psychological and social support alone. Like it or not, that costs money, but it’s an investment that pays dividends if done right.

A strong induction program also has a clearly articulated vision shared by all stakeholders. Key stakeholders are not just the mentor and new teacher--but also the program site administrator, instructional coaches, school principal, and district superintendent. The program vision should include supporting new teacher development and strengthening student learning—not simply reducing teacher attrition and improving teacher morale.

Finally, I should note that the despite the positive results, New York City did take some shortcuts in implementing this program. Its decision to provide mentor support only during the first year of teaching may have been a fundamental flaw. It's a short cut that many states and districts take, sometimes for financial reasons and sometimes because they fail to understand that second-year teachers have unique needs as they continue to develop into veteran educators. In the Big Apple, this decision could well have depressed the potential for additional student learning gains. New Teacher Center research [see Figure 1, page 2] from California shows that greater student achievement gains are realized as a result of intensive induction sustained over two years. That's a finding that should be investigated further in different school and district contexts.

---------------------------------------

UPDATE: Here is Alexander Russo's post on this topic.

You have read this article Education Sector / induction / Jonah Rockoff / mentoring / New Teacher Center / New York City / teacher quality with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2008/06/teacher-mentoring-and-student.html. Thanks!

The Graduation Rate Gap: Minding What Matters?

One of the dirtiest secrets in higher education is that getting in--the focus of 18 years of hard work in some families--is no guarantee of college success. The sad truth is that at many public four-year colleges and universities fewer than half of entering freshmen complete a degree within six years. And in some cases those low average rates conceal incredible disparities based on race and family income.

Kevin Carey of Education Sector has done a remarkable job at turning the nation's attention to this often overlooked problem in recent years. He helped create College Results Online which enables the average Joe to spend hours of fun picking out individual schools and seeing just how low their graduation rates can go. (Seriously, go check it out-- it's FUN!)

Most recently Carey issued Graduation Rate Watch, a report that uses six years of federal data to look at changes over time in the black-white gap in college completion. His primary point: between 2002 and 2006 some schools made significant strides in closing their gaps, while others did not. And in fact, some schools had (and have) no gap at all. Therefore, Carey concludes, the answers are clear: too many colleges are indifferent to the plight of minority students, they know what they need to do, and we just need to push them to do it.

Kevin and I agree on most things. We agree that the feds and states are paying far too little attention to low rates of college success. We agree that disparities in degree completion rates are unacceptable. And we agree that institutions of higher education need to be a part of the solution But on one crucial point we disagree: what should be done.

Drawing on a case study of a single college (Florida State) and its lovely college-support program CARE, Kevin contends that the solution to closing the college achievement gap lies in "creating incentives for institutional leaders to act on the knowledge that already exists." His goal in making this kind of statement is to get educators to step up-- they know what needs to be done, and what remains is to take action.

But the researcher in me hesitates to endorse this plan. Here are a few reasons why:

1. There is little evidence that differences in students' college graduation rates are primarily or even substantially attributable to school-level factors. Go back to the Coleman report--his intelligent hypothesis was that inequities in schools due to segregation were driving black/white differences in educational outcomes. But instead he found that family background and other student-level characteristics drove far more of the variation in student achievement. Yes, some school-level factors matter--among them teacher quality--but empirical evidence indicated that changing schools themselves would not diminish educational inequality much.

Why would we think things would be different in higher education? The typical argument goes like this: We know that institutional choices and policies (financial aid, academics, advising, faculty, incentives, student engagement, etc) matter in terms of student learning, retention, and attainment. Given our diverse, decentralized higher education "system" there is substantial inter-institutional variation in the quality of those choices and policies. Therefore, "institutional effects" should be significant and real.

Well, they may be significant, they may be "real," but there is little hard data to say that they are large and should be the main focus of policy reform. I'm not saying that colleges and universities aren't different from one another in ways that matter, but because higher education is not compulsory and its gates are closely guarded by the gods of test scores and $$ the level of stratification in the college entry process is so intense that students attending different schools are very different from one another. There have been very few studies of institutional effects in higher ed that deal with that selection issue appropriately. I strongly suspect that if a good study were done (and given data constraints it'd be hard to do), we'd find that policymakers would get the biggest bang for their buck by working to reduce race, class and gender disparities in who goes to college and where they go.

2. Another important concern: There can be unintended consequences when you push colleges to close achievement gaps. Take this scenario: You've got 30 percentage point gap in college completion, with 40% of black students finishing a degree in 6 years, compared to 70% of whites. You want to close that gap. So you tell your admissions director to solve the problem. And he does: by being more selective in which black kids he admits. Admitting only very highly-prepared black students may mean a decline in the number of black kids on campus, but it's likely to increase the completion rate of those who are admitted. And if he coupled his efforts with a greater effort at targeted recruiting of minorities (including directing merit-based scholarships at them) , he might succeed in simultaneously increasing diversity and completion rates. Not a bad thing per se (perhaps) but definitely not what Carey has in mind...

3. There is not a shred of evidence that FSU's CARE program actually works to increase student retention or achievement. No evaluation that takes into account who they serve to start with, nada. Yes, they've got data indicating that CARE students start with lower entering test scores than other students and end up with higher retention rates, but NO--this doesn't automatically mean that success is due to the program. What if CARE students are simply more motivated than other students-- meaning that they were more likely to be retained from the start? We don't have any way to account for that in the absence of a good ol' random assignment trial where we assign students at random to participate in CARE. Now that would be worth doing!


Of course I understand that Kevin's goals are different from mine. Ultimately he operates in a world where being 51% sure that you're right when making policy recommendations is sufficient-- after all, if we don't advocate for changes in colleges, there might not be attention paid to higher ed at all (or dollars spent). But in my little research world, I need to be much more confident that the recommendation I'm making will be effective in creating the kind of change we aim for--AND, I want to focus on what will make the biggest difference. And there's simply little hard evidence to suggest that adopting a program like FSU's would help to close the black/white gaps in completion at other schools.

Why not--instead of "tinkering at the margins" with school reform--tackle the larger and more systemic problems like ending poverty and racial segregation? (Why is it that educators and educational policymakers have a hard time seeing what happens outside of schools as being part of their domain?) Why not make higher education compulsory, and in doing so greatly increase the motivation to make high school meaningful? Why not re-envision the entire system and create a p-16 structure which compels higher ed and secondary ed to take an interest in one another?

The last thing we need to do is fan the flames of fears about accountability, encouraging colleges and universities to turn even more inward, engaging in ego-think, and directing their attention at self-preservation rather than student success. We do want them to do more-- and they can be an important part of the process of change-- but they need direct their attention at the things that matter most.

You have read this article college / college completion / Education Sector / graduation rate / higher education / Kevin Carey with the title Education Sector. You can bookmark this page URL http://apt3e.blogspot.com/2008/04/the-graduation-rate-gap-minding-what.html. Thanks!