Educational Law Center's Analysis of NJ School Report Card

ANALYSIS OF NJ DEPARTMENT OF EDUCATION'S NEW SCHOOL PERFORMANCE REPORTS FINDS FAULT ON MANY LEVELS WITH METHODOLOGY AND INACCURATE DATA

On April 10, 2013, the NJ Department of Education (DOE) released their much anticipated, and much delayed, School Performance Reports, which replace the State School Report Cards.

In releasing the Performance Reports, the DOE claimed they will "help users better understand school performance in the context of state performance and the performance of similar 'peer schools.'"

The Performance Reports, however, fail to live up to this claim. Unlike the School Report Cards, the new reports are dense, confusing and needlessly complex. NJ school administrators have already raised serious concerns about inaccurate data and the convoluted and controversial school "peer rankings."

Most importantly, the complexity of the Performance Reports defeats their basic purpose: to give parents and taxpayers key information about the overall performance of their public schools and districts - successes, gains and challenges. Instead, the reports use very complicated methods of sorting and comparing individual schools with "peer" groupings, statewide averages and other benchmarks. This complexity makes the reports difficult, if not impossible, for parents, concerned citizens, lawmakers and others to understand and use to engage in positive efforts to support New Jersey's public education system.

The cornerstones of the new Performance Reports are comparisons of individual schools' test scores, graduation rates and other indicators with schools that supposedly share similar student enrollment characteristics. The DOE has decided to no longer use District Factor Groups (DFG) for comparison. DFGs placed districts, not schools, into eight groups based on the socioeconomic conditions of the communities they served. Instead of the DFGs, the DOE is using a methodology called "Propensity Score Matching," which creates a list of "peers" for each school in New Jersey, grouping schools together based on shared demographic characteristics, namely student poverty, limited English proficiency, and Special Education classification.

However, the DOE has made some questionable analytic decisions that result in comparisons among schools that actually vary quite dramatically in terms of their student makeup. This variation in so-called "peer" groupings of schools has generated confusion and frustration among local educators and stakeholders.

In addition, the DOE took the additional step of comparing each individual school to both its "peers" and the state overall using percentile ranks. The reports compare a school's position relative to other schools using a scale from zero to ninety-nine, representing the percentage of "peer schools" that school is outperforming.

The DOE's use of this method creates problems because percentile ranks are relative, or in other words a zero-sum game. A school can only be seen as successful, or "highly performing," if it is outpacing its "peer schools," regardless of its actual achievement. The DOE's failure to provide an adequate context for these rankings means users will have no idea about the absolute distance between a school ranked at the bottom and one ranked at the top. The schools may vary widely in performance, or hardly at all. Without offering any additional data on the range of scores, the user is unable to determine how meaningful those rankings are.

The DOE then goes further by labeling schools using an even broader categorization of the percentile rankings. The computer-generated "school narratives" assign schools to one of five performance categories ranging from "very high" to "significantly lagging." This means that, regardless of absolute achievement, many schools are labeled as "lagging" simply because they are on the lower end of their peer group, not because they are underperforming in any meaningful sense.

For example, if a school has a proficiency rate of 95%, but the majority of its peers score even higher, this school will have a low percentile ranking and will be labeled as "lagging," despite a high level of achievement. In another scenario, a school may have a proficiency rate of 75% and a low peer percentile rank, but could be separated from its top performing peer by just a few, or as many as 25, percentage points.

In choosing to present the data in this way, the DOE has created a framework of competitive rankings and an emphasis on labeling performance as "lagging," even among the state's highest performing schools. The reports do not give parents clear information to realistically judge their children's schools' performance, and they burden school administrators with the unforgiving task of explaining the complicated and sometimes contradictory classifications.

"The over-emphasis on complex rankings is consistent with NJ Education Commissioner Christopher Cerf's continuing narrative of 'failing public schools' when, in fact, New Jersey's public schools are among the best in the nation," said David Sciarra, ELC Executive Director. "Rather than helping facilitate community conversations and collaborative efforts to improve our schools, the new Performance Reports are clearly designed to justify the Christie Administration's agenda of cutting State investment in public education and imposing heavy-handed, top-down interventions from Trenton."

Using the DOE's own labeling, the new Performance Reports are "significantly lagging."

Education Law Center Press Contact:

Sharon Krengel

Policy and Outreach Director

skrengel@edlawcenter.org

973-624-1815, x 24

Is Right-Wing School Reform (Texas) Toast?

On occassion I find it necessary to share someone's perspective.  I picked this up from AASA and consider it worth reading:

by Jeff Bryant

It’s a conventional wisdom among Democrats to write off the state of Texas as a land of gun wielding troglodytes who genuflect to Rush Limbaugh and swill Fox News Kool-Aid. (Full disclosure: I was born and raised in the Lone Star State.)
But it may surprise most Democrats that the education policies that our current Democratic administration advances were, in a large part, invented in the oh-so awful red state of George W. Bush and Rick Perry.
The widespread idea that government operatives working in cubicles buried deep in the bowels of state capitals can monitor the “effectiveness” of schools in the hinterlands of the country was a scheme born and enacted first in a state known to be among the most oppressive in its treatment of people who Democrats like to refer to as “the least of these.”
But what happened this weekend in the Texas capital of Austin revealed a groundswell of resistance, from multiple political factions, against what has been heretofore defined as “education reform.”
A rally that brought thousands of people into the streets to protest deep cuts to the state’s education budget became a mass outcry against education policies that enforce high-stakes testing and accountability systems.
Education historian Diane Ravitch declared Texas the place where reform “madness” started and where “the vampire gets garlic in its face and a mirror waved and a stake in its heart.”
Former Texas Education Commissioner Robert Scott talked about turning in his “reformer card” and described promoters of school accountability schemes as people who are “selling two ideas and two ideas only: No. 1, your schools are failing, and No. 2, if you give us billions of dollars, we can convince you [of] the first thing we just told you.”
And Texas school superintendent John Kuhn called the pushback to school reform measures, “our San Jacinto.”
If Texas set the precedent for the last 20 years of education governance, is it now the state about to hurl the current reform model into the dustbin of history?
A Texas-Sized Mess
A recent article in that bastion of radical leftist thought, The American Conservative, took us “back in time” to recount how education policies that became the law of the land got their start in cowboy culture.
The author of the article, Texas Workforce Commissioner Tom Pauken, explained, “For the past two decades, excessive emphasis on high-stakes standardized testing and a one-size-fits-all focus on preparing all students for college came to dominate education policy in Texas and later, in Washington, D.C. with the passage of the Bush-Kennedy “No Child Left Behind” legislation.”
To trace this history, Pauken actually dialed his time machine back even further to the 1980s when computer mogul and zany presidential candidate H. Ross Perot pushed for a “basic skills test” requirement for earning a Texas high school diploma. A test-based accountability system gained momentum in the 1990s when state lawmakers decided to use test scores and passing rates to categorize schools as “Exemplary,” “Recognized,” “Acceptable,” and “Low Performing.” (Sound familiar?) http://usatoday30.usatoday.com/news/education/2010-03-13-education13_ST_N.htm
Pauken noted, “These categories … had little to do with measuring whether schools were preparing students for success in college or for meaningful employment. But the labels played well from a public-relations standpoint.”
Then, during the Bush governorship, local school districts throughout Texas ratcheted up their attention to the “performance measurements put in place by the state particularly the testing system.”
Now, 15 years later, according to Pauken, “The state’s one-size-fits-all accountability system pressures school districts to spend an inordinate amount of time teaching to the test. As one teacher told me, it all becomes a numbers game to get the most students to pass the single test.”
The test driven approach, according to Pauken, has led to a narrowed curriculum that has produced “worker shortages in the skilled trades,” declines in student performance on college entrance exams, and “a serious problem with high school dropouts.”
The Myth Of The Texas Miracle
Nevertheless, the Texas approach to education policy provided the model for accountability measures pushed by the Obama administration’s Race to the Top and other measures.
Writing at the website of “liberal” MSNBC, Jason Stanford recounted pretty much the same history that Pauken imparted.
Even when scores on the state assessments rose, Stanford explained, “SAT scores dropped. Researchers discovered that the Texas tests designed by Pearson primarily measured test-taking ability.” And “over all Texas lost ground to the rest of the country.”
Both men pin a lot of the blame for test-crazed education policies on a Democrat, Sandy Kress, Bush’s chief education adviser. According to Stanford, Kress convinced Bush, “The ‘soft bigotry of low expectations’ was holding back minority students in failing schools. His solution: if Texas made all schools give the same tests, the state could direct resources where they would do the most good, and eventually African-American and Hispanic kids would catch up to the white kids. It was a great theory, and initially the scores rose.”
This became known as the “Texas Miracle,” according to Stanford, and once Bush became president, “Kress lobbied Sen. Ted Kennedy to add bipartisan legitimacy” to NCLB, which then “spread the Texas Miracle to the other 49 states.”
The Texas Miracle started to collapse when CBS News exposed Texas school officials routinely hiding drop out figures.
But Republicans and Democrats alike remained united in thinking that test pressures would eventually yield higher achievement levels for all students. But if that were indeed the case, wouldn’t those higher levels have started to become reality in the place were test pressures have been in place the longest?
Holding School Accountability To Account
To answer that question, Texas-based education professor Julian Vasquez Heilig has spent a lot of time examining the results of the Texas education regime. Writing at his own website, he found, over the past decade, the state’s students have performed “poorly,” relative to other states, on the benchmark National Assessment of Education Progress exam, a.k.a “The Nation’s Report Card.
“Texas dropped 21 spots in 4th grade math, four spots in 4th grade reading, and eight spots in 8th grade reading,” Heilig observed.
“As a former employee of the Houston Independent School District, we inside the belly of the beast had access to our data and knew accountability hadn’t delivered on the scale that was being promoted in the popular press,” Heilig explained.
When Heilig and other reform-doubters warned testing pressures were producing “teaching to the test, push-out of children, and the narrowing curriculum,” they were summarily dismissed by those “still drinking the high-stakes testing and accountability Kool-Aid.”
“The reason why we’re seeing, well, what we’re seeing, after 10 years of No Child Left Behind is the fact that we didn’t close the gaps, the fact that our graduation rates haven’t gone anywhere, our dropout rates haven’t improved because Texas never did that in the 1990s,” said Heilig. “Accountability had never delivered that. It had never done it. And that’s why over the last 10 years now that we have Texas-style accountability and policy in the whole United States, the reason why it didn’t deliver is because it never delivered in Texas then.”
Testing Backlash Breaks Out
The extent of the test mania now appears to know no bounds.
A recent article in The New York Times reported that gym teachers around the country are being forced to incorporate test-prep into PE by teaching reading, writing and arithmetic as well as sports and exercise.
In Chicago, kindergartners may spend up to a third of their class time taking tests.
Educators, parents, and students are pushing back – not just in Texas, but around the country.
Prominent and respected school superintendents from around the country are now speaking out against the damage being done by over-testing plus the misuse of testing in Charlotte, NC, Montgomery County, MD, and Sacramento.
A test boycott started by teachers at a high school in Seattle drew national press. Parents and students joined in support of the teachers, and now the boycott has spread to Portland, OR.
High school students in Providence, RI recently staged a “zombie protest” to protest a high stakes test required for graduation.
“We are finally waking up,” Heilig concluded in his blog post cited above.
The Next “Education Bipartisanship”?
So with both conservatives and liberals questioning the whole school accountability movement, Democrats need to reconsider their support for these flawed policies.
The notion of accountability came from a desire – approved by both political parties – to create a mechanism to ensure that schools everywhere didn’t overlook the rights of poor and minority children to receive the same quality of education their white, better-off peers get.
More than a decade after NCLB became law, the achievement gap hasn’t closed, schools have become more segregated, and there’s evidence that test-driven accountability mandates are doing irreparable harm to students everywhere.
People who happen to actually know something about education have proposed alternatives to the testing craze. Democrats who want to avoid getting blind-sided by the next bipartisan agenda for education had better start checking those alternatives out.

Securing America's Schools and Her Future

In the mid 1990’s school administrators nationwide were concerned about an upward trend in violence and drug use among teenagers.  As a result school officials embraced the idea of adding the position of School Resource Officer (SRO) in America’s public schools.    The inclusion of police in a school setting was a relatively new concept and defining a proper role for armed police in public schools was problematic.  At that time I wrote an article for the National Association of Secondary School Principals titled “The Legal Implications of School Resource Officers in Public Schools” detailing the conflicts that confronted police officers in public schools.  I recall a school board meeting at that time in Williamsburg/James City County Public Schools where enraged parents insisted that any police officer assigned to a public school not carry a gun.   Imagine that!

Fast-forward twenty years.  Two decades of school shootings have redefined the climate and landscape for public schools.  The most recent tragedy in Newtown, Connecticut fueled a national debate regarding the need for teachers to carry  guns and a serious discussion concerning the National Rifle Association’s idea to place armed police officers in every school. All this even though teen violence has trended downward in the past decade.  So the question becomes:  How much security is necessary to protect our children in schools?

The answer to the above may have to be reframed: How much security personnel can public schools afford?  Recent economic conditions and shortfalls in state budgets have had a negative impact on local school budgets.  Many public schools are struggling to maintain the current level of educational services and educational personnel during the downward trend in public school financing.  Adding a police officer in every school will be an expensive proposition and needs to be weighed against many competing priorities.

One could argue that while adding armed personnel to public schools may be politically appealing, it may also create an atmosphere of apprehension that redefines school climate and culture.  There is no simple solution to such a complex problem. What may be necessary are a variety of approaches to addressing the safety issues concerning school facilities and the participants who occupy them.

The proliferation of security cameras and the infusion of digital technology have enhanced our ability to monitor school facilities.  More and more sophisticated electronic equipment designed to limit access to buildings is evolving and in use.  Identification badges, visitor sign in protocols, and criminal background checks for all school personnel inclusive of vendors and contractors are emerging in public schools. 

Monthly school safety and security drills, usually conducted in collaboration with local police authorities are now commonplace.   The employment of School Security Officers (SSO) for use in public schools appears to be on the rise in order to monitor school buildings and grounds.  If federal funding is provided, educational administrators will consider bringing School Resource Officers (SRO) back into school facilities. 

Sadly, all of the above items may not prevent another school tragedy like the one in Newtown, Connecticut.   This is the price of a free and uncensored society where individuals have unlimited access to the internet and information that potentially validates any radical ideology or perversion they choose to pursue.  Limiting access to assault weapons may help and President Obama appears to be leading the charge on this effort in spite of strong resistance from the N.R.A.  There are no simple solutions to a very complex issue and government regulations, executive orders, or decrees may not provide the ultimate solution.

Much will be written about the mental condition of individuals who commit horrific crimes, mass murders, and other travesties.  What we can do to protect our children and society from the rage that drives an individual to the brink of insanity?  This will most likely be left to local communities to decide what approaches are feasible and affordable.   There is no one-size-fits-all solution to the security challenges confronting public schools and other public facilities.

Public awareness of how difficult it is to secure the safety of individuals and particularly our children needs to evolve into public support for American public schools in general.  We need to realize and acknowledge how valuable public education is to the whole fabric of American society. Living in a free society carries risks as well as rewards and our charge is to find a way to mitigate the threats against civility.  Public education itself may provide the best answer.

It is paramount that we put the financial resources in place to ensure that we can accomplish the mission of developing a well-balanced generation of informed citizens for America’s future.  To accomplish this will require a wholesale shift in the perception regarding what we are capable of doing in America’s public schools to ensure safe conditions for the moment and stability for the future.  Our very own safety and security as a society will be affected by our willingness to make a significant investment in public education.  What do you think?