Search Right-Wing Genius's Blog

Showing posts with label The Washington Post. Show all posts
Showing posts with label The Washington Post. Show all posts

Wednesday, November 12, 2014

WaPo: In nearly every race, Republican Senate candidates outperformed the polls

Here's another thing for me to file under "I wanted to do something like this, but now I don't have to because someone else did it."
 
Washington Post reporter Philip Bump has a short but sweet post on The Fix that includes this neat little graphic:
A week before Election Day, I posited on the accuracy of the polls in key Senate races and compared candidates' poll positions in Senate contests in the same states four years ago with the 2010 election results. I found that, in every U.S. Senate race except the one in Colorado, the candidate who was ahead in the RealClearPolitics rolling average of polls won, but in many states, the polls greatly exaggerated or undertold the winner's eventual margin of victory. By comparison, Bump's analysis shows that, this year, the polls were more skewed in one direction, although - with the exception of North Carolina - they were correct about who would win.
  
The most important line from Bump's piece was its last sentence: "In this case, the polls largely predicted the correct winners, but -- with select exceptions such as the Des Moines Register poll last weekend that showed Joni Ernst (R) ahead by as wide a margin as occurred -- the scale of the victories was way off." This is what I suggested could be the case in my October 28 post.

Tuesday, October 28, 2014

What 2010 Can Tell Us about How Accurate the Polls Are a Week from Election Day

With polls showing Republicans in good shape as Election Day approaches, one may wonder: Are these polls accurate? In the battle for control of the U.S. Senate, we can look to the previous midterm election for some empirical data to help answer that question before we actually see the results on election night. It's especially helpful because most of the states with competitive Senate races this year--including New Hampshire, North Carolina, Georgia, Louisiana, Arkansas, Iowa, Kansas, Colorado and Alaska--all had Senate races in 2010 as well. So, how were the Senate candidates in those states doing in the polls one week out from Election Day four years ago?
 
The short answer is: In all of the aforementioned states except Alaska (where incumbent Sen. Lisa Murkowski (R) was campaigning as a write-in candidate against Republican Joe Miller and Democrat Scott McAdams), the Republican candidate for U.S. Senate led his/her Democratic opponent in all or most of the polls heading into the final week of the campaign. Were the polls correct? Well, sort of...
 
In New Hampshire, Republican Kelly Ayotte had a nine-point lead over her Democratic adversary, then-Congressman Paul Hodes, in the RealClearPolitics Average of polls on October 26, 2010 (one week before Election Day 2010). Ayotte continued to expand her lead and ended up routing Hodes, 60% to 37%, in the general election. This year, the Senate race in New Hampshire is much closer: As of today, the incumbent, Sen. Jeanne Shaheen (D), holds a lead of 2.2 percentage points in the RCP Average over her challenger, former Sen. Scott Brown (R-MA). If Brown finishes strong the way Ayotte did in 2010, then he may yet return to the U.S. Senate.

Similarly, in Iowa, the Senate race in 2010 wasn't competitive, unlike the one this year. Four years ago, Sen. Charles Grassley (R) was such a heavy favorite to win re-election that there wasn't much polling of the race, at least not by independent (unaffiliated) outfits. A Des Moines Register poll conducted in late October nailed Grassley's margin of victory over former U.S. Attorney Roxanne Conlin (D): 31 percentage points. This year, the Register's latest poll found state Sen. Joni Ernst (R) ahead of Rep. Bruce Braley (D) by just one percentage point, 47% to 46%. That comports with the current RCP Average of polls, which has Ernst leading Braley by 1.7 percentage points.

Same story in Georgia: The Senate race in 2010 wasn't competitive at all; this year it's a nail-biter. However, there was something unique about the Georgia Senate race four years ago. Sen. Johnny Isakson (R) handily defeated his Democratic challenger, State Labor Commissioner Mike Thurmond, 58% to 39%, but polls overestimated his margin of victory. All the publicly released poll results in late October that year showed Isakson winning by anywhere from 21 to 30 percentage points. This year, the polls are showing a virtual tie between businessman David Perdue (R) and Points of Light CEO Michelle Nunn (D); Perdue's lead over Nunn in the RCP Average today is one-half a percentage point. Because this race will go to a runoff if no candidate wins an outright majority of the vote in the general election, and because all the polls are indicating that is exactly what will happen, further analysis of this race seems premature right now.

Admittedly, the dynamics of an open race are so different from a race with an incumbent in it that trying to use one to predict what will happen in the other may not be empirically sound. So, what races can we look to for an apples-to-apples comparison? Try Arkansas, where Sen. Blanche Lincoln (D) was probably the most vulnerable incumbent in the U.S. Senate in 2010, and she lost in a landslide to then-Congressman John Boozman (R), who won Lincoln's seat with 58% of the vote. (Lincoln received 37%, almost exactly her poll position in the RCP Average--37.8%--on October 26, 2010.) Late deciders must have broken for Boozman, who stood at 54.5% in the RCP average one week out. If history repeats itself this year, then the lone remaining Democrat in Arkansas's congressional delegation, Sen. Mark Pryor (D), will soon be replaced by Rep. Tom Cotton (R), who today leads Prior by five percentage points in the RCP Average, 46.8% to 41.8%.

Colorado was one state where the pollsters really missed the mark in 2010. A week before Election Day that year, Weld County District Attorney Ken Buck (R) led Sen. Michael Bennet (D), who had been appointed to the seat in 2009, by one percentage point in the RCP Average. By Election Day, however, that lead had grown to three percentage points. Buck appeared to have the momentum going into the election, but Bennett managed to pull out one of the tightest victories that year, winning election to a full term by less than 2% of the vote. (Buck received 46.4% of the vote, significantly less than his 49.3% standing in the RCP average on Election Day, while Bennett took 48.08%, slightly more than the 46.3% the RCP average of polls had him winning. What probably happened is that a lot of voters who were going to vote for Buck ended up voting for one of the third-party or "Independent" candidates on the ballot, or maybe they just didn't vote.) This year, Rep. Cory Gardner (R) has a slightly larger lead over incumbent Sen. Mark Udall (D), 3.3 percentage points in the RCP Average, and he hasn't made a lot - or really any - of the stupid mistakes Buck made four years ago. Republicans are also determined this year not to get caught off guard again by the Democrats' ground game. Suppose for a minute, though, that this race plays out from here just like the 2010 contest did. That would mean Gardner would continue to gain in the polls and head into Election Day with a lead of between five and six percentage points in the RCP average. If the actual election results then showed a four-point swing towards Udall, then Gardner would still win. It's also worth noting that Udall is polling much lower than Bennett was four years ago. However, there are also more "undecided" voters than there were at this point in the 2010 race, so while Udall has more ground to make up than Bennett did in the last week of the campaign, there are also more potential late deciders who could swing this race.
 
In North Carolina, the polls were much more accurate in 2010. Incumbent Sen. Richard Burr (R) led his Democratic challenger, then-North Carolina Secretary of State Elaine Marshall, by 10.7 percentage points in the RCP Average on October 26th, and he won re-election by a twelve-point margin, 55% to 43%. (One interesting thing to note is that, in that race, undecided voters appear to have split evenly between the two candidates, something we don't usually see in a race where one candidate has a comfortable lead in the polls and is expected to win.) That may be good news for Sen. Kay Hagan (D), who is currently clinging to a one-percentage-point lead over her Republican challenger, State House Speaker Thom Tillis (R). Unlike Marshall in 2010, however, Tillis appears to be chipping away at the incumbent's lead, which was 3.8 percentage points in the RCP Average at the beginning of this month. But Hagan may be helped by the Libertarian candidate in the race, Sean Haugh, who was not a factor in 2010.

In Louisiana four years ago, Democrats tried to pick off incumbent Sen. and probable prostitute patron David Vitter (R) with Rep. Charlie Melancon (D). In another state where a candidate must receive a majority of the vote to win, Vitter avoided a runoff, winning 57% of the vote to Melancon's 38%. One week before Election Day, Vitter led Melancon by exactly sixteen percentage points in the RCP Average. This year, it's Republicans who are trying to pick off a Democratic incumbent, and let's just say it's going to a runoff; the latest polls indicate that Sen. Mary Landrieu (D) will likely receive a plurality of the vote in the "jungle primary" next week but not even close to the 50% needed to avoid a runoff, which would likely be won by Rep. Bill Cassidy (R). In a one-on-one matchup, Cassidy leads Landrieu by 4.5 percentage points in the RCP Average. 
 
Although there were also Senate races in Kansas and Alaska in 2010, those races were so different from the ones in those states this year that I don't think it's useful to look at them. Also, while there was a Senate race in Kentucky four years ago, I don't really consider the race between incumbent Sen. Mitch McConnell (R) and Kentucky Secretary of State Alison Lundergan Grimes (D) to be "competitive" at this point; McConnell has a small but consistent lead, and Grimes is showing no traction whatsoever.

What about the "generic ballot"? Republicans have a six-point advantage among likely voters in the Washington Post/ABC News poll out this week. That's even better than they were doing four years ago, writes FOX News digital politics editor Chris Stirewalt:
Double, double, toil and trouble - Struggling with female voters and young voters, Democrats fare worse in the latest WaPo/ABC poll than they did at the same point ahead of the disastrous 2010 election and at the same level as the punishing 1994 midterm elections that cost them both houses of Congress. Without any significant increase in Democratic intensity since the previous polling cycle, the auguries are getting dire for the president’s party.
Among registered voters, Democrats lead Republicans in the poll, 47% to 44%, but that's "identical to the difference recorded at this point in 2010." That poll may actually be good news for Democrats when compared with the results of a recent Wall Street Journal/NBC News/Annenberg survey, in which 52% of likely voters in the survey said they wanted the election to produce a Republican-led Congress, while 41% favored Democratic control.

"By historical measures, an 11-point lead on the question of which party should control Congress is large," Janet Hook wrote for the WSJ. "Republicans held a seven-point lead on the question at this point in the 2010 election in a Journal/NBC survey, which used a different method to determine which voters were most likely to cast ballots."

Finally, if you're one of those statistics-mongers who just can't get enough data and analysis, or if you're looking for a number to put on the GOP's odds of taking over the U.S. Senate this cycle, then consider this:

Today, One week out from Election Day 2014, Nate Silver’s infamous FiveThirtyEight gives the GOP a 64.6% chance of winning the Senate, and "the Upshot" (a similar thing affiliated with the New York Times) says there’s a 70% likelihood of the same. The Washington Post’s Election Lab, meanwhile, forecasts that Republicans will see a net gain of seven Senate seats and projects a 93% chance Republicans take the upper chamber. Make of this what you will; I'm going back to running my law practice.

Tuesday, July 23, 2013

How You Know They're Not Serious About "Securing the Border"

Have you seen this ad? We're told that S.744 (the "comprehensive immigration reform" bill that passed the Senate last month) contains "the tough border security America needs." Sen. Marco Rubio (R-Fla.), one of the "Gang of Eight" Senators who supposedly drafted this legislation,  assures us that the bill “puts in place the toughest enforcement measures in the history of the United States, potentially in the world.” Is that a fair statement?

Last week, I posted a critique of S.744 and detailed some of the problems with the bill's approach to immigration reform. I originally endeavored to go into even greater detail about the language of the legislation, but then I decided that the more tedious analysis belonged in a separate post. So, here, I will analyze with greater specificity where and how the Senate's bill comes up short. First, though, a brief passage on the policy lingo of immigration reform is needed.

The principal responsibility for protecting our country's border security, cybersecurity and economic security lies with the Department of Homeland Security (DHS). DHS is also charged with overseeing citizenship and immigration in the United States. The United States Citizenship & Immigration Services (USCIS) oversees legal immigration to the United States and is the agency that grants immigration and citizenship benefits. Immigration & Customs Enforcement (ICE) is "the principal investigative arm of DHS, and its primary mission is to promote homeland security and public safety through the criminal and civil enforcement of federal laws governing border control, customs, trade, and immigration," according to this thing somebody referred me to.
 
When discussing/debating "legalizing" illegal immigrants, it's important to understand that a pathway to legal status is not necessarily a pathway to citizenship. (To some, any legalization is "amnesty", but that's a topic for a separate post.) S.744 speaks of "registered provisional immigrant status," which if granted would then allow an immigrant to stay in the U.S. legally, without receiving all the rights and benefits of a U.S. citizen. In this post, I'll refer to applications for registered provisional immigrant status by the acronym "ARPISs".

It's also important to understand that, as was the case with past efforts toward "comprehensive immigration reform", the end game of S.744 is to legalize immigrants who are currently in the U.S. unlawfully without requiring them to resort to existing legal channels (which, in most cases, would require them to first leave the U.S. and re-enter the country legally). There are some who are categorically opposed to this, but most Americans who have an opinion on this subject are not. Most members of Congress, too, are willing to support legislation that would provide illegal aliens with a path to legal status or even citizenship, but a lot of them want any legislation that provides such a pathway to address the problem of illegal immigration with, inter alia, more border security, enhanced & increased "interior enforcement" and measures that discourage the wrong kind of immigration (to wit, illegal immigration and the immigrants who come here legally with an eye toward living off the government).

Rubio and other Republican proponents of the bill have insisted that it contains multiple security "triggers" that must be met before any immigrant currently in this country illegally can be legalized. Skeptics have contended that these "triggers" are weak, meaningless and/or can be easily manipulated/circumvented to fast-track the legalization process. Many Senators offered amendments to S.744 (discussed herein) that would strengthen the triggers or add additional preconditions to legalization.

You've probably heard of the "border surge" provisions in S.744. In the bill, what some are calling "the Border Surge" is part of the Comprehensive Southern Border Security Strategy. I'll just call it the Border Surge. There's also the "Southern Border Fencing Strategy," which is exactly what it sounds like. (S.744 requires the `Southern Border Fencing Strategy' to identify "where fencing (including double-layer fencing), infrastructure, and technology, including at ports of entry, should be deployed along the Southern border.") "E-verify" refers to a mandatory employment verification system required by current law  that the federal government never got around to implementing (at least not as originally conceived). There is currently a federal employment verification program in place, but it is not very potent or effective.

Now to the "triggers." I haven't read and analyzed the entire bill yet, so I can't tell you exactly what's in it. However, I can say what's not in it, and that's critical. By looking at what the proponents of this legislation voted against, you can see that they're not at all serious about border security.
 
Six weeks ago, the Senate agreed to table (kill) an amendment by Sen. Chuck Grassley (R-IA) that would have allowed DHS to begin processing ARPISs only after the Secretary of Homeland Security has certified to Congress that "the Secretary has maintained effective control of the Southern border for a period of not less 6 months." Such a nebulous standard could hardly be regarded as a serious precondition, so it should come as no surprise that Handsome John Thune's amendment, which contained more specific prerequisites, was also defeated by a vote of 39 to 54. Thune's amendment would have required 350 miles of Southern border fencing to be completed before the Secretary could commence processing ARPISs and conditioned any adjustment in the status of aliens who have been granted registered provisional status on the Secretary's written certification that:

  • the Comprehensive Southern Border Security Strategy “has been substantially deployed and is substantially operational;” 

  • the Southern Border Fencing Strategy has been submitted to Congress, implemented, and is "substantially completed;"
  • 700 miles of Southern border fencing “that is double-layered and constructed in a way to effectively restrain pedestrian traffic” has been completed;
  •  the Secretary has implemented E-verify; and
  •  
  • the Secretary is using an electronic exit system at air and sea ports of entry that operates by collecting machine-readable visa or passport information from air and vessel carriers.
Note that both Grassley's and Thune's amendments suffer from the same flaw: The predicate for legalizing illegals is not actually securing the borders but rather the Secretary of Homeland Security's certification that the southern border is secured. (Grassley's amendment also contained a special carve-out for aliens granted blue card status, which is a special legal status for agricultural workers; Grassley’s home state of Iowa is about 95% farmland.) We already know that members of Obama's cabinet have no compunction about lying to Congress, so what good is the DHS Secretary's word on anything?

Later that same day,  Sen. Rand Paul, R-Ky., offered a good fix: require Congress to vote annually for five years on whether the border is secure. "If Congress believes that the border is not secure," the Senator explained in a speech on the Senate floor, "then the processing of undocumented immigrants stops until it is secure." As David Nakamura of the Washington Post reported it:

Paul’s amendment would require the Department of Homeland Security to implement specific border security measures, including hundreds of miles of additional fencing along the U.S.-Mexico border, and provide a report to Congress each year on its progress.
Then Congress would vote annually as to whether the agency had met its goals. Each year, another group of illegal immigrants would earn legal work visas if the metrics are met, Paul said.
This short summation--while accurate and concise--does not do Paul's amendment (styled the "Trust But Verify Act of 2013") justice. Indeed, those who crafted this particular legislation appear to have thought of everything. The bill specified what the Secretary must conduct an annual comprehensive review of and provided specific border security metrics, the progress toward which must be reported on. It stated what the joint resolution affirming that the border is secure must say and prevents the resolution from being amended. It even contained a provision limiting debate on the joint resolution "and on all debatable motions and appeals in connection with such resolution" and curtailing the ability of would-be obstructionists to use parliamentary shenanigans to delay a vote on the resolution or dispose of it without a vote. 

Paul's amendment also addressed the problem of DHS officials using their administrative authority to not enforce the law (as discussed in my earlier post) by prohibiting the Secretary from making "any alteration to the Border Patrol sectors in operation or the boundaries of such sectors" without first notifying both the House and Senate Homeland Security Committees of the proposed change "not later than 120 days before any such change would take effect," by which time Congress could act to thwart any undesired changes. It also required the Secretary to establish a Student Visa National Security Registration System and submit an annual report to Congress that describes the effectiveness with which DHS is screening student visa applicants through the System and "indicates whether the System has been implemented in a manner that is overbroad or results in the deportation of individuals with no reasonable link to a national security threat or perceived threat." And, the amendment capped the number of  applicants who may be granted registered provisional immigrant status under the law in any calendar year at 2,000,000 (a ridiculously high limit, but apparently too low for some people). 61 Senators voted to table Paul's "Trust But Verify" amendment.

Maybe Senator Paul was just asking for too much. (I don't think so, but reasonable minds can differ.)  Sen. David Vitter (R-LA) had a simpler request: Hey, remember that integrated entry & exit data system (a system to track the border comings and goings of foreigners) that was supposed to be developed and implemented under a 1996 law? You know, the one we're still waiting on? Well, forget all the stuff that Paul wanted. Let's just condition the temporary grant of legal status to, or adjustment to citizenship status of, any individual who is unlawfully present in the United States on the Secretary's written certification that that biometric border check-in/check-out system (officially the US-VISIT System) has been fully implemented at every land, sea and airport of entry. Oh, and Congress has to pass a joint resolution stating that this integrated entry and exit data system has been sufficiently implemented, because, you know, we don't trust this administration's word. Senator Vitter proposed an amendment to that effect over a month ago. It even included "fast track" procedures for getting the requisite joint resolution through Congress without unnecessary delay.

Now, it seems that a piece of legislation that basically just says, "Hey, let's incentivize the executive branch to do what they're already required to do by conditioning something that they want but that nobody really needs on them doing that thing they're supposed to do." shouldn't be something that a lot of Senators would find a reason to vote against, but if you view their votes on Senator Vitter's amendment, which was rejected by a vote of 36 to 58, in the context of most of them not giving a damn about securing our borders, then it makes sense. Another amendment, proposed by Sen. Mike Lee (R-Utah), that would have required “fast-track congressional approval” of what the Gang of 8 legislation merely requires the Secretary of Homeland Security to certify was also voted down, 39 to 59.

Enter Sen. John Cornyn (R-Texas), tall, learned and circumspect, a conservative Republican from a state with a large population of illegal immigrants. As the Senate Minority Whip, it's his job to make sure GOP Senators vote the party line on critical pieces of legislation. Cornyn, who was Texas Attorney General before being elected to the U.S. Senate, had criticized the "border-security triggers" in S.744 as "talking points disguised as policy." Could he offer a serious bill for predicating any legalization of illegal immigrants on actual, verified border security measures? Well, a little over a month ago, after the amendments proposed by Senators Grassley, Thune, Paul, Vitter and Lee had all been voted down, Senator Cornyn offered an amendment to the immigration bill that would have kept newly legalized immigrants from becoming permanent residents or pursuing citizenship until certain border goals were met. Those goals were:
  • to achieve and maintain operational control of the Southern border;
  • to achieve and maintain full situational awareness of the Southern border;
  • to fully implement a biometric entry and exit system at all land, air and sea ports of entry; and
  • to implement E-verify.
All these goals would have to be met "within 5 years of the date of the enactment of this Act," i.e., the comprehensive immigration reform bill. As used in Cornyn's amendment, the term "operational control" meant that, "within each and every sector of the Southern border, a condition exists in which there is an effectiveness rate, informed by situational awareness, of not lower than 90 percent." The term "situational awareness" was defined as "knowledge and an understanding of current illicit cross-border activity, including cross-border threats and trends concerning illicit trafficking and unlawful crossings along the international borders of the United States and in the maritime environment, and the ability to predict future shifts in such threats and trends." The Secretary and the U.S. Customs & Border Protection Commissioner would have to "jointly submit" to the President and Congress a written certification, under penalty of perjury, that the Secretary had met these goals.  This submission could not be made sooner than 9½ years after the comprehensive immigration reform bill becomes law and must include "a comprehensive report detailing the data, methodologies, and reasoning" justifying the certification. And, the Comptroller General of the United States would be required to "review such certification and provide Congress with a written report reviewing the reliability of such certification" and expressing the Comptroller General's own conclusion as to whether or not the specified border goals have been achieved.

The Secretary would still be required to submit a strategy "for achieving and maintaining operational control and full situational awareness of the Southern border" to the Comptroller General, and within 60 days of the submission of such strategy, the Secretary would also have to submit "an implementation plan for each of the border security components of the Department to carry out the Strategy." This plan must include, at a minimum:
  • a comprehensive border security technology plan for continuous and systematic surveillance of the Southern border, including a documented justification and rationale for the technologies selected, deployment locations, fixed versus mobile assets, and a timetable for procurement and deployment;
  • the resources, including personnel, infrastructure and technologies that must be developed, procured and successfully deployed, to achieve and maintain operational control and full situational awareness of the Southern border; and
  • a set of interim goals and supporting milestones necessary for the Department to achieve and maintain operational control and full situational awareness of the Southern border.
It seemed doomed to fail, but Cornyn's amendment had something the others didn't: an appropriate acronym. The Senator dubbed his amendment, "Requiring Enforcement, Security and safety while Upgrading Lawful Trade and travel Simultaneously (RESULTS)," and it fared better with his colleagues than the other amendments I've described here did. Only 54 Senators voted to kill it. Two Democrats, Mark Pryor of Arkansas and Joe Manchin of West Virginia, voted against the Motion to Table, as did Senator Rubio, who had opposed the aforementioned amendments offered by  Senators Grassley, Thune, Paul, Vitter and Lee. (Curiously, Rand Paul voted with the anti-enforcement coalition, but that may have been because of the amendment's hefty cost.) Heritage Action ("The Heritage Foundation’s lobbying arm") urged Senators to vote “NO” on Cornyn's RESULTS amendment "because it fails to solve the enforcement problems in the underlying bill" and would "serve as political cover for [multiple Senators] to justify their support for the Gang of Eight’s amnesty."

So, on June 27th, S.744 passed the Senate, free of any amendments that would require DHS to actually secure the borders before illegal aliens could be legalized. It did include language from a border security and enforcement amendment proposed by Sen. Patrick Leahy (D-VT) and modified by the much-touted Corker-Hoeven Amendment, which inludes a lot of stuff that should turn off congressmen and women  on both sides of the aisle. (Leahy, who "begrudgingly" supported the changes to his amendment offered by Senators Corker and Hoeven, complained that their legislation “reads like a Christmas wish list for Halliburton.”) While Corker-Hoeven does delay the legalization of illegals (No Registered Provisional Immigrants can receive Green Cards until at least ten years after the bill becomes law) and strengthens the border-security "triggers" to legalization, it fails to cure many of the bill's other deficiencies and focuses on border security but not interior enforcement. Then there's the price tag; the CBO reported it would add $38 billion to the cost of the act. At the very least, the amendment added language to S.744 aimed at preventing the abuse of federal benefits by illegals who would be legalized under the legislation, including:

·         preventing immigrants who used a fraudulent or false Social Security number while they were unlawfully present in the U.S. from getting Social Security credits for that period;

·         restricting certain non-immigrant visa holders, such as tourists and foreign students, from accessing Medicaid, SCHIP and Obamacare benefits; and

·         providing that the Department of Health & Human Services may not grant waivers to states to allow them to use Temporary Assistance for Needy Families (TANF) dollars to give cash assistance benefits to registered provisional immigrants.

That ad I mentioned at the beginning of this post calls the Border Surge "the toughest border security plan ever passed by Congress." That it may be, but again, border security is only part of what we need. S.744 falls short of ensuring the aggressive interior enforcement that is desperately needed and eliminating the government-created magnets for illegal immigrants. Also, it fails to predicate legalization on a congressional affirmation of border security or other objectively verifiable metrics. If any of that bothers you, then please contact your Representative and admonish him or her not to vote for any bill that contains the same flaws as the Senate bill. But first, follow us on Twitter.

Tuesday, December 4, 2012

A Disturbing New Twist on the Bradley Effect

The defeat of Rep. Allen West (R., Fla.) means that, come January, the United States Congress will have just one black Republican member, Rep. Tim Scott of South Carolina. West's loss came as a surprise to a number of people, especially considering that every public poll of his race had shown the freshman congressman leading his Democratic opponent, Patrick Murphy. Yet, on Tuesday morning, it was West who conceded to Murphy after a two-week recount confirmed that the incumbent had fallen more than 2,000 votes short of winning a second term.

The race in West’s congressional district wasn't the only one where the outcome defied multiple polls: In Utah's newly-created 4th District, Rep. Jim Matheson, the lone Democrat in the Beehive State's congressional delegation, eked out a victory over Saratoga Springs Mayor Mia Love (R). Less than a week before Election Day, a Mason-Dixon poll showed the six-term incumbent trailing Love by an eye-popping twelve percentage points. About a month earlier, the Deseret News/KSL poll pegged Love's lead at six points, 49% to 43%, but that was a huge shift from their previous poll of the race, conducted in June, which had Matheson up by a healthy fifteen-point margin, 53% to 38%. Despite this apparent momentum toward Love and a huge voter turnout in a state that went for Mitt Romney by a wider margin than any other, Matheson managed to win with 49% of the vote to 48% for his Republican challenger.

The surprising losses of these two prominent black candidates--and the poll numbers that turned out to be so wrong--bring to mind the so-called "Bradley Effect," a phenomenon in which polls of races with one African-American candidate tend to overestimate his/her support or underestimate the support for his/her opponent, or both. The term arose out of another surprising election result, in the 1982 race for governor of California. Los Angeles Mayor Tom Bradley (D) was poised to become the Golden State's first African-American governor. He went into Election Day with a healthy lead over his Republican opponent, then-Attorney General George Deukmejian, in all the polls. Even the exit poll results indicated a big Democratic win. Much to the Bradley campaign's chagrin, however, Deukmejian triumphed, winning the election by less than 100,000 votes out of nearly 8 million cast.

The following year, in the race for mayor of Chicago, Harold Washington, an African-American congressman from Illinois's 1st congressional district, defeated his caucasian opponent, former State Rep. Bernard Epton (R), by about 47,500 votes (less than four percentage points). Multiple polls conducted in the run-up to Election Day indicated it would be a rout for Washington. Three separate polls taken in the final weeks of the campaign showed Washington with a fourteen-point lead.

The obvious discrepancies between the pre-election polling and the actual results of these two high-profile races seemed to cry out for an explanation. Arguably, racial tensions between blacks and whites were stronger, or at least more overt, 30 years ago than they are today, so naturally race was floated as a possible factor in Bradley's loss and Washington's unexpectedly narrow win. (Veteran California pollster Mervin Field was quoted two days after the 1982 election as saying that "race was a factor in the Bradley loss.") While it was clear that these elections had turned out much differently than the polls had foretold, the notion that races between a black candidate and a white candidate presented a special problem for pollsters did not gain widespread acceptance until after a couple of elections with history-making black candidates at the end of the decade.

In 1989, a U.S. Attorney named Rudy Giuliani made his first run for mayor of New York City. Just before the election, two different polls showed him trailing his African-American opponent, Manhattan Borough President David Dinkins, by double digits: Dinkins led Giuliani by fourteen points in the ABC/New York Daily News tracking survey of the race, fifteen points according to Gallup, and by eighteen points in a poll conducted by the New York Observer. Even the exit polls overestimated Dinkins's eventual two-point margin of victory.

The more infamous race that year took place in Virginia. I'll let Scott Keeter and Nilanthi Samaranayake of the Pew Research Center explain:
Virginia Democrat and African American Douglas Wilder edged white Republican Marshall Coleman by less than one percentage point to become the nation's first elected black governor. But two of three polls conducted just days before the election showed Wilder leading by double-digits; a third poll had him 4 points ahead.
The exit poll results from the Virginia governor's race were particularly jarring; according to Pew, "an exit poll conducted on Election Day showed Wilder winning by 10 points, while accurately tallying the vote in the other two statewide races." This indicated that some voters must not have been honest with poll-takers. Keeter and Samaranayake noted that, unlike other exit polls that used an anonymous written ballot to collect voters' responses, "this one had interviewers asking voters face-to-face how they voted, a situation that might increase the pressure to provide a socially desirable response."

It didn't take long for major media outfits to trumpet explanations for why the polls in these two particular elections were so far off. Two days after Election Day, the New York Times ran an article about the "broad disparities" between the polls and election results in New York and Virginia: 
In Virginia, Mason-Dixon Opinion Research Inc. acknowledged making a fundamental error in the way it conducted its Election Day polling. Its workers stopped voters outside polling places and asked them face to face how they had voted, rather than following the more widely accepted practice of having them fill out ''secret ballots'' and drop them in a box. All the New York polls used this method.

Brad Coker, Mason-Dixon's president, said that in hindsight he thought some voters might have been reluctant to admit to a poll taker that they had not voted for the black candidate, L. Douglas Wilder. The organization's Election Day polls gave a 10-point lead to Mr. Wilder, the Democrat, and at least three television stations reported the figures right after the polls closed and declared Mr. Wilder the winner. Today it appears that Mr. Wilder did defeat J. Marshall Coleman, the Republican, but by only one quarter of a percentage point.

''I'm a big supporter of pre-election polls and exit polls,'' said Peter D. Hart, a leading Democratic poll taker here, ''but only for figuring out the dynamics of an election and not who is going to win. What bothers me about yesterday is that the exit polls are obscuring the news, setting up incorrect expectations and leaving everyone talking about 'Oops, what happened?' rather than the historic story of the elections of Wilder and Dinkins.''
Virginia political scientist Larry Sabato was even more blatant in his analysis. ''Whites tell pollsters ahead of time that they are voting for the black candidate,'' he said, ''and then they go into the voting booth and can't quite pull down that lever.'' And, a Richmond Times-Dispatch article that ran the same day quoted Coker as saying, "Virginians were put on the spot: If we don't vote for Wilder, [then] we're going to be perceived of as racist." Whatever the reason was, the gap between poll results and election results in races between a black candidate and a white candidate was becoming a recurring problem.

For Wilder, it wasn't the first time his poll numbers had foretold a different election result. In 1985, he was elected lieutenant governor by three precentage points, but pre-election polls had pegged his lead at anywhere from nine to 22 points. The same polls were much more accurate with regard to the governor's race that year. In part because Wilder's experiences provided stronger evidence (and more of it) that voters lied to pollsters about voting for the black candidate than did Tom Bradley's 1982 campaign, some individuals prefer the term "Wilder Effect" to "Bradley Effect". 

Whatever you call it, the concept that the effect was attributable to white voters who were reluctant to tell poll takers they were not voting for a black candidate gained steam following the '89 elections. In 1993, ahead of Mayor Dinkins's rematch wth Giuliani, the New York Times reported on a study by Larry Hugick, the director of political and media surveys at Princeton Survey Research Associates, who concluded that, in general elections that pitted a black Democrat against a white Republican, polls consistently overestimated the black candidate's strength. Said the Times:
[Mr. Hugick] concluded that white Democrats and Democratic-leaning independent voters were more likely than blacks to be undecided, torn between their allegiance to political party and to race. These ambivalent voters told interviewers conducting polls that they were undecided, but overwhelmingly supported the white candidate in the voting booth.
Mr. Hugick speculated that some of these voters were genuinely undecided, while others were reluctant to admit they were voting against a black Democrat, perhaps worried they would be perceived as racially biased.

"To some extent, these voters are torn between party and doubts about the ability of black candidates to do a good job," he said. "It's racial stereotyping. It's a way race affects us in this country. We've understood racial intolerance is highly unacceptable, but that doesn't mean that people have internalized tolerance."
After Giuliani defeated Dinkins in another close race, Hugick sent out a memo to his clients and friends, detailing his findings and asserting that the 1993 New York mayoral election "again demonstrated that late polls in biracial general elections overstate the black candidate's vote-getting ability." Following that election, however, the Wilder/Bradley effect appeared to be waning. African-American candidates who waged multiple campaigns provided a good "control" group with which to examine whether the Bradley/Wilder effect still occurred. Polls showed Mayor Dinkins's 1993 rematch with Giuliani, for example, to be a real dead heat. Giuliani, of course, won the race by a margin comparable to Dinkins's narrow 1989 victory.

In 1990, former Charlotte Mayor Harvey Gantt, an African-American Democrat, challenged longtime Sen. Jesse Helms (R-NC). Two of three independent polls conducted just before the election showed Gantt leading Helms, but none showed the controversial  senator with a lead of six percentage points, his eventual margin of victory. Six years later, when Helms again fended off a challenge from Gantt, the polls were much more accurate; the incumbent's 53%-46% win was consistent with polling that gave him a modest lead.

Similarly, when Carol Moseley Braun became the first (and, to date, only) African-American woman elected to the U.S. Senate in 1992, the election results were much closer than polls had suggested. Braun defeated Republican Richard Williamson by a ten-point margin, but polls taken just days before the election showed her with a lead ranging from 17 to 20 points. (Notably, all of these polls pegged Braun's level of support at less than the 53% of the vote she ultimately received, so it's certainly possible that everyone who claimed to be voting for Braun did actually vote for her, but if these polls were accurate, then it would mean that Williamson won nearly all of the late deciders.) Six years later, Braun lost her bid for re-election by less than 100,000 votes, or three percentage points, to Peter Fitzgerald, a wealthy state senator from the Chicago suburbs. It could have been a lot worse; pre-election polls gave Fitzgerald a lead of three to ten points. A strong turnout among Illinois's black voters may have boosted Braun.
 

The cases of Gantt and Braun suggest that, to the extent that polls significantly overstate a black candidate's support and/or undertell his/her white opponent's strength in the black candidate's first run for statewide office, this phenomenon does not repeat itself when the same black politician runs again for the same office. Tom Bradley himself got creamed by Deukmejian in their 1986 rematch, but at least the mayor wasn't misled by poll numbers that showed him to be ahead. Prior to the election, all public polls had Bradley trailing the incumbent by double digits. Though Deukmejian won re-election by a larger margin than the polls indicated he would, none greatly overtold Bradley's level of support.
 
The paucity of competitive general-election match-ups between a white candidate and a black candidate in 2000, 2002 and 2004 made it difficult to gauge whether the Bradley effect was still waning. There was the race for governor of  New York in 2002, in which incumbent Gov. George E. Pataki (R) crushed his opponents to win a third term. Pataki won 48%, finishing well ahead of the state's African-American Comptroller, H. Carl McCall (D), who took 32%, and a slew of other candidates who weren't black. Pataki had led McCall by eleven points in a survey taken in late September. 

In 2004, two members of the U.S. House of Representatives from the Atlanta area squared off against each other for the Senate seat being vacated by Georgia Democrat Zell Miller. Rep. Johnny Isakson, a white Republican, defeated Rep. Denise Majette, an African-American Democrat, winning 58% of the vote to Majette's 40%. The weekend before the election, the Atlanta Journal-Constitution published the results of a poll that had Isakson leading Majette by thirteen points, 49% to 36%. 
  
Then came 2008. There wasn't a single black candidate, Republican or Democrat, in a highly competitive gubernatorial or senatorial contest that year, but the presidential candidacy of Barack Obama revived talk about the Bradley/Wilder effect. Other than Obama's surprise loss to then-Sen. Hillary Clinton in the New Hampshire primary, however, there were no conspicuous gaps between Obama's poll numbers and share of the vote he actually received in the 2008 Democratic primaries. In the general election, Obama experienced a fate similar to that of Washington, Wilder and Dinkins: he won, but some polls overtold hs eventual margin of victory. A lot of polls underestimated voter support for the Republican nominee, Sen. John McCain of Arizona, both nationally and at the state level, but very few overestimated Obama's support. This could simply have meant that late deciders broke for McCain. Whether that's true or not, there was not a single battleground state in which all the polls were way off. Obama did not experience the Wilder effect in November.

Even before the 2008 election, there was a growing consensus among analysts who examined the data that the Wilder effect was a thing of the past. Daniel J. Hopkins, a post-doctoral fellow at Harvard, conducted the most extensive study and analysis I've seen of the phenomenon and concluded that "there was indeed a Wilder effect" in elections during the 1980s and early 1990s but that "this effect subsequently disappeared."



To what, then, should we attribute the salient discrepancy between the poll and election results in Utah and Florida this year? Regardless of the cause(s), this much is clear: The Wilder Effect still exists, but with a twist; the effect seems to only manifest itself in races between a black Republican and a white Democrat.

I first noticed this correlation between the presence/absence of a Bradley/Wilder effect and the political affiliations of the candidates in 2006. That year saw more competitive races for statewide office between a black candidate and a white candidate than any other election cycle I can remember. The U.S. Senate races in Tennessee and Maryland provided data for a good comparative analysis. Both featured a charismatic, energetic black politician vying against an older, duller white politician for an open U.S. Senate seat. (In Mississippi, State Rep. Erik R. Fleming, a black Democrat, challenged Sen. Trent Lott, a white Republican, for his Senate seat, but that race was not considered competitive, and Lott won re-election by a decisive margin.) 

In Tennessee, Democrats nominated Harold Ford, Jr., a five-term congressman from Memphis, to run for the seat of then-Senate Majority Leader Bill Frist (R), who was retiring. Ford and his Republican opponent, former Chattanooga Mayor Bob Corker, traded the lead in the polls back and forth until October, when Corker appeared to be pulling away. Heading into Election Day, independent polls put Corker's lead at anywhere from three to twelve points. The only public polls that showed Ford leading were conducted by partisan Democratic pollsters. Corker did end up winning, but only by about 50,000 votes (less than three percentage points). There had been no Wilder effect in the Tennessee Senate race.

In Maryland, it was a different story. Perhaps no U.S. Senate candidate in 2006 garnered as much enthusiasm as Michael Steele. Four years after becoming Maryland's first African-American lieutenant governor, Steele appeared to be on the verge of beating the odds to win a U.S. Senate seat in a state where 55% of voters were registered Democrats. He had fought the Democratic candidate, ten-term Rep. Ben Cardin, to a draw in the polls. There was even talk of a "Reverse Wilder Effect" in the race.

“With Steele sporting a 7-point lead among white voters,” pondered RealClearPolitics’s John McIntyre just days before the election, “what happens to Cardin's lead if Steele's 12% of the black vote goes to 20% or 25%?” We found out: Steele managed to take 25% of the black vote, according to exit polls, but lost to Cardin by a 54%-44% margin.  

Having followed the 2004 elections as closely as I did, I knew how pollsters could sometimes get it wrong (This was before I looked into the history of the "Bradley Effect" and discovered just how wrong.), but the results of the Maryland Senate race had left a particularly bad taste in my mouth because (1) I had gotten my hopes up, based on what appeared to be momentum towards Steele in the final weeks of the campaign, and (2) I couldn't find another comparably high-profile race that cycle in which so many polls had been so far off.

I should mention that, in the three 2006 gubernatorial contests between a black candidate and a white candidate, there was no such disparity between the polls and the outcome. In Ohio, Republican Ken Blackwell, an African-American who had served as mayor of Cincinnati, state Treasurer and Ohio Secretary of State, lost in a rout to Ted Strickland, a white Democratic congressman from Lucasville. Strickland's 900,000-vote margin of victory accorded with polls that put him ahead of Blackwell by about 20 points. Next door, in Pennsylvania, Gov. Ed Rendell (D) held off a challenge from businessman and former Pittsburgh Steelers Wide Receiver Lynn Swann (R) to win a second term. Swann actually outperformed his standing in most polls, winning 40% of the vote to Rendell's 60%.

In Massachusetts, Deval Patrick (D) was elected the Bay State's first black governor by a 56%-35% margin. Some polls had shown him leading his opponent, Lt. Gov. Kerry Healey (R) by more, but every poll of the race had understated Healey's support. Thus, much like the 2008 presidential election, it appears that most undecided voters in the Massachusetts and Pennsylvania gubernatorial races ended up supporting the Republican. Race made no difference.

In light of this data, I was inclined to write off Steele's discouraging loss as a fluke. Then, in 2010, I was again bemused by the results of an election that did not comport with the polling. This time, it was in Colorado. Rep. Ed Perlmutter (D), an aging, white pol from Golden, faced a tough challenge from Aurora City Councilman Ryan Frazier (R), an ebullient young African-American. Things seemed to be going Frazier's way: the left leaning Denver Post endorsed him, and a desperate-looking Perlmutter actually smacked him at a debate in October. Even more encouraging, the only public poll of the race showed Frazier leading Perlmutter, 40%-39%. While a one-point lead is no cause for ecstasy, an incumbent polling at 39% is generally not considered to be in good shape. So you can imagine how disappointed I was to learn that Perlmutter won re-election. It wasn't even close; the incumbent won by nearly 25,000 votes, taking 53% of the vote to Frazier's 42%. Ouch.

Now, the losses of Love and West, belied by multiple polls, add to this pattern of black Republicans who lost elections despite poll results that showed them tied or leading their white Democratic opponents. What could explain this 21st-century version of the Wilder Effect? Are large numbers of white voters reluctant to unequivocally declare their support for a black candidate's opponent? If so, then why did Ford, Blackwell, Swann, Patrick and Obama not experience the Wilder effect? It's certainly possible that the polls in Utah and Florida, like many other polls this cycle, underestimated Democratic turnout and/or overestimated Republican turnout, but then what accounts for the unexpectedly lopsided defeats of Frazier and Steele? I will not endeavor to answer these questions in this post; my only point is that the Bradley/Wilder effect still occurs but (for whatever reason) only in races between a white Democrat and a black Republican. (Take note, Herman Cain.)

Tuesday, October 23, 2012

The Most (and Least) Accurate Polls

As we head into the final stretch of the campaign, we're likely to be inundated by a spate of poll results from all kinds of different organizations, and if, well, everything is any indication, then different pollsters are likely to show very different things about the state of the race in key swing states. There may be no point in trying to make sense of mind-bogglingly contradictory survey results, but in case anyone is interested, I took a look at the polling done in the run-up to the 2010 midterms and constructed a little table on which pollsters came the closest to foretelling the election results in states that are now considered "battleground" territory and which missed the mark.

I measured polling accuracy by looking at the results of the last public poll of each race by each pollster and comparing the margin between the two highest-polling candidates (sometimes the only two candidates) to the eventual winner’s margin of victory over the second-place finisher. If there was more than one poll tied for most or least accurate by this metric, then I compared each candidate’s individual poll position in the survey results with their eventual share of the final vote. This is why PPP is listed as having the most accurate poll of the Florida governor’s race despite the fact that their final pre election poll had Alex Sink (D) leading Rick Scott (R), the eventual victor, 48% to 47%. This was rated "most accurate" notwithstanding Scott’s victory because it underestimated Scott’s eventual share of the vote (48.9%) by only 1.9 percentage points, while overestimating Sink’s vote share (47.7%) by only 0.3 points. The next-most accurate survey of this race was conducted by Rasmussen Reports, which had Scott leading by three points heading into Election Day.

Race
Most Accurate Poll
Least Accurate Poll
Ohio Gov.
Pennsylvania Gov.
Florida Gov.
Colorado Gov.
Colorado Sen.
Florida Sen.
Iowa Sen.
Nevada Sen.
New Hampshire Sen.
North Carolina Sen.
Ohio Sen.
Pennsylvania Sen.
Wisconsin Sen.
Wisconsin Gov.
Nevada Gov.
Iowa Gov.
New Mexico Gov.
Michigan Gov.
New Hampshire Gov.
*Nailed it! (Poll results conformed exactly to election results.)

Of course, not all these races were as close as the presidential race is this year. The Senate races in Florida, Iowa and New Hampshire, and the gubernatorial elections in Colorado and Michigan, were all blowouts. Take that however you like. Also notice that the above table does not include any data from Virginia, which had neither a U.S. Senate race nor a gubernatorial election in 2010. In the 2009 race for governor there, SurveyUSA was right on the money; its final poll before Election Day showed Bob McDonnell (R) leading his Democratic opponent, Creigh Deeds, by eighteen points 58% to 40%. McDonnell won that election with 59% of the vote to 41% for Deeds. That was three years ago, however, so it may not be worth much.

Finally, there are a handful of states that are assuredly not up for grabs in the presidential election but are experiencing heated U.S. Senate and/or gubernatorial races. For those who are interested, here are my findings on polls of 2010 races in those states.

Race
Most Accurate Poll
Least Accurate Poll
Connecticut Governor
U.S. Senate – Connecticut
Maine­ Governor
Massachusetts Governor
U.S. Senate – Washington