Limitations of the Marquette Surplus Poll

80% Support

Marquette's poll from Monday asking Wisconsin adults whether the Surplus Deal should have been passed has been making the rounds.

The stunning topline figure showing 80% support among adults has been used to signal to opponents of the Deal that they made the wrong decision. As many have said online, "If you're voting against something that 80% of people want, you might be on the wrong side of things."

But the poll doesn't so much measure 'support' as it does knowledge. The proposed Deal was complex, so just because it's popular doesn't mean its supporters understand it, or even that implementing it would make for good policy.

The reason the poll is really a measure of knowledge is because most people look to politicians and media outlets to inform themselves on issues. Relatively few have the time to find and read the relevant bills, or to listen to the testimony given for and against an issue. As a consequence, people's opinions tend to be shaped by politicians and media personalities. If a person's representative or trusted media source says that a bill does X, many people believe them. After all, there's a reason we pay people to serve as politicians and work as journalists––we don't have the time to become experts ourselves, so we outsource expertise to others.

This is neither good nor bad; it's just a fact.

But whether the results of this arrangement are good depends on the quality of the assessments made by these opinion shapers. And whether the results of a poll, such as Marquette's, ought to be considered as representative of a population or used as a guide for creating policy, depend on its method, and quality of the questions.

The Sample & Questions

If you want to read the full list of questions you can either scroll through the poll results here, or look at the Instrument. For even more detail on the results, a demographic breakdown can be viewed by looking at the Crosstabs.

The first thing to discuss is the sample size. The poll was sampled from 454 Wisconsin Adults who appear in the SSRS Opinion Panel database. 423 of these were registered voters.

Now, the proportion of survey respondents who are registered voters is very good. But I have to take issue with the sample size. Sorry, statisticians, I understand the sample is 'reasonably' representative, but we also know that surveys always entail selection and response biases. How can we say that the 80% support figure is representative when other platforms––like X and Facebook––seem to indicate support is more like 50%? Has anyone reading this ever met a pollster or participated in a poll? Does anyone reading this know anyone who has? There's a reason Disraeli and Twain said "Lies, Damned Lies, and Statistics."

In any case, the meat of this poll is in the questions. Here are the two most important ones...

Question: On May 11, Democratic Gov. Tony Evers and Republican legislative leaders announced an agreement on a bill that would have used $1.8 billion of the projected $2.5 billion state budget surplus to increase funding for special education, reduce school property taxes by about 5%, and send rebate checks of $300 to individuals, $600 to married couples, who paid state income taxes. It would also have eliminated taxes on tips and overtime. Do you think the state legislature (should have passed the bill), or (should it have defeated the bill)?

This question is what gave rise to the 80% support figure. Think about how respondents will answer this given what the question presents as information. "$1.8 billion," "$2.5 billion," "reduce...5%," "checks of $300...$600," "eliminate taxes on tips and overtime."

The first two figures make it sound like the state will have $700 million left in the bank. Reducing school property taxes by 5% and rebate checks of $300-600 all sound like unambiguous wins. But words like "projected" are likely to be overlooked. Because of this, Marquette asked their second question, but it also fails to give respondents sufficient context...

Question: Some have argued that the bill was fiscally irresponsible for spending a projected surplus now that might lead to a deficit in future budgets. Would it be better to delay providing special education funding, property tax reductions, and rebate checks until next year, or would it be better to provide them now, even if it might affect the budget next year?

Again, some have "argued," "might lead," "future budgets," "delay funding, tax reductions, rebate checks..."

As far as the respondent is concerned, only a small subset of people are concerned with these supposed fiscal problems. Problems that might occur. Problems that if they did occur, would only occur in the future. The question concludes with a choice between getting funding, tax reductions and rebate checks today with a slight risk in the future, or getting those things tomorrow.

What the questioners have done here is unwittingly downplayed the negative aspects of the Deal, and also revealed that people have time preference! Future things are discounted by individuals. So when you offer them a good today and a risk tomorrow, the risk is valued less. If you make the same offer to them at a later date, the good is valued less and the risk is valued even less. You have essentially asked them if they want a better deal now, or a worse deal later.

Much better results could have been attained by asking questions that actually get to the heart of the mechanics of this deal. For example:

Question: Should the state tax workers an extra $819 million in income taxes so that property owners' property taxes are reduced by $819 million?

Question: Should the legislature give $819 million in overpaid taxes to schools even though it won't increase total school funding?

Question: Should the legislature spend $1.8 billion of it's surplus this year, and $819 million every year after, even if the Legislative Fiscal Bureau estimates it will result in a -$2.95 billion balance by the end of FY29 unless tax collections increase by 4% per year?

If you ask questions like this, as imperfect as they are (I do not envy you Pollsters), then you can begin to understand why both Democrats and Republicans opposed the Deal.

If you would like to truly understand how the Deal would have worked, its redistributive effects, and the risk it posed in exacerbating the structural deficit given the state's recent spending patterns, read MacIver's research here.