Defending Social Science Against Those Who Would Prefer to Know Nothing

Earlier this week PM included this line in defense of NSF funding for political science at DoM:

Indeed, the alternative to good social science is not no social science but bad social science.

The implication from that is not that all social science is good but that it strives to be, is useful when it is, and in aggregate has improved the stock of human knowledge. Enough has been written about the battle over NSF funding that I feel no need to weigh in, but I must object to this ridiculous post by Gary Gutting at the NYT (via The Monkey Cage). It's titled "How Reliable Are the Social Sciences?" and it argues, I guess, "not enough for it to influence policy". Or in the authors words:

How much authority should we give to such work in our policy decisions? The question is important because media reports often seem to assume that any result presented as “scientific” has a claim to our serious attention. But this is hardly a reasonable view. There is considerable distance between, say, the confidence we should place in astronomers’ calculations of eclipses and a small marketing study suggesting that consumers prefer laundry soap in blue boxes.

He then goes on to make a nearly uncountable number of false assertions, misplaced blames, understatements of the usefulness of the social sciences, and overstatements of the development of the natural sciences. It's hard to know where to begin. Take the above quote for starters: The first sentence asks about policy decisions. The concluding sentences reference two potential avenues for inquiry that, so far as I can tell, have nothing whatsoever to do with policy.

Or take this:

But often, as I have pointed out for the case of biomedical research, popular reports often do not make clear the limited value of a journalistically exciting result. Good headlines can make for bad reporting.

Biomedical research may have implications for policy -- e.g. what treatments should be covered in insurance plans or something -- but it is not a social science. Additionally, if journalists misread research findings that is an indictment of journalists, not researchers.

In the next paragraph:

Second, and even more important, there is our overall assessment of work in a given science in comparison with other sciences. The core natural sciences (e.g., physics, chemistry, biology) are so well established that we readily accept their best-supported conclusions as definitive. (No one, for example, was concerned about the validity of the fundamental physics on which our space program was based.) Even the best-developed social sciences like economics have nothing like this status.

Are the core sciences so settled? I'm no expert in any of them, but my impression is that they are filled with hotly-contested debates concerning almost all of their "definitive" conclusions. Do we exist in a multiverse? Does evolution operate at the level of the genotype or phenotype? Can a universe (or multiverse) arise from nothing, and if so what is the definition of "nothing"? Is consciousness a biological function, and if so how did it arise and to what species does it extend?

Should we accept the validity of bodies of work based only on their perceived "status"? As determined by who, exactly? 40% of Americans don't believe that evolution occurred. Despite clear evidence, only 60% of Americans believe that climate change is happening at all, and a majority believe that humans do not play a major role in altering the climate. Granted, those polls are examples of social science so Gutting would likely give them no credence but even then the question remains: where does this status come from, and why should it matter?

Perhaps more pointedly, everyone should well be concerned about the "validity of the fundamental physics on which our space program was based" considering the impressive number of boondoggles and outright tragedies that resulted from it.

There are many questions of policy relevance that social scientists have not yet answered sufficiently well to base policy on them. There are also many questions with policy relevance that are more or less "settled" by social scientists, in that they have coherent theoretical explanations that are supported by multiple empirical studies. There's no point in running down a list, which could only be illustrative in any case, but one example might be that if you pay people to put silly arguments into print under their own name then they will be more likely to do it than if they were taxed for it.

A bit further down:

Is there any work on the effectiveness of teaching that is solidly enough established to support major policy decisions?

The case for a negative answer lies in the predictive power of the core natural sciences compared with even the most highly developed social sciences. Social sciences may be surrounded by the “paraphernalia” of the natural sciences, such as technical terminology, mathematical equations, empirical data and even carefully designed experiments. But when it comes to generating reliable scientific knowledge, there is nothing more important than frequent and detailed predictions of future events. We may have a theory that explains all the known data, but that may be just the result of our having fitted the theory to that data. The strongest support for a theory comes from its ability to correctly predict data that it was not designed to explain.

This is both true and one of the weaknesses of the social sciences. But refer to the quote from PM with which I began this post, and consider the relative youth of social scientific inquiry that has employed the "paraphernalia" of the natural sciences. In some cases we've only been doing it for a decade or several, mostly with pretty poor data and not much sophistication. But these problems are being corrected with time and experience -- as science should do -- and even the flawed efforts of the past are better, in aggregate, than their alternative: setting policy according to how Thomas Friedman's (or Gary Gutting's) gut feels about it.

Gutting then goes on to suggest that social science's problem is that it cannot do randomized controlled experiments on its subjects and therefore cannot make "detailed and precise predictions". Setting aside the fact that, in many cases, neither can astrophysicists or evolutionary biologists, this is a) changing in the social sciences; b) not a panacea in any science, social or otherwise; c) RCTs are about testing hypotheses (or, sometimes, just seeing what happens) not generating them; d) the analysis of observational data has always been considered a valid way to examine the rightness of theories in the sciences. Should Galileo not have been trusted because he couldn't perform a randomized controlled experiment on the earth's orbit around the sun? Had Darwin no insight on natural selection because he could not perform a trial to determine why the beaks of finches in different places varied in thickness?

Even more absurdly Gutting then writes:

Without a strong track record of experiments leading to successful predictions, there is seldom a basis for taking social scientific results as definitive.

True. True not because social science findings are so unreliable as to make them useless. True because all findings in all sciences are provisional. That is the hallmark of science. To criticize science for not being definitive is like criticizing philosophers for asking questions.

Having painted himself into such a corner Gutting has no choice but to conclude:

Given the limited predictive success and the lack of consensus in social sciences, their conclusions can seldom be primary guides to setting policy. At best, they can supplement the general knowledge, practical experience, good sense and critical intelligence that we can only hope our political leaders will have. 

Well then God help us all.

More seriously, perhaps we should use research to influence policy decisions when the research is relevant and when there is strong empirical support from multiple research programs, while recognizing that changed circumstances or new information may cause us to modify these programs later. I'm not suggesting that ever will quite happen -- if they notice it all opportunistic leaders will cherrypick findings that support their preferences and disregard the rest -- but surely that's a better goal than relying on the "good sense and critical intelligence" our overlords do not possess to solve our problems.

And when our leaders stray, or when they set policy cynically, it is the role of intellectuals to use every faculty at their disposal to point out the emperor's nakedness. That is easier done with the arsenal of social science at one's disposal than with intuition or impression alone.

Gutting is a professor of philosophy at Notre Dame. It's worrying to see him so dismissive of his colleagues in the academy. Perhaps he should give them some more of his time.