In 1870, Alfred Russell Wallace wagered ÂŁ500âa huge sum in those daysâthat he could prove the flat-Earther John Hampden wrong. Wallace duly did so, but the aggrieved Hampden never paid up.
Since then, a lively history of scientific wagers has ensuedâmany of them instigated by Stephen Hawking. Just last month in New York, the most famous recent wager was settled: a 25-year-old bet over one of the last great mysteries in science and philosophy. The bettors were neuroscientist Christof Koch and philosopher David Chalmers, both known for their pioneering work on the nature of consciousness. Chalmers won. Koch paid up.
Back in the late 1990s, consciousness science was full of renewed promise. Kochâa natural optimistâbelieved that 25 years was more than enough time for scientists to uncover the neural correlates of consciousness: those patterns of brain activity that underlie each and every one of our conscious experiences. Chalmers, a philosopher and therefore something of a pessimist by profession, demurred. In 1998, the pair staked a crate of fine wine on the outcome.
The bet was finally called at the annual meeting of the Association for the Scientific Study of Consciousness in New York a couple of weeks ago. Koch graciously handed Chalmers a bottle of Madeira on the conference stage. While much more is known about consciousness today than in the â90s, its true neural correlatesâand indeed a consensus theory of consciousnessâstill elude us.
Einsteinâs relativity isnât the last word in physics.
What helped resolve the wager was the outcome, or rather the lack of a decisive outcome, of an âadversarial collaborationâ organized by a consortium called COGITATE. Adversarial collaborations encourage researchers from different theoretical camps to jointly design experiments that can distinguish between their theories. In this case, the theories in question were integrated information theory (IIT), the brainchild of Giulio Tononi, and the neuronal global workspace theory (GWT), championed by Stanislas Dehaene. The two scientists made predictions, based on their respective theories, about what kinds of brain activity would be recorded in an experiment in which participants looked at a series of imagesâbut neither predicted outcome fully played out.
Given IITâs focus on specific causal structures in neuroanatomy, Tononi hypothesized that certain forms of neural activity would be observed during the experiment in the so-called âposterior hot zoneâ toward the back of the brain. But Dehaene predicted finding consciousness-related activity in the front of the brain, reflecting the emphasis of GWT on the âbroadcastâ of information throughout broad swathes of cortex. The results of the experiment did not fully conform to either prediction. Arguably, IIT fared a little better than GWT but not to the extent that any reasonable person could say that GWT had been disproved. And another experiment from COGITATE is still ongoing, which could yet change things around.
Though the adversarial collaboration failed to identify a clear winner, it was by no means in vain. It doesnât matter that the data didnât decisively refute (or indeed confirm) either theory. That was never going to happen. The theories are just too different in their assumptions and explanatory goals, and our experimental methods too coarse, to enable one theory to conclusively win out over another in any single set of experiments. Whatâs more, the COGITATE studies tested only limited aspects of each theoryâindeed, the core claims of each theory regarding the basis of consciousness were not tested at all.
What the COGITATE collaboration demonstrated was a new way of doing things in consciousness science. It brought scientists from different camps together to jointly figure out what challenges they must overcome to test their theories, and to collectively pursue the refinement of both theories and methods. Science, after all, is a process, not a destination.
Consciousness remains a tough nut to crack.Â
The findings of the collaboration remain extremely valuable. They will push forward the development of both IIT and GWTâand other theories of consciousness, tooâby providing new constraints and new explanatory targets. The data are also interesting in their own right, painting a detailed picture of where and when in the brain information about visual experience can be decoded, with both frontal and posterior regions playing a role. And because the COGITATE consortium will make the data publicly available, other researchers will have plentiful opportunities to take advantage of it. The junior researchers in the various labs who actually ran the experiments deserve enormous credit here, as do the project coordinators.
The experiment also taught us a lot about adversarial collaboration itself. As the Nobel Prize-winning economist Daniel Kahneman has frequently noted, adversarial collaborations are hard. The COGITATE collaboration sets an admirable precedent as the first of several supported by the Templeton World Charity Foundation, reflecting the rich array of theories of consciousness on offer. (Iâm involved in one of them, pitting IIT against a theory called âactive inferenceâ which emphasizes the role of neural predictions in conscious perception).
What I really like about the adversarial collaboration model is the requirement that theorists must state in advance how they expect the results from the proposed experiments to pan out. So if and when theories are subsequently adapted, it cannot be said that they predicted what actually happened all along. Such âhypothesizing after the results are knownââHARKing for shortâprogressively denudes theories of their explanatory and predictive power. By mitigating this temptation, theory development can become more transparent and more responsive to challenging findings. Future adversarial collaborations could take this further by asking theory proponents not only to pre-specify their predictions, but to also pre-specify their confidence in each prediction as well as the extent to which each prediction is core to their theory.
Adversarial collaborations also encourage theorists to design experiments that have the potential to be problematic for their preferred theory. Of course, this is what scientists should be doing anyway (a view Karl Popper took to an extreme), but it is not so common in practice. The pressures of academia instead incentivize researchers to design experiments that will produce evidence in support of their ideas. (You might think Iâm being uncharitable, but a recent survey of over 400 different experiments in consciousness science found the large majority were confirmatory, and delivered evidence in favor of their preferred theory.)
Consciousness remains a tough nut to crack, and none of the current crop of theories will be the last word on this last great mysteryânot even mine. Nor will pinning down the neural correlates of consciousness by itself be enough to answer the deeper question of how consciousness happens. Thereâll be many more iterations of theory and experiment before we reach the stage when decisive experiments can be doneâexperiments of the sort exemplified by Eddington and Dysonâs measurement of how the sun bends light, which confirmed Einsteinâs triumph over Newton. And Einsteinâs relativity isnât the last word in physics. Adversarial collaborations, even in an ideal form, are not a panacea. They will only ever be part of a scientific solution.
Back in New York, Christof Kochâever the optimistâwas keen to double down on his bet about finally finding the elusive neural correlates of consciousness. I imagine Chalmers might take it. But maybe, with adversarial collaborations added to the mix, and with exciting new developments in both theories and experimental methods in the pipeline, another 25 years might just do the trick. ![]()
Anil Seth is a neuroscientist at the University of Sussex, co-director of the Canadian Institute for Advanced Research (CIFAR) Program on Brain, Mind, and Consciousness, and author of Being You â A New Science of Consciousness. Thanks to Tim Bayne, Megan Peters, and Lucia Melloni for comments on early drafts.
Lead image: Ivan_Nikulin / Shutterstock






