Is theoretical physics broken? Or is it just hard?

  

 When you don’t have enough clues to bring your detective story to a close, you should expect that your educated guesses will all be wrong. Is all of modern theoretical physics pointless? If you listen to a disillusioned high-energy physicist, you might conclude that it is. After all, the 20th century was a century of theoretical triumphs: we were able, on both subatomic and cosmic scales, to at last make sense of the Universe that surrounded and comprised us. We figured out what the fundamental forces and interactions governing physics were, what the fundamental constituents of matter were, how they assembled to form the world we observe and inhabit, and how to predict what the results of any experiment performed with those quanta would be. Combined, the Standard Model of elementary particles and the standard model of cosmology represent the culmination of 20th century physics. While experiments and observations have revealed a number of hitherto unsolved puzzles — puzzles like dark matter, dark energy, cosmic inflation, Baryogenesis, massive neutrinos, the strong CP problem, and numerous others — theorists have failed to make significant progress on all of these issues over the past 25+ years. Have they all simply been wasting their time? That’s an unfair accusation. It’s easy to criticize, but suggestions for what they should be doing instead are largely even worse. Here’s a fairer look at the situation. This chart of the particles and interactions details how the particles of the Standard Model interact according to the three fundamental forces that Quantum Field Theory describes. When gravity is added into the mix, we obtain the observable Universe that we see, with the laws, parameters, and constants that we know of governing it. Mysteries, such as dark matter and dark energy, still remain. (Credit: Contemporary Physics Education Project/DOE/SNF/LBNL) It’s true, in the 20th century, there were a slew of theoretical advances that led to meaningful predictions that were later verified. Some of these include: the prediction of positrons: the antimatter counterpart of electrons, the prediction of the neutrino: a subatomic, energy-and-momentum-carrying particle participating in nuclear reactions, the prediction of quarks as constituents of the proton and neutron, the prediction of additional “generations” of both quarks and leptons, the structure of the Standard Model, with the strong nuclear force, the weak nuclear force, and the electromagnetic force, the prediction of electroweak unification and the Higgs boson, the prediction of the Big Bang and the cosmic microwave background, the prediction of cosmic inflation and the imperfections in the cosmic microwave background, and the prediction of cold dark matter and its implications for large-scale structure formation in the Universe. These remarkable successes led to our standard picture of the Universe today: a picture which, at its heart, consists of the Standard Model of elementary particles and of General Relativity governing the gravitational force. The largest-scale observations in the Universe, from the cosmic microwave background to the cosmic web to galaxy clusters to individual galaxies, all require dark matter to explain what we observe. At both early times and late times, that same 5-to-1 dark matter-to-normal matter ratio is required. (Credit: Chris Blake and Sam Moorfield) On the other hand, physics didn’t end with these discoveries or with this picture, which has been in place — more or less — since the early 1980s. Sure, details of cosmic inflation, the massive nature of neutrinos, and the existence of dark energy have been revealed since then: a triumph of perhaps a more modest nature. But what has recent work in theoretical physics given us atop this standard picture? Supersymmetry, whose particles do not appear to exist. Extra dimensions, whose predictions do not appear in our experiments or observations. Grand unification, which has no evidence supporting its existence. String theory, which has not given us a single testable prediction. Modifications to gravity, which add additional parameters but have failed to create a consistent picture that supersedes General Relativity. Modifications to cold, collisionless dark matter, which, again, add additional parameters that are wholly unnecessary, failing to supersede the simplest cold dark matter models. And modifications to the simplest picture of (constant) dark energy, which yet again add additional parameters but have nothing to offer above and beyond the simplest model of dark energy. There are all sorts of ways that people have attempted to break-and-bend the existing laws of physics over the past few decades, and none of them do a better job at explaining what we observe and measure than the standard picture without any additional modifications. The far distant fates of the Universe offer a number of possibilities, but if dark energy is truly a constant, as the data indicates, it will continue to follow the red curve, leading to the long-term scenario frequently described on Starts With A Bang: of the eventual heat death of the Universe. If dark energy evolves with time, a Big Rip or a Big Crunch are still admissible, but we don’t have any evidence indicating that this evolution is anything more than idle speculation. (Credit: NASA/CXC/M. Weiss) This is not what “failure” looks like. This is what theoretical physics looks like — and what at least a portion of theoretical physics has always looked like — when we have insufficient data to point us in the right direction about what lies beyond the currently accepted consensus picture of reality. It’s easy to go back to the 20th century and point to the successes and say, “look how good we were at predicting what would come next!” Sure, but one could just as easily go back to the 20th century and pick out any of the much more numerous conjectures that turned out to not describe our reality very well at all. It turns out we all have a selective memory when we look back on our triumphs; we overlook all of the attempts that didn’t pan out. We remember the quark model, not the Sakata model. We remember General Relativity, not the Newcomb and Hall modifications to Newton’s laws. We remember quantum chromodynamics, not the “guess the S-matrix” approach. We remember the neutron, not the idea that there were proton-electron bound states within the nucleus. We remember the Higgs model, not technicolor models. We remember the expanding Universe, not the tired light theory. We remember the Big Bang, not the Steady State model. We remember cosmic inflation, not a variable speed of light. That’s the first problem with the “theorists are all wrong” take: when we grow up, scientifically, we take for granted what was achieved in the past, but not how we got there, nor the missteps along the way. The main galaxies of Stephan’s Quintet, as revealed by JWST on July 12, 2022. The galaxy on the left is only about ~15% as distant as the other galaxies, and the background galaxies are many scores of times farther away. And yet, they’re all equally sharp, demonstrating that the tired light hypothesis, which predicts increasing “blurriness” with increasing redshift, is without merit. (Credit: NASA, ESA, CSA, and STScI) The second problem is this: theorists have no expectation of knowing what comes next when the experimental and observational data we do possess is insufficient to light the way. During the 20th century, revolutionary data came in at an alarming rate as new particle physics experiments were performed at higher energies, with better statistics, and in novel environments, such as above Earth’s atmosphere. Similarly, in astronomy, larger apertures, advances in photography and spectroscopy, the development of multi-wavelength astronomy beyond the visible light spectrum, and the first space telescopes all brought in new observational data that upended many pre-existing ideas. A heavier “cousin” of the electron, the muon, was first revealed by balloon-borne experiments that enabled us to detect their presence among the cosmic rays. Deep inelastic scattering experiments — i.e., high-energy collisions between particles with precision measurements of the particle shrapnel that comes out — revealed that the proton and neutron were composite particles, but the electron was not. Nuclear reactors, where heavy elements were transmuted into lighter ones, released antineutrinos that could be absorbed by atomic nuclei outside of the reactor, leading to their discovery.