Centennial: Einstein’s Desperate Mistake, the Cosmological Constant

There are multiple implications of the cosmological constant that make Einstein’s mistake so ironically important in the study of the universe.

Galaxy NGC 1398, photographed by the Dark Energy Survey. Credit: uclmaps/Flickr, CC BY 2.0

Galaxy NGC 1398, photographed by the Dark Energy Survey. Credit: uclmaps/Flickr, CC BY 2.0

“[The] concept of the cosmos that [Albert] Einstein derived form this general theory of relativity was elegant and magical. But there seemed to be one hitch, a flaw that needed to be fixed or fudged. His theory indicated that the universe would have to be either expanding or contracting, not staying static. According to his field equations, a static universe was impossible because the gravitational forces would pull all the matter together.” – Walter Isaacson, Einstein: His Life and Universe, 2007, p. 254.

On February 8, 1917, Einstein published a paper titled ‘Kosmologische Betrachtungen zur allgemeinen Relativitätstheorie‘ (‘Cosmological Considerations in the General Theory of Relativity’). In it, he described a number called the cosmological constant. The constant had a value such that, when used in his newly created equations describing the behaviour of the gravitational force, a non-changing universe was spit out – agreeing with knowledge at the time, as well as his belief, that the universe was static. Without the constant in the picture, on the other hand, Einstein’s general theory of relativity suggested that the gravitational pull of masses contained in the universe would pull all the matter together, keeping the universe dynamic.

It would be more than a decade before evidence would begin to emerge that the universe was expanding. And it would be scores of years before astronomers would find that the expansion was also accelerating. Then again, it would be many years before Einstein realised his actual mistake.

While he popularly considered his addition of the constant to be an affront to his own work, it may not have been as bad as he thought. The cosmological principle states, rather assumes, that the universe at the largest scales has the same properties everywhere. This is a spatial definition. An extension called the ‘perfect’ cosmological principle states that the universe at the largest scales has the same properties everywhere and at every time, implying that it has always remained the way it is today and it will continue to be this way forever. This is also called the steady-state theory, an alternative to the Big Bang theory that has been widely discredited – and which Einstein himself pursued for a while in 1931.

Anyway, in defence of Einstein, theoretical astrophysicist Peter Coles writes on his blog, “General relativity, when combined with the cosmological principle, but without the cosmological constant, requires the universe to be dynamical rather than static. If anything, therefore, you could argue that Einstein’s biggest blunder was to have failed to predict the expansion of the Universe!” Indeed, if Einstein had not decided to fudge his own monumental equations, he may have been onto something.

As Coles goes on to explain, the field equations, like all equations, have two sides: on the left are the effects of gravity due to the bending of spacetime; on the right are the effects of massive bodies and their motion through spacetime. Einstein added the cosmological constant to the left side, indicating that he thought it was gravity’s side of things that needed to be modified to imply a static universe. Modern cosmologists, however, use the term on the right side, to indicate that it is matter and its behaviour through spacetime that need a modification. As a result, the cosmological constant is often related to the energy of spacetime itself, also known as dark energy or vacuum energy.

A flat universe

In 1998, two astronomy teams found that the universe’s expansion was accelerating. They were able to independently come to the same conclusion by observing stars around the universe that were dying as Type Ia supernovae. When a mid- to low-mass star runs out of hydrogen to fuse, it will attempt to fuse carbon. However, if it can’t become sufficiently hot to commission the fusion process, the star will accumulate carbon and oxygen in its core. Over time, the outer layers will dissipate to leave behind a carbon-oxygen lump called a white dwarf. If it has a binary companion – an object, such as another star, orbiting it – then the white dwarf could start siphoning material from the object’s surface, but only upto a point.

In 1930, a 19-year-old Indian astrophysicist named Subrahmanyan Chandrasekhar had found that white dwarf stars should blow up as soon as they became more than 1.44-times as heavy as our Sun. This explosion is called a Type Ia supernova. Because it is moderated by the Chandrasekhar limit, its characteristics – such as its brightness – are always the same no matter whatever else happened before the limit was breached. The two astronomy teams were able to use this information to determine how far various white dwarfs were in the universe and, using the Doppler effect, how quickly they seemed to be moving away.

They found that the cosmological constant was positive. More specifically, assuming that the energy density of our universe was close to its critical density, they calculated the value of a number called omega-lambda to be about 0.7. The universe’s critical density determines how information behaves within the universe. For example, if the universe’s density is higher than its critical density, then astrophysicists predict it will be shaped like a sphere, and that particles that originate on one point of it will eventually circle back to it if they persisted for long enough – and the information will reset and ‘restart’. Our universe is assumed to be flat, “like a sheet of paper”, because its density is lower than the critical density. And in a flat universe, omega-lambda is the fraction of dark energy. This is how we say, “The universe is 70% dark energy”, etc. – because omega-lambda hovers around 0.7.

As is evident, there are multiple implications of the cosmological constant that make Einstein’s anxious mistake so ironically important in the study of the universe. Yet another example of this is the so-called Lambda-CDM model, which is a set of theories and principles that together strive to describe the universe’s properties. Its name itself is made up of two things that Einstein helped us discover: the ‘lambda’ refers to the cosmological constant and CDM stands for ‘cold dark matter’, which is thought to interact with normal matter only through the gravitational force.

However, irrespective of its internal consistency, the LCDM model has off late been challenged for its shifting attempts to explain dark matter. Most recently, a Dutch physicist named Erik Verlinde rooted for a modified theory of gravity to altogether explain away the need for this enigmatic substance, although his claims have been disputed.

On second thought, LCDM is a pseudo-pitstop to take a break at and mull the cosmological constant; a more meaningful place would be the philosophy of science itself. It is evident that the constant and its relationship to the number called omega-lambda are consequential to information theory, the study of how information flows in this universe – from one end of time to another, through materials and fields, into and out of blackholes. It also affects quantum mechanics and, in one way, string theory.

While observational studies have found that the constant is a really small number between 0 and 1, quantum mechanics predicts that the number should be 10120 higher! The size of the hierarchy problem in particle physics pales in comparison, but more importantly, the discrepancy hints at some violation of the idea of naturalness that some physicists are fond of. Brian Greene spun this for his cause in The Elegant Universe: “This presents a wonderful challenge and opportunity for string theorists: can calculations in string theory improve on this mismatch[?] … Should string theorists be able to rise to this challenge – as yet they have not – it would provide a compelling piece of evidence in support of the theory.”

The constant’s origins themselves were rooted in philosophical considerations – they had to be because its inclusion in the field equations had been arbitrary. In a paper published in 1919, he stated that the term was “gravely detrimental to the formal beauty of the theory” – significant in opposition to his belief that simple was beautiful, as elucidated in a lecture in 1933: “nature is the realisation of the simplest that is mathematically conceivable”. Finally, neither Einstein nor the physicists of today have been able to explain why the constant is necessary at all and why the theory can’t account for the universe’s expansion without it.

The ideas of Ernst Mach

But Einstein only sowed as much dissonance because he was an adherent of the philosophy of the Austrian phenomenalist Ernst Mach and his so-called Mach principle. Mach had a problem with Newton’s second law of motion, that a body’s acceleration is proportional to the force acting on it, because the Englishman hadn’t specified a frame of reference against which these observations would be true. Although Newton had invoked centrifugal and Coriolis forces when proving his second law in the context of one body rotating around another – like a satellite around Earth – his defence also relied on the existence of ‘absolute space’.

In other words, it mattered to Newton if Earth was going around the satellite or the satellite was going around Earth when working out the answer. However, space is invisible and Mach disliked its use because his philosophy of science was that only that which could be perceived by human senses could be true. For him, moreover, it didn’t matter if the planet was orbiting the satellite or vice versa. It only mattered that both situations shared a common geometry, which in turn morphed into the principle in 1872 that Einstein was inspired by later.

But as fate would have it, Einstein’s field equations, born 100 years ago last year, ended up postulating a universe that would either have to expand or contract for all eternity. And the introduction of the cosmological constant was a last-ditch attempt to stall this overwhelming outcome, to prevent invisible energies from hijacking what was already a four-dimensional spacetime continuum – all antithetical to Mach’s view of the universe. In a review paper, Cormac O’Raifeartaigh, Michael O’Keeffe, Werner Nahm and Simon Mitton, physicists from Ireland and the UK, wrote,

Where he intended to introduce a term to the field equations representing an attenuation of the gravitational interaction at large distances, he in fact introduced a term representing a very different effect. Indeed, the later interpretation of the cosmological term as representing a tendency for empty space to expand would have been deeply problematic for Einstein in 1917, given his understanding of Mach’s Principle at the time. Thus, while there is no question that relativity allowed the introduction of the cosmic constant, it appears that Einstein’s interpretation of the term may have been to some extent founded on a misconception.

Just as well, Einstein’s desperation did not work but in fact backfired: he would shed his faith in the Mach principle in the early 1930s and the cosmological constant would become a firm part of modern cosmology.

Note: This article was edited on February 9, 2017, to reflect doubts about attributing the phrase “biggest blunder” to Albert Einstein.

  • Pentcho Valev

    Einstein’s general relativity was an empirical concoction, not a deductive theory. Here are the two alternatives (there is no third one) described by Einstein himself:

    Albert Einstein: “From a systematic theoretical point of view, we may imagine the process of evolution of an empirical science to be a continuous process of induction. Theories are evolved and are expressed in short compass as statements of a large number of individual observations in the form of empirical laws, from which the general laws can be ascertained by comparison. Regarded in this way, the development of a science bears some resemblance to the compilation of a classified catalogue. It is, as it were, a purely empirical enterprise. But this point of view by no means embraces the whole of the actual process ; for it slurs over the important part played by intuition and deductive thought in the development of an exact science. As soon as a science has emerged from its initial stages, theoretical advances are no longer achieved merely by a process of arrangement. Guided by empirical data, the investigator rather develops a system of thought which, in general, is built up logically from a small number of fundamental assumptions, the so-called axioms.”

    Special relativity was indeed “built up logically from a small number of fundamental assumptions” (even though a false assumption and an invalid argument spoiled it from the very beginning) but general relativity was, to use Einstein’s words, “a purely empirical enterprise”. Einstein and his mathematical friends changed and fudged equations countless times until “a classified catalogue” was compiled where known in advance results and pet assumptions (such as the Mercury’s precession, the equivalence principle, gravitational time dilation) coexisted in an apparently consistent manner. Being an empirical concoction, general relativity allows Einsteinians to introduce, change and withdraw fudge factors until the “theory” manages to predict anything Einsteinians want. Then the prediction turns out to be confirmed by observations (surprise surprise).

    Here Michel Janssen describes endless empirical groping, fudging and fitting until “excellent agreement with observation” was reached:

    Michel Janssen: “But – as we know from a letter to his friend Conrad Habicht of December 24, 1907 – one of the goals that Einstein set himself early on, was to use his new theory of gravity, whatever it might turn out to be, to explain the discrepancy between the observed motion of the perihelion of the planet Mercury and the motion predicted on the basis of Newtonian gravitational theory. […] The Einstein-Grossmann theory – also known as the “Entwurf” (“outline”) theory after the title of Einstein and Grossmann’s paper – is, in fact, already very close to the version of general relativity published in November 1915 and constitutes an enormous advance over Einstein’s first attempt at a generalized theory of relativity and theory of gravitation published in 1912. The crucial breakthrough had been that Einstein had recognized that the gravitational field – or, as we would now say, the inertio-gravitational field – should not be described by a variable speed of light as he had attempted in 1912, but by the so-called metric tensor field. The metric tensor is a mathematical object of 16 components, 10 of which independent, that characterizes the geometry of space and time. In this way, gravity is no longer a force in space and time, but part of the fabric of space and time itself: gravity is part of the inertio-gravitational field. Einstein had turned to Grossmann for help with the difficult and unfamiliar mathematics needed to formulate a theory along these lines. […] Einstein did not give up the Einstein-Grossmann theory once he had established that it could not fully explain the Mercury anomaly. He continued to work on the theory and never even mentioned the disappointing result of his work with Besso in print. So Einstein did not do what the influential philosopher Sir Karl Popper claimed all good scientists do: once they have found an empirical refutation of their theory, they abandon that theory and go back to the drawing board. […] On November 4, 1915, he presented a paper to the Berlin Academy officially retracting the Einstein-Grossmann equations and replacing them with new ones. On November 11, a short addendum to this paper followed, once again changing his field equations. A week later, on November 18, Einstein presented the paper containing his celebrated explanation of the perihelion motion of Mercury on the basis of this new theory. Another week later he changed the field equations once more. These are the equations still used today. This last change did not affect the result for the perihelion of Mercury. Besso is not acknowledged in Einstein’s paper on the perihelion problem. Apparently, Besso’s help with this technical problem had not been as valuable to Einstein as his role as sounding board that had earned Besso the famous acknowledgment in the special relativity paper of 1905. Still, an acknowledgment would have been appropriate. After all, what Einstein had done that week in November, was simply to redo the calculation he had done with Besso in June 1913, using his new field equations instead of the Einstein-Grossmann equations. It is not hard to imagine Einstein’s excitement when he inserted the numbers for Mercury into the new expression he found and the result was 43″, in excellent agreement with observation.”

    Pentcho Valev