Statistical Interpretation of Entropy and the Second Law of Thermodynamics: The Underlying Explanation

What are the possible outcomes of tossing 5 coins? Each coin can land either heads or tails. On the large scale, we are concerned only with the total heads and tails and not with the order in which heads and tails appear. The following possibilities exist:

These are what we call macrostates. A macrostate is a description of a system in terms of its macroscopic properties. It does not specify the details of the system, such as the order in which heads and tails occur or which coins are heads or tails.

Using this nomenclature, a system of 5 coins has the 6 possible macrostates just listed. Some macrostates are more likely to occur than others. For instance, there is only one way to get 5 heads, but there are several ways to get 3 heads and 2 tails, making the latter macrostate more probable. Table 8.14.1 lists of all the ways in which 5 coins can be tossed, taking into account the order in which heads and tails occur. Each sequence is called a microstate—a detailed description of every element of a system.

The macrostate of 3 heads and 2 tails can be achieved in 10 ways and is thus 10 times more probable than the one having 5 heads. Not surprisingly, it is equally probable to have the reverse, 2 heads and 3 tails. Similarly, it is equally probable to get 5 tails as it is to get 5 heads. Note that all of these conclusions are based on the crucial assumption that each microstate is equally probable. With coin tosses, this requires that the coins not be asymmetric in a way that favors one side over the other, as with loaded dice. For this counting argument, the accessible microstates must be equally probable.

For five fair coins, the macrostates with all heads or all tails each have one microstate. Together, they account for only 2 of the 32 equally likely outcomes. The macrostates with three heads and two tails, or two heads and three tails, together have 20 microstates. They are therefore much more likely. Starting with five heads and tossing the coins again is likely to produce a macrostate compatible with more microstates. The reverse remains possible, but is less likely.

This result becomes dramatic for larger systems. Consider what happens if you have 100 coins instead of just 5. The macrostates with the fewest microstates are 100 heads or 100 tails. The macrostate with the most microstates is 50 heads and 50 tails. There is only 1 way (1 microstate) to get the all-heads arrangement of 100 heads. There are 100 ways (100 microstates) to get the arrangement of 99 heads and 1 tail (also 100 to get its reverse). And there are $1.0\times {10}^{29}$ ways to get 50 heads and 50 tails, the most probable macrostate. Table 8.14.2 is an abbreviated list of the various macrostates and the number of microstates for each macrostate. The total number of microstates—the total number of different ways 100 coins can be tossed—is an impressively large $1\text{.}27\times {10}^{30}$. Now, if we start with a macrostate like 100 heads and toss the coins, there is a virtual certainty that we will get a macrostate with more microstates. If we keep tossing the coins, it is possible, but exceedingly unlikely, that we will ever get back to the all-heads or all-tails macrostate. If you tossed the coins once each second, you could expect to get either 100 heads or 100 tails once in $2\times {\text{10}}^{\text{22}}$ years! This period is 1 trillion (${\text{10}}^{\text{12}}$) times longer than the age of the universe, and so the chances are essentially zero. In contrast, there is an 8% chance of getting 50 heads, a 73% chance of getting from 45 to 55 heads, and a 96% chance of getting from 40 to 60 heads. Macrostates with more microstates are more likely.

The rapid growth in the odds favoring macrostates with more microstates that we see in going from 5 to 100 coins continues as the number of entities in the system increases. Let us now imagine applying this approach to perhaps a small sample of gas. Because counting microstates and macrostates involves statistics, this is called statistical analysis. The macrostates of a gas correspond to its macroscopic properties, such as volume, temperature, and pressure; and its microstates correspond to the detailed description of the positions and velocities of its atoms. Even a small amount of gas has a huge number of atoms: $1.0{\text{ cm}}^3$ of an ideal gas at 1.0 atm and $\mathrm{0°\; C}$ has $2\text{.}7\times {\text{10}}^{\text{19}}$ atoms. So each macrostate has an immense number of microstates. In plain language, this means that there are an immense number of ways in which the atoms in a gas can be arranged, while still having the same pressure, temperature, and so on.

For a gas in equilibrium, a nearly uniform distribution of molecules throughout the available volume is compatible with far more microscopic arrangements than a state with all molecules confined to one small region. A substantial spontaneous concentration of the molecules is therefore overwhelmingly unlikely. This statistical imbalance explains why an isolated gas expands into available space.

The gas has greater entropy when more positions are available to its molecules. We can reduce its entropy by compressing it at constant temperature, doing work and transferring heat to the surroundings. The entropy gained by the surroundings equals the gas’s entropy decrease for reversible compression, and exceeds it for irreversible compression. If the compressed gas is then allowed to expand freely into an evacuated part of an insulated container, its entropy increases. A spontaneous return to the concentrated state (Figure 8.14.2(b)) is overwhelmingly unlikely. The expanded macrostate is compatible with vastly more microscopic arrangements.

Macrostates with more compatible microstates have greater entropy and are more probable. The great Austrian physicist Ludwig Boltzmann (1844–1906)—who, along with Maxwell, made so many contributions to kinetic theory—proved that the entropy of a system in a given state (a macrostate) can be written as

where $k_{\mathrm{B}}=1\text{.}38\times {10}^{-23}\text{J/K}$ is Boltzmann’s constant, and $\text{ln}W$ is the natural logarithm of the number of microstates $W$ corresponding to the given macrostate. $W$ is proportional to the probability that the macrostate will occur when the accessible microstates are equally probable. Thus entropy is directly related to the probability of a state—the more likely the state, the greater its entropy. Boltzmann proved that this expression for $S$ is equivalent to the definition $\mathrm{\Delta }S=Q/T$, for reversible heat transfer at constant temperature, which we have used extensively. The formula is included here as a reference; no logarithm calculations are needed for this section. The key point is that more microstates correspond to greater entropy.

Thus the second law of thermodynamics is explained on a statistical level: the entropy of an isolated macroscopic system remains constant or increases. Macrostates with greater entropy have vastly more microstates, making a substantial spontaneous decrease overwhelmingly unlikely. A system that exchanges heat with its surroundings can decrease in entropy while the total entropy of the system and surroundings increases or remains constant.