Entropy and Temperature

How do they crop up in statistical thermodynamics?

Most probable configuration of combined systems

Consider two spin systems, initially separate, each insulated. Schematically, each system has its own energy, and contains a fixed number of spins, $N_A$ and $N_B$.

They're brought together and allowed to exchange energy, though not particles.

The spins have the same magnitude, so that the allowed changes in energy of one system exactly matches the allowed changes in the other system ($2mB$).

The initial energy of the two systems is $$U=U_A(s_A) + U_B(s_B) = -2mB(s_A+s_B) =-2mBs.$$

So, the total spin excess must be equal to the sum of the initial spin excesses: $$s=s_A+s_B.$$


Now, the two systems are brought into contact in such a way that energy may be exchanged (there is weak coupling), but not particles. We call this 'thermal' contact.

The energy of A or B can change, though the sum must still be equal $$U_A'+U_B' = U_A+U_B.$$

We also have $N=N_A+N_B$.

What determines equilibrium now?

  • We assume that all microstates are equally likely.
  • Consider different states that have different values of $s_A$ (and by implication, $s_B=s-s_A$.
  • To find $s_A$ at equilibrium, we need to find which which joint state (as a function of $s_A$ has the most microstates.

The total number of microstates for the combined system, in which the spin excess in the first system is $s_A$ is the product: $$w_A(N_A,s_A)w_B(N_B,s-s_A)=f(s_A).$$


The configuration with the maximum possible number of microstates should be the solution to... $$\frac{df}{ds_A}=0.$$

Writing out the product: $$\begin{align}w_A(N_A,s_A) & w_B(N_B,s-s_A) =\\
&= w_A(N_A,0)\cdot w_B(N_B,0)\cdot exp\(-\frac{2s_A^2}{N_A} -\frac{2(s-s_A)^2}{N-N_A}\).\end{align}$$

It's a little easier to take the derivative of the logarithm of this: $$\begin{align} \frac{\partial}{\partial s_A}& \[ \ln\(w_A(N_A,0) w_B(N_B,0)\) -\frac{2s_A^2}{N_A} -\frac{2(s-s_A)^2}{N_B}\]= \\
&=-\frac{4s_A}{N_A} +\frac{4(s-s_A)}{N_B}= 0.\end{align}$$

Re-arranging, and dividing by 4: $$\frac{s_A}{N_A} = \frac{s_B}{N_B}.$$

In the equilibrium configuration the spin excesses on both systems are the same.

If this is really a maximum, the second derivative should be negative: $$\frac{\partial}{\partial s_A}\( -\frac{4s_A}{N_A} +\frac{4(s-s_A)}{N_B}\) = -4\(\frac{1}{N_A} + \frac{1}{N_B}\).$$

Thermal equilibrium - alternate approach

To get the total number of microstates, we sum the product over all possible macrostates of system $A$ alone: $$w(N,s) = \sum_{s_A=-N_A/2}^{+N_A/2}w_A(N_A,s_A)w_B(N_B,s-s_A).$$

Since $U\propto s$, we could instead write this as a function of the internal energies: $$w(N,U)=\sum_{U_A}^{U_A<U}w_A(N_A,U_A)w_B(N_B,U-U_A).$$

Assuming that the product of two highly-peaked functions is also very peaky, the equilibrium state corresponds to the single, largest term in this sum, with largest joint number of microstates. That largest term where the number of microstates is a maximum) should be characterized by $dw=0$ for small variations in the partition of the energy. That is, if we write the Pfaffian of *one* term in the sum as: $$\begineq dw &=& d\[w_A(N_A,U_A)w_B(N_B,U_B)\]_{N_A,N_B}\\ &=& \(\frac{\partial w_A}{\partial U_A}\)_{N_A} w_B \, dU_A + w_A \(\frac{\partial w_B}{\partial U_B}\)_{N_B} \, dU_B,\endeq$$ then the *most likely* [equilibrium] term will be characterized by $dw=0$.

Because energy is conserved, $dU_A = -dU_B$.Also, dividing by $w_A w_B$... $$ \frac{1}{w_A}\(\frac{\partial w_A}{\partial U_A}\)_{N_A}=\frac{1}{w_B}\(\frac{\partial w_B}{\partial U_B}\)_{N_B}.$$

This is equivalent to:

$$\(\frac{\partial \ln w_A}{\partial U_A}\)_{N_A}=\(\frac{\partial \ln w_B}{\partial U_B}\)_{N_B}.$$

This is the condition for thermal equilibrium between system A and system B. In classical thermodynamics (or everyday experience!) the condition for thermal equilibrium between two systems is....?

So, this suggests that either side of the equation is proportional to its temperature.

What should we make of the role of internal energy here?

Classical thermodynamics: When $N$ is held constant, the second law says $$dU=T\,dS-P\,dV.$$

If we further hold the volume constant, $T\,dS = [dU]_{V,N}$, so... $$\left(\frac{\del S}{\del U}\right)_{V,N}=\frac{1}{T}.$$

The expression above would express equality of temperature if $\ln w \propto S$. Writing this as... $$\ln w = \frac{S}{k},$$ it can be seen that the proportionality constant $k$ must have the same units as $S$, namely [J]/[K], since both $w$ and $\ln w$ must be unitless. ($w(N,U)$ is the # of microstates in a system with $N_A$ spins and an internal energy of $U$.

With minimal algebra:

Experimental studies of simple, well understood quantum systems allows the measurement of $k$ which turns out to be $$k_B=1.38066\times 10^{-23} \text{ J/K}.$$ This is the connection between statistical mechanics and classical thermodynamics!

Some texts will talk about the fundamental entropy, $\sigma$ (no units): $$\sigma(N,U) \equiv \ln w(N,U) = \frac{S}{k_B},$$ and a fundamental temperature, $\tau$ with units of energy: $$\( \frac{\partial \sigma }{\partial U}\)_{N}\equiv \frac{1}{\tau}=\frac{1}{k_B T}.$$

Increase of entropy

$$\sigma_{f}\approx \ln(w_Aw_B)_{max} \ge \sigma_i \approx \ln(w_Aw_B)_0.$$ Increase of entropy apparently means that an isolated system will spontaneously change in the direction of maximizing the number of available microstates.

Or, energy tends to *spread out* into all available states.

The state in which we have the most possible microstates is also the state in which we have a minimum for the information about exactly which microstate the system is in.

Image credits