Entropy and Temperature

  • It turns out that for a system of 2-state spins, we can calculate the entropy (to within a constant).
  • We shall find that for such a system, the entropy is proportional to $\ln w$, the logarithm of the number of microstates (rather than just the number of microstates).
  • Generalizing, that this system is not unique, we reason that for *any* system, entropy is proportional to $\ln w$.

Two spin systems in 'thermal' contact

Consider two spin systems, $A$ and $B$, which come into thermal contact with each other through a diathermal wall, while they're isolated from their surroundings. This means:

  • Diathermal wall: Energy can be exchanged across the boundary. The spin excesses, $s_A$ and $s_B$, may change.
  • Isolated from surroundings: The overall energy of the joint system stays constant. So $U\propto s_{system}=s_A+s_B$, the total spin excess does not change. Hereafter, we'll write this total spin excess for the system as just $s$.
  • Diathermal wall: No spins can cross the boundary (only energy), so $N_A$ and $N_B$ remain constant and do not change.
The function, $w(...)$ that gives the total number of ways or arranging the joint system is just the product of $w_A$, the number of ways of arranging system A, and $w_B$, the number of ways of arranging $w_B$: $$w_\text{system}=w_A(N_A,s_A)\cdot w_B(N_B,s_B)=w_A(N_A,s_A)\cdot w_B(N_B,s-s_A).$$

The most probable state (equilibrium state) of the system occurs when the derivative w.r.t $s_A$ of the function which gives the total number of microstates is 0: $$\frac d{ds_A}w_A(N_A,s_A) w_B(N_B,s-s_A) =0$$ Of course, the derivative of the logarithm of this function also vanishes at the same place, and since $\ln(xy)=\ln x+\ln y$: $$\left(\frac{\partial \ln w_A(N_A,s_A}{\partial s_A}\right)_{N_A}+\left(\frac{\partial \ln w_B(N_B,s-s_A)}{\partial s_A}\right)_{N_B}=0.$$

From the definition of the (constant) total spin excess, we have $s_A=s-s_B$ and therefore $ds_A=-ds_B$, so... $$\left(\frac{\partial \ln w_A(N_A,s_A}{\partial s_A}\right)_{N_A}-\left(\frac{\partial \ln w_B(N_B,s_B)}{\partial s_B}\right)_{N_B}=0.$$

Now, the energy, $U_A$ is proportional to the spin excess $s_A$, and $U_B\propto s_B$ with the same constant of proportionality, so the relation above should also hold true if we take derivatives with respect to the internal energies, instead of w.r.t. spin excesses. This means:

$$\left(\frac{\partial \ln w_A}{\partial U_A}\right)_{N_A}=\left(\frac{\partial \ln w_B}{\partial U_B}\right)_{N_B}.$$

This is the condition for thermal equilibrium between system A and system B. In classical thermodynamics (or everyday experience!) the condition for thermal equilibrium between two systems is....?

So, this suggests that either side of the equation is proportional to its temperature.

What should we make of the role of internal energy here?

Classical thermodynamics: When $N$ is held constant, the second law says $$dU=T\,dS-P\,dV.$$

If we further hold the volume constant, $T\,dS = [dU]_{V,N}$, so... $$\left(\frac{\del S}{\del U}\right)_{V,N}=\frac{1}{T}.$$

The expression above would express equality of temperature if $\ln w \propto S$. Writing this as... $$\ln w = \frac{S}{k},$$ it can be seen that the proportionality constant $k$ must have the same units as $S$, namely [J]/[K], since both $w$ and $\ln w$ must be unitless. ($w(N,U)$ is the # of microstates in a system with $N_A$ spins and an internal energy of $U$.

With minimal algebra:

Experimental studies of simple, well understood quantum systems allows the measurement of $k$ which turns out to be $$k_B=1.38066\times 10^{-23} \text{ J/K}.$$ This is the connection between statistical mechanics and classical thermodynamics!

Some texts will talk about the fundamental entropy, $\sigma$ (no units): $$\sigma(N,U) \equiv \ln w(N,U) = \frac{S}{k_B},$$ and a fundamental temperature, $\tau$ with units of energy: $$\left( \frac{\partial \sigma }{\partial U}\right)_{N}\equiv \frac{1}{\tau}=\frac{1}{k_B T}.$$

Increase of entropy

$$\sigma_{f}\approx \ln(w_Aw_B)_{max} \ge \sigma_i \approx \ln(w_Aw_B)_0.$$ Increase of entropy apparently means that an isolated system will spontaneously change in the direction of maximizing the number of available microstates.

Or, energy tends to *spread out* into all available states.

The state in which we have the most possible microstates is also the state in which we have a minimum for the information about exactly which microstate the system is in.

Alternative development #2


Consider two spin systems, initially separate, each insulated. Schematically, each system has its own energy, and contains a fixed number of spins, $N_A$ and $N_B$.

They're brought together and allowed to exchange energy, though not particles.

The spins have the same magnitude, so that the allowed changes in energy of one system exactly matches the allowed changes in the other system ($2mB$).

The initial energy of the two systems is $$U=U_A(s_A) + U_B(s_B) = -2mB(s_A+s_B) =-2mBs.$$

So, the total spin excess must be equal to the sum of the initial spin excesses: $$s=s_A+s_B.$$

 

Now, the two systems are brought into contact in such a way that energy may be exchanged (there is weak coupling), but not particles. We call this 'thermal' contact.

The energy of A or B can change, though the sum must still be equal $$U_A'+U_B' = U_A+U_B.$$

We also have $N=N_A+N_B$.

What determines equilibrium now?

  • We assume that all microstates are equally likely.
  • Consider different states that have different values of $s_A$ (and by implication, $s_B=s-s_A$.
  • To find $s_A$ at equilibrium, we need to find which which joint state (as a function of $s_A$ has the most microstates.

The total number of microstates for the combined system, in which the spin excess in the first system is $s_A$ is the product: $$w_A(N_A,s_A)w_B(N_B,s-s_A)=f(s_A).$$

 

The configuration with the maximum possible number of microstates should be the solution to... $$\frac{df}{ds_A}=0.$$

Writing out the product: $$\begin{align}w_A(N_A,s_A) & w_B(N_B,s-s_A) =\\
&= w_A(N_A,0)\cdot w_B(N_B,0)\cdot exp\left(-\frac{2s_A^2}{N_A} -\frac{2(s-s_A)^2}{N-N_A}\right).\end{align}$$

It's a little easier to take the derivative of the logarithm of this: $$\begin{align} \frac{\partial}{\partial s_A}& \left[ \ln\left(w_A(N_A,0) w_B(N_B,0)\right) -\frac{2s_A^2}{N_A} -\frac{2(s-s_A)^2}{N_B}\right]= \\
&=-\frac{4s_A}{N_A} +\frac{4(s-s_A)}{N_B}= 0.\end{align}$$

Re-arranging, and dividing by 4: $$\frac{s_A}{N_A} = \frac{s_B}{N_B}.$$

In the equilibrium configuration the spin excesses on both systems are the same.

If this is really a maximum, the second derivative should be negative: $$\frac{\partial}{\partial s_A}\left( -\frac{4s_A}{N_A} +\frac{4(s-s_A)}{N_B}\right) = -4\left(\frac{1}{N_A} + \frac{1}{N_B}\right).$$

Thermal equilibrium - alternate approach #3

To get the total number of microstates, we sum the product over all possible macrostates of system $A$ alone: $$w(N,s) = \sum_{s_A=-N_A/2}^{+N_A/2}w_A(N_A,s_A)w_B(N_B,s-s_A).$$

Since $U\propto s$, we could instead write this as a function of the internal energies: $$w(N,U)=\sum_{U_A}^{U_A<U}w_A(N_A,U_A)w_B(N_B,U-U_A).$$

Assuming that the product of two highly-peaked functions is also very peaky, the equilibrium state corresponds to the single, largest term in this sum, with largest joint number of microstates. That largest term where the number of microstates is a maximum) should be characterized by $dw=0$ for small variations in the partition of the energy. That is, if we write the Pfaffian of *one* term in the sum as: $$\begineq dw &= d\left[w_A(N_A,U_A)w_B(N_B,U_B)\right]_{N_A,N_B}\\ &= \left(\frac{\partial w_A}{\partial U_A}\right)_{N_A} w_B \, dU_A + w_A \left(\frac{\partial w_B}{\partial U_B}\right)_{N_B} \, dU_B,\endeq$$ then the *most likely* [equilibrium] term will be characterized by $dw=0$.

Because energy is conserved, $dU_A = -dU_B$.Also, dividing by $w_A w_B$... $$ \frac{1}{w_A}\left(\frac{\partial w_A}{\partial U_A}\right)_{N_A}=\frac{1}{w_B}\left(\frac{\partial w_B}{\partial U_B}\right)_{N_B}.$$

This is equivalent to:

$$\left(\frac{\partial \ln w_A}{\partial U_A}\right)_{N_A}=\left(\frac{\partial \ln w_B}{\partial U_B}\right)_{N_B}.$$

Image credits

Daderot