Epstein Files Full PDF

This article's factual accuracy is disputed. Relevant discussion may be found on the talk page. Please help to ensure that disputed statements are reliably sourced. (May 2025) (Learn how and when to remove this message)

In information theory and statistics, negentropy is used as a measure of distance to normality. It is also known as negative entropy or syntropy.

The concept and phrase "negative entropy" was introduced by Erwin Schrödinger in his 1944 book What is Life?.^[1] Later, the French physicist Léon Brillouin shortened the phrase to néguentropie (transl. negentropy).^[2][3] In 1974, Albert Szent-Györgyi proposed replacing the term negentropy with syntropy. That term may have originated in the 1940s with the Italian mathematician Luigi Fantappiè, who tried to construct a unified theory of biology and physics. Buckminster Fuller tried to popularize this usage, but negentropy remains common.^{[citation needed]}

In a note to What is Life?, Schrödinger explained his use of this phrase:

... if I had been catering for them [physicists] alone I should have let the discussion turn on free energy instead. It is the more familiar notion in this context. But this highly technical term seemed linguistically too near to energy for making the average reader alive to the contrast between the two things.

This section is missing information about the mathematical treatment of negentropy in information theory. Please expand the section to include this information. Further details may exist on the talk page. (December 2024)

In information theory and statistics, negentropy is used as a measure of distance to normality.^[4][5][6] Out of all probability distributions with a given mean and variance, the Gaussian or normal distribution is the one with the highest entropy.^{[clarification needed][citation needed]} Negentropy measures the difference in entropy between a given distribution and the Gaussian distribution with the same mean and variance. Thus, negentropy is always nonnegative, is invariant by any linear invertible change of coordinates, and vanishes if and only if the signal is Gaussian.^{[citation needed]}

Negentropy is defined as

${\displaystyle J(Y)=h(Y_{G})-h(Y),}$

where ${\displaystyle h(Y_{G})={\tfrac {1}{2}}\log \left(2\pi \mathrm {e} \cdot \sigma ^{2}\right)}$ is the differential entropy of a normal distribution ${\displaystyle Y_{G}\sim N(\mu ,\sigma ^{2})}$ with the same mean ${\displaystyle \mu }$ and variance ${\displaystyle \sigma ^{2}}$ as ${\displaystyle Y}$, and ${\displaystyle h(Y)}$ is the differential entropy of ${\displaystyle Y}$, with ${\displaystyle p_{Y}}$ as its probability density function:

${\displaystyle h(Y)=-\int p_{Y}(u)\log p_{Y}(u)\,\mathrm {d} u}$

Negentropy is used in statistics and signal processing. It is related to network entropy, which is used in independent component analysis.^[7][8]

The negentropy of a distribution is equal to the Kullback–Leibler divergence between ${\displaystyle Y}$ and a Gaussian distribution with the same mean and variance as ${\displaystyle Y}$ (see Differential entropy § Maximization in the normal distribution for a proof):$${\displaystyle J(Y)=D_{KL}(Y\ \Vert \ Y_{G})}$$In particular, it is always nonnegative (unlike differential entropy, which can be negative).

This section may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (June 2025) (Learn how and when to remove this message)

There is a physical quantity closely linked to free energy (free enthalpy), with a unit of entropy and isomorphic to negentropy known in statistics and information theory. In 1873, Willard Gibbs created a diagram illustrating the concept of free energy corresponding to free enthalpy. On the diagram one can see the quantity called capacity for entropy. This quantity is the amount of entropy that may be increased without changing an internal energy or increasing its volume.^[9] In other words, it is a difference between maximum possible, under assumed conditions, entropy and its actual entropy. It corresponds exactly to the definition of negentropy adopted in statistics and information theory. A similar physical quantity was introduced in 1869 by Massieu for the isothermal process^[10][11][12] (both quantities differs just with a figure sign) and by then Planck for the isothermal-isobaric process.^[13] More recently, the Massieu–Planck thermodynamic potential, known also as free entropy, has been shown to play a great role in the so-called entropic formulation of statistical mechanics,^[14] applied among the others in molecular biology^[15] and thermodynamic non-equilibrium processes.^[16]

${\displaystyle J=S_{\max }-S=-\Phi =-k\ln Z\,}$

where:

${\displaystyle S}$ is entropy

${\displaystyle J}$ is negentropy (Gibbs "capacity for entropy")

${\displaystyle \Phi }$ is the Massieu potential

${\displaystyle Z}$ is the partition function

${\displaystyle k}$ the Boltzmann constant

In particular, mathematically the negentropy (the negative entropy function, in physics interpreted as free entropy) is the convex conjugate of LogSumExp (in physics interpreted as the free energy).

In 1953, Léon Brillouin derived a general equation^[17] stating that the changing of an information bit value requires at least ${\displaystyle kT\ln 2}$ energy. This is the same energy as the work Leó Szilárd's engine produces in the idealistic case. In his book,^[18] he further explored this problem concluding that any cause of this bit value change (measurement, decision about a yes/no question, erasure, display, etc.) will require the same amount of energy.

• Exergy
• Free entropy
• Entropy in thermodynamics and information theory

Look up negentropy in Wiktionary, the free dictionary.