Skip to main content
    • Aa
    • Aa

Algorithmic thermodynamics

  • JOHN BAEZ (a1) and MIKE STAY (a2)

Algorithmic entropy can be viewed as a special case of the entropy studied in statistical mechanics. This viewpoint allows us to apply many techniques developed for use in thermodynamics to the subject of algorithmic information theory. In particular, suppose we fix a universal prefix-free Turing machine and let X be the set of programs that halt for this machine. Then we can regard X as a set of ‘microstates’, and treat any function on X as an ‘observable’. For any collection of observables, we can study the Gibbs ensemble that maximises entropy subject to constraints on the expected values of these observables. We illustrate this by taking the log runtime, length and output of a program as observables analogous to the energy E, volume V and number of molecules N in a container of gas. The conjugate variables of these observables allow us to define quantities we call the ‘algorithmic temperature’ T, ‘algorithmic pressure’ P and ‘algorithmic potential’ μ, since they are analogous to the temperature, pressure and chemical potential. We derive an analogue of the fundamental thermodynamic relation dE = TdSPdV + μdN, and use it to study thermodynamic cycles analogous to those for heat engines. We also investigate the values of T, P and μ for which the partition function converges. At some points on the boundary of this domain of convergence, the partition function becomes uncomputable – indeed, at these points the partition function itself has non-trivial algorithmic entropy.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

C. H Bennett , P. Gàcs , M. Li , M. B. Vitányi and W. H. Zurek (1998) Information distance. IEEE Transactions on Information Theory 44 14071423.

C. S. Calude (2002) Information and Randomness: An Algorithmic Perspective, Springer-Verlag.

C. S. Calude and M. A. Stay (2006) Most programs stop quickly or never halt. Advances in Applied Mathematics 40 295308.

C. S. Calude and M. A. Stay (2006) Natural halting probabilities, partial randomness, and zeta functions. Information and Computation 204 17181739.

C. S. Calude , L. Staiger and S. A. Terwijn (2006) On partial randomness. Annals of Pure and Applied Logic 138 2030.

G. Chaitin (1975) A theory of program size formally identical to information theory. Journal of the ACM 22 329340.

G. Chaitin (1976) Algorithmic entropy of sets. Computers and Mathematics with Applications 2 233245.

E. Fredkin and T. Toffoli (1982) Conservative logic. International Journal of Theoretical Physics 21 219253.

E. T. Janyes (1957) Information theory and statistical mechanics. Physical Review 106 620630.

S. Kullback and R. A. Leibler (1951) On information and sufficiency. Annals of Mathematical Statistics 22 7986.

M. Li and P. Vitányi (2008) An Introduction to Kolmogorov Complexity Theory and its Applications, Springer-Verlag.

M. B. Pour-El and J. I. Richards (1989) Computability in Analysis and Physics, Springer Verlag.

R. J. Solomonoff (1964) A formal theory of inductive inference, part I. Information and Control 7 122.

L. Szilard (1929) On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings. Zeitschrift für Physik 53 840856. (English translation (2003) In: H. S. Leff and A. F. Rex (eds.) Maxwell's Demon 2: Entropy, Information, Computing, Adam Hilger.)

K. Tadaki (2002) A generalization of Chaitin's halting probability Ω and halting self-similar sets. Hokkaido Mathematical Journal 31 219253.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Mathematical Structures in Computer Science
  • ISSN: 0960-1295
  • EISSN: 1469-8072
  • URL: /core/journals/mathematical-structures-in-computer-science
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 9 *
Loading metrics...

Abstract views

Total abstract views: 102 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 19th September 2017. This data will be updated every 24 hours.