According to Aristotle, it is our ability to reason which sets humans apart from the rest of the animal kingdom. The understanding and manipulation of our environment that has made us so successful has only been possible through this unique ability.
Reasoning is often broken down into two broad categories. Firstly there is
deductive reasoning which can be thought of as the process of drawing logically valid conclusions from some assumed or given premise. Deductive reasoning is the type of reasoning used in mathematical proofs or when dealing with formal systems. Although this type of reasoning is obviously necessary, it is not always adequate.
When reasoning about our world we often want to make predictions that involve estimations and generalizations. For this we use inductive reasoning.
Inductive reasoning can be thought of as drawing the “best” conclusions from a set of observations. Unfortunately these observations are almost always incomplete in some sense and therefore we can never be certain of the conclusions we make. This process is analogous to the scientific process in general. In science, rules and models are found by generalizing
patterns observed locally. These models are then used to understand and predict our environment which in turn allows us to benefit, usually with great success. But like inductive inference, scientific hypotheses can never be completely validated, so we can never know whether they are true for certain. The difference between reasoning inductively or deductively can also be simply thought of as the difference between reasoning about the known or the unknown.
Ray Solomonoff (1926–2009),
the discoverer and inventor of Universal Induction.
In 1964 Ray Solomonoff published the paper“A Formal Theory of Inductive Inference” . In this paper he proposed a universal method of inductive inference which employs the Bayesian framework and his newly created theory of algorithmic probability. This method of Solomonoff induction appears
to address the issues that plagued previous attempts at formalizing induction and has many promising properties and results. Solomonoff induction and related concepts are the central focus of this article.
The formalization of Solomonoff induction makes use of concepts and results from computer science, statistics, information theory, and philosophy. It is interesting that the development of a rigorous formalization of induction, which is fundamental to almost all scientific inquiry, is a highly multi-disciplinary undertaking, drawing from these various areas. Unfortunately this means that a high level of technical knowledge from these various disciplines is necessary to fully understand the technical content of Solomonoff induction. This has restricted a deep understanding of the concept to a fairly small proportion of academia which has hindered its discussion and hence progress.