|
|
|
|
|
|
|
in the interaction between the adaptive system and its environment can be subsumed in the stochastic action of the operators. (See chapter 5 and section 7.2, however.) |
|
|
|
|
|
|
|
|
Much can be learned about adaptive plans in general by studying plans which act only in terms of payoff, plans for which |
|
|
|
|
|
|
|
|
In particular, plans which receive information in addition to payoff should do at least as well as plans which receive only payoff information. Thus, the efficiency of payoff-only plans with respect to e sets a nontrivial lower bound on the efficiency of other plans. |
|
|
|
|
|
|
|
|
To pose a problem in adaptation unambiguously one more element is required: a criterion  for comparing the efficiency of different plans  under the uncertainty represented by e. Such a criterion must of necessity be fairly sophisticated since it must somehow take into account the varying efficiency of a plan in different environments. Thus, even with a definite measure of efficiency such as the average rate of increase of payoff, there is still the problem of variations across the environments e. How is a plan which is highly efficient only in some subset of e to be compared with a plan which is moderately efficient in all the environments in e? It should be clear that the plan favored will often depend upon the particular application. In spite of this there are some broadly based criteria which have quite general applicability. The simplest of these requires that a plan accumulate payoff in each  more rapidly than an enumerative plan which has the same domain of action  . The intuitive content of this criterion is clear: A plan which does not accumulate payoff at least as rapidly as the extremely inefficient enumerative plans should, except in simple situations, be eliminated as a hypothesis (about natural systems) or an algorithm (for artificial systems). Because it is often useful to smooth out short-term variations in judging a plan, several broadly based criteria are stated in terms of the long-term average rate of payoff. When the adaptive plan has the deterministic form  , other, more general criteria are based on the cumulative payoff function |
|
|
|
|
|
|
|
|
 |
|
|
|
|
|
|
|
|
where  is the structure selected by t in E at time t, µE() is the corresponding payoff, and Ut,E(T) is the total payoff received by t in E to time T. (The average rate of payoff is just the function Ut,E(T)/T based on the cumulative payoff function Ut,E(T).) When the adaptive plan is stochastic,  , it is natural |
|
|
|
|
|
