|
|
|
|
|
|
|
The foregoing discussion can be made applicable to function optimization by so arranging it that a single "trial" of a policy 1A produces a sufficient estimate of its average performance  .For instance the policy could be repeatedly tried over some extended interval of time which would then be taken to be a single time-step in the discrete formulation. In any case let us assume that each  has a unique value  which can be determined (with sufficient accuracy) in a single time-step. Moving the problem of estimating  into the background in this way, reduces the control objective to finding the optimum of the function µE. |
|
|
|
|
|
|
|
|
If the elements of  are represented as points in an n-dimensional Euclidean space  the problem becomes one of optimizing an n-dimensional (nonlinear) real function. For example the elements of can be represented as strings of length n over some basic alphabet S. Since S can be recoded as a subset of {0, 1}m, where m is the first integer greater than log2 (card S), this can be looked upon as optimization of an n-dimensional function having m-place binary fractions as arguments. (Thus, if S= {s0, s1, s2, .. .}, the coding s0« .00. . .00, s1« .00. . . 01, s2 «00. . . 10, etc., can be used. Then with 1A Î a1 represented by the string s2s2s1, say, the argument of µEbecomes (.00 . . . 010, .00 . . . 010, .00 . . . 001).) |
|
|
|
|
|
|
|
|
With this arrangement an adaptive plan t uses its operators to generate a sequence of points (1), (2), (3), . . . converging to an optimum, much in the manner of standard iterative procedures. The adaptive approach, however, suggests important differences in what information from prior calculations should be retained (in  (t)) in preparation for generation of the next point  . In particular certain adaptive plans proceed simultaneously and efficiently with global and local optimization of µE. (See chapters 4 and 5 for basic techniques.) |
|
|
|
|
|
|
|
|
In the case of function optimization, high-dimensionality and nonlinearity of the function to be optimized (µE),in all but a few special cases, constitute insurmountable barriers to standard optimization algorithms. In the general control problem there is the added difficulty of nonstationarity. The schemata concept (first interpreted in function optimization terms in chapter 4, pp. 70-71) and the algorithms based upon it (chapter 6) provide specific remedies for the first two problems. The latter problem is substantial and difficultit is discussed in section 9.3. |
|
|
|
 |
|
|
|
|
Summarizing: |
|
|
|
|
|
|
|
|
 , [control] a set having as its basic component the set of admissible control policies augmented by a memory component and a set of time subscripts  so that  ; [function optimization] the domain of the function to be optimized. |
|
|
|
|
|