Hybridizing GP with Other Techniques

During this session, we broke into three groups, brainstormed, and then regrouped with the following three summaries:

Eric Siegel`s subgroup (loosely regarding classification problems) {Siegel}

- Global hillclimbing (via mutation) is simple to integrate.
- Brood selection [Tacket] is like crossover-based hillclimbing.
- Both mutation and crossover hillclimbing can use a cheaper, noisier fitness
  measure.

- GP hybridized with other techniques may not need as big a population.
- Chaining classifiers [Stolfo]/multistage modeling: NN -> NN been done, and
  GP -> GP could be done, and that starts to sound like encapsulation.
- Problem: adding complexity to learning system, eg hybrid, makes it harder
  to evaluate which parts of the system are making it work.

- Two types of hybrid: 1. During GP run and  2. pre/post-processing.
- It pays to spend more time on preprocessing input data (eg via hybrid) since
  GP will do well with predictive data.


Brij Masand's subgroup (regarding control problems) {Masand}

Breakout session: Brij Masand's subgroup (supposedly on control)
(Synergy between GP and machine learning
approaches to control problems  -- Neural nets, inducing decision
trees, simulated annealing) 

-- different kinds of control problems
        alife creatures, broom balancing, elevator control, process control
-- one-shot control vs. planning vs. real-time dynamic control

-- issue: representation: if there is specific appropriate
   representation such as  a neural network, an analogue circuit for
   control, schedules,   plans, then GP  can work on the
   representation directly  

-- GP population can be initially seeded by variants of successful 
   control systems in their representation (Neural nets, rules for
   decision trees, plans) and then GP can refine/enhance the solution
   to the control problem

-- issue: fitness:  Sometimes historical data is available for testing control
   but  for real-time   control problems (network management) how does 
   one evaluate fitness of a controller? It may not be always possible
   to use  an object in a safe environment  to test it.

-- GP can help discover/abstract new features for the control problem
   (from raw  sensor data) that can enhance a machine learning method
   being used for the actual control

-- GP can provide experience in simulating environments for estimating
   fitness for solutions to control problems   

-- synergy the other way:

If a GP run  is viewed as a "control problem" in terms of providing a
schedule for different parameters,  then knowledge of parameters can
help treat it as a control problem (for instance apply neural nets 
decision trees  to the parameters for optimizing them, abstract
patterns of successful runs and use it to control future runs -- meta
learning ).


Astro Teller's subgroup (loosely regarding prediction problems) {Teller}

   GP + NNs
        1. Cellular encoding is an example of such a synthesis
        2. Try learning NN's and then use them as GP primitives.
          (that is, if the NN takes 6 inputs and outputs 1 value,
           make a primitive NN-primitive(x,y,z,u,v,w))

   GP + Decision Trees
        1. Use GP to evolve alternatives to information gain
        2. Use a Decision Tree but put small GP trees in each of
                the leaves instead of simply returning the 
                majority classification at that leaf.
        3. 2 is a way of putting "expert knowledge" into your learning
                system.  It is debatable if this is really better than
                simply seeding your initial population with a GP tree
                with that expert knowledge.

   GP + Simulated annealing
        1. GP already has most of the benefits of simulated annealing
        2. Try doing simulated annealing on each of the gen 0 programs
                before starting the actual run.
        3. Simulated annealing as a genetic operator.


   GP + GA
        1. Use GA's instead of constants so the constants can evolve.