CMSC421 Project Three: Learning

 

This project has two components.  Your tean (2-3 members) will create:

Each of these is described below.

 

The Agent/World Simulator

Your agent/world program simulates a world or agent of your choice.  It's core will be a function that you create that maps States and Actions to Rewards.  This function can be a deterministic mapping, or, preferably a probawill bilistic mapping.

The abstract representation you  will use is the following:

We will constrain everyone to use the same abstract representation so that learning agents will be able to interact with all of the simulators created by the class.

Part of your job is to come up with an interpretation for the states, actions and reward in your world.  Possibilities include:

*name: Boring Agent

State:
    mood: happy, sad, mad, bored
    physical: hungry, sleepy
    personality: optimist, pessimist

*Action:
    smile, hit, tell-joke, tickle
*State x Action -> Reward:
    s x a -> 0, i.e. no matter what the state is, no matter what the action is, the reward is 0 (another name for this agent might be 'the oblivious agent'!)

 

name: Agent-with-a-Personality

State:
    mood: happy, sad, mad, bored
    physical: hungry, sleepy
    personality: optimist, pessimist
*Action:
    smile, hit, tell-joke, tickle
*   
*State x Action -> Reward:
      mood = ? x physical <> sleepy, smile -> 5
      mood = ? x physical <> sleepy, tell-joke -> 10 w/ prob. .8,
                                                                     -10 w/ prob. .2
              etc.
*
*State x Action -> State
       s x a -> mood = happy if reward is positive
       s x a -> mood = mad if action is hit
                  etc.
*

name: Robot Agent

State:
    location

*Action:
    forward, back, left, right
*State x Action -> Reward:
     define rewards of states in your grid
*
*State x Action -> State
     defined by movements
*
 

 

The agent program will have an interface that will generate random examples.  There may be a uniform distribution over states, or you may wish to make some states more likely than others (perhaps there is a black hole, so states near the center are more likely than those around the edges).

However, the actions should be generated uniformly at random, so that the learning agent gets a chance see the rewards for different actions.

 The Learning Agent

*Calls Agent Program to get a training set
*Learns a function
*Calls Agent Program to get an evaluation set
*Computes Optimal set of actions
*Calls Agent Program to evaluate the set of actions
 

You can use whatever representation you like for the learning agent: neural networks, decisions trees, nearest neighbor, etc.,

Schedule

  • *Thursday, Dec. 4
    In class, hand in a 1 page description of your agent, and some notes on your learning approach
  • *Friday, Dec. 5
    Electronically submit your agent/world program by midnight
  • *Midnight Friday, Dec. 12 (Updated with Extension)
    Submit your learning agent
  • *7PM Wednesday, Dec. 17 (NO EXTENSIONS)
    Turn in a hardcopy of your writeup to Dr. Getoor's office, AVW 3217

Examples

 

 

 

 

*

 

Web Accessibility