Download the most recent version of ALE (The Arcade Learning Environment) from http://www.arcadelearningenvironment.org/downloads/.
Extract the download ALE file into a directory, such as /ale.
Install dependencies
sudo apt-get install cmake sudo apt-get install libsdl1.2-dev
Assuming ALE was extracted to /ale. Then, compile it with CMake.
cd /ale ale> cmake -DUSE_SDL=ON -DUSE_RLGLUE=OFF -DBUILD_EXAMPLES=ON . ale> make -j 4
Create a directory for the ROM files.
mkdir /ale/roms
and copy the binary rom files into that directory.
The DeepMind-Atari-Deep-Q-Learner-master.zip file at https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner contains breakout.bin.
Download this file too; or obtain roms from another source.
Check that ALE has compiled correctly.
cd /ale ale -help
This should display help tips
Check the rom is able to be processed by ALE.
./ale rom/breakout.bin
This should display some details about the game.
READ the manual at /ale/doc/manual/manual.pdf
cd /ale/doc/examples ./sharedLibraryInterfaceExample ./sharedLibraryInterfaceExample breakout.bin
./ale -display_screen true roms/breakout.bin ./ale -display_screen true -game_controller fifo roms/breakout.bin
Create a test program.
vi test.cpp g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale ./test ls /etc/ld.so.conf cat /etc/ld.so.conf cat /etc/ld.so.conf.d/libc.conf ls /usr/local/lib cp libale.so /usr/local/lib g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale ./test ldconfig ./test ./test roms/breakout.bin g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL locate SDL.h g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL -I/usr/include/SDL ./test roms/breakout.bin ./ale -display_screen true -game_controller fifo roms/breakout.bin
apt-get install imagemagick apt-get install libmagick++-dev
Brain learns from images, i.e. image snapshots, or differences between one image and another.
Images therefore need to be readable by the Brain. Use jpg, png, bmp, own format libraries to do this.
Standardise all images to a standard size to standardise learning. Do we therefore assume X*Y is pre-set for any image. The larger that X and Y is will result in more pixels, so more training / longer training needed to process, but more accurate.
Perhaps different type neurons for different size images with some sort of standardisation comparison between unequal image sizes.
Images - Convolutions
update resolver's epsilon (chance of random action instead of optimal one) # epsilon decreases over time current_epsilon = 0.05 + 0.95 * np.exp(-epoch / 2500.) resolver.epsilon.set_value(np.float32(current_epsilon))
# gamma - delayed reward coefficient - what fraction of reward is retained if it is obtained one tick later gamma = theano.shared(np.float32(1), name='q_learning_gamma')