====== Brain - Arcade Games ======

Download the most recent version of ALE (The Arcade Learning Environment) from http://www.arcadelearningenvironment.org/downloads/.

Extract the download ALE file into a directory, such as **/ale**.

Install dependencies

<code bash>
sudo apt-get install cmake
sudo apt-get install libsdl1.2-dev
</code>

Assuming ALE was extracted to /ale. Then, compile it with CMake.

<code bash>
cd /ale
ale> cmake -DUSE_SDL=ON -DUSE_RLGLUE=OFF -DBUILD_EXAMPLES=ON .
ale> make -j 4
</code>


Create a directory for the ROM files.

<code bash>
mkdir /ale/roms
</code>

and copy the binary rom files into that directory.

<WRAP info>
The DeepMind-Atari-Deep-Q-Learner-master.zip file at https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner contains **breakout.bin**.

Download this file too; or obtain roms from another source.
</WRAP>


Check that ALE has compiled correctly.

<code bash>
cd /ale
ale -help
</code>

<WRAP info>
This should display help tips
</WRAP>

Check the rom is able to be processed by ALE.

<code bash>
./ale rom/breakout.bin
</code>

This should display some details about the game.

READ the manual at /ale/doc/manual/manual.pdf

<code bash>
cd /ale/doc/examples
./sharedLibraryInterfaceExample
./sharedLibraryInterfaceExample breakout.bin
</code>

<code bash>
./ale -display_screen true roms/breakout.bin
./ale -display_screen true -game_controller fifo roms/breakout.bin
</code>


Create a test program.

<code bash>
vi test.cpp
g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale
./test
ls /etc/ld.so.conf
cat /etc/ld.so.conf
cat /etc/ld.so.conf.d/libc.conf
ls /usr/local/lib
cp libale.so /usr/local/lib
g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale
./test
ldconfig
./test
./test roms/breakout.bin
g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL
locate SDL.h
g++ -o test test.cpp src/ale_interface.cpp -I/ale/src -L/ale -lale -D__USE_SDL -I/usr/include/SDL
./test roms/breakout.bin
./ale -display_screen true -game_controller fifo roms/breakout.bin
</code>

===== Install ImageMagick and supporting libs =====

apt-get install imagemagick
apt-get install libmagick++-dev



===== Ideas =====

Brain learns from images, i.e. image snapshots, or differences between one image and another.

Images therefore need to be readable by the Brain.  Use jpg, png, bmp, own format libraries to do this.

Standardise all images to a standard size to standardise learning.  Do we therefore assume X*Y is pre-set for any image.  The larger that X and Y is will result in more pixels, so more training / longer training needed to process, but more accurate.

Perhaps different type neurons for different size images with some sort of standardisation comparison between unequal image sizes.

Images - Convolutions

update resolver's epsilon (chance of random action instead of optimal one)
# epsilon decreases over time
current_epsilon = 0.05 + 0.95 * np.exp(-epoch / 2500.)
resolver.epsilon.set_value(np.float32(current_epsilon))


# gamma - delayed reward coefficient - what fraction of reward is retained if it is obtained one tick later
gamma = theano.shared(np.float32(1), name='q_learning_gamma')


===== References =====

http://www.arcadelearningenvironment.org/

https://www.nervanasys.com/deep-reinforcement-learning-with-neon/

http://www.imagemagick.org/Magick++/tutorial/Magick++_tutorial.pdf