The Game of Nim is a simple two player game. The rules of the game are indeed very simple, however, to play it well is difficult for the uninitiated.
Windy Walk (part 2) – addendum
Recap:
- Previously we constructed a very simple class to emulate the type of environment which is provided by OpenAI. Windy Walk (part 1)
- Then we implemented the same windy walk model as an extension to OpenAI. For this we created a custom python package, we named it gym_drifty_walk, you can grab it from github .
- These two versions we used, to produce exactly the same “Windy Walks” results, as shown by the plots.
I this addendum post we’ll take a look at the package code in gym_drifty_walk.
There are excellent articles about how to create a python package, I have no intention to duplicate those, I recommend python-packaging. One warning: the packaging article is written for python 2, so be aware of that.
There are things missing in this (like e.g. tests) that you would typically add. We choose to use the most basic approach that works, which we believe lowers the barrier to comprehension.
The most basic steps you need are these:
- ✓ Choose a package name
We have already done that: gym_drifty_walk - ✓ Follow the basic package structure
Here the structure is like this :
123456789gym_drifty_walk/├── gym_drifty_walk│ ├── envs│ │ ├── drifty_walk_env.py│ │ └── __init__.py│ └── __init__.py├── LICENSE.md├── README.md└── setup.py
(Remark; This tree-representation of the directory structure can be obtained using tree )
Windy Walk (part 2)
Recap: Previously we constructed a very simple class to emulate the type of environment which is provided by OpenAI. Then we simulated two windy walks and used the result of these walks to produce some plots.
Now we want to produce the same results again but via a different and more interesting route. This time we want a windy walk model that is implemented as an extension to OpenAI.
Windy Walk
- In the first post we introduce a very simple windy walk model. To make things as simple as possible, this model is implemented without using OpenAI
- in the second post this same windy walk model is implemented as an extension to OpenAI
In [22]:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
import numpy as np class env(): def __init__(self): # the walk starts at 0 self.state = 0 return None def reset(self): # the walk again restarts at 0 self.state = 0 return None def step(self, action): IntendedStepSize = action # IntendedStepSize is step the walker wants to take # on average we allow him to achieve that (so it's the average step size) # ... however we subject the walker to a random influence, the influence of the wind # represented by np.random.normal() windInfluence = np.random.normal() # the change in the walker's position is the sum of the # his IntendedStepSize and the windInfluence exerted by the environment self.state += IntendedStepSize + windInfluence return None |
1 2 3 4 5 6 7 8 9 10 11 12 |
## do walk with small IntendedStepSize: 0.1 # set random seed, for reproducibility np.random.seed(166) test = env() Walk1 = [] for i in range(50): test.step(0.1) Walk1.append(test.state) |
1 2 3 4 5 6 7 |
# import matplotlib for plotting import matplotlib.style as style import matplotlib.pyplot as plt style.use('ggplot') plt.plot(Walk1) |
1 2 3 4 5 6 7 8 9 10 11 12 |
## do walk with small IntendedStepSize: 1 # set random seed, for reproducibility np.random.seed(167) test.reset() Walk2 = [] for i in range(50): test.step(1) Walk2.append(test.state) |
1 2 3 4 5 6 7 8 9 10 |
# import matplotlib for plotting import matplotlib.style as style import matplotlib.pyplot as plt style.use('ggplot') plt.plot(Walk1, label='Walk1, IntendedStepSize 0.1') plt.plot(Walk2, label='Walk2, IntendedStepSize 1') plt.legend(loc='best') plt.show() |