The current state of open source AI

I recently discovered the company OpenAI, which has a significant endowment, and has a couple of notable projects that enable work on training / testing AI algorithms.  In particular, these projects are Gym (https://github.com/openai/gym) and Universe (https://github.com/openai/universe).  Gym is based on this paper (https://arxiv.org/abs/1606.01540) which is essentially a toolkit for benchmarking the performance of various AI algorithms in reinforcement learning.  From the project page:

gym makes no assumptions about the structure of your agent, and is compatible with any numerical computation library, such as TensorFlow or Theano.

Universe fits in with Gym, in that Gym provides the interface to Universe, which provides an AI algorithm with an environment within which to train.  An example of this is given on the project page, which essentially demonstrates use of Universe and Gym to test an agent running the game DuskDrive in a Docker container:

import gym
import universe  # register the universe environments

env = gym.make('flashgames.DuskDrive-v0')
env.configure(remotes=1)  # automatically creates a local docker container
observation_n = env.reset()

while True:
  action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n]  # your agent here
  observation_n, reward_n, done_n, info = env.step(action_n)
  env.render()

Evidently this is potentially quite powerful.  Indeed, from the main promotional page for universe (https://universe.openai.com/), apparently there are already more than 1000 environments to test / train learning agents against, which apparently many many more in the works.

To get started, one could use the openai code in the projects https://github.com/openai/improved-gan or https://github.com/openai/universe-starter-agent, however, those with a more sophisticated bent might instead be interested in investigating DeepMind’s project https://github.com/deepmind/learning-to-learn, which makes use of Google Tensorflow and DeepMind Sonnet (https://github.com/deepmind/sonnet).  Sonnet is a Tensorflow neural network library.

Towards increasingly unorthodox approaches, I think it would be fascinating to watch the space around the development of algorithms to take advantage of the architecture of IBM’s neuromorphic chips (based I believe on Field Programmable Gate Arrays), or looking a bit further out, towards the opportunities based in machine learning associated to Topological Quantum Computers (Majorana fermion based), and possibly same with a slight Qudit boost (hybrid/pseudo tetracomputer, maybe with switches consisting up to a 100 states).

I will continue to follow developments in this area with some interest.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: