Setting up your Deep Learning Amazon Machine

While organising the Deep Learning Workshop #1, we wanted people to have a real experience of Deep Learning on GPU, without all the hassle of setting up everything.

So we made a public amazon image, which has all the necessary parts for you to start coding your deep learning algorithms directly, on an IPython Notebook.

This tutorial shows how to set up your amazon DL machine, https://github.com/deeplearningparis/dl-machine

Prerequisite

  • Having an AWS account with a valid credit card attached. You can create one in a few minutes.
  • Knowledge of Linux and ssh connections is a plus

That’s pretty much it to set up the machine!

Setting up the machine in AWS management Console

  • log in to AWS management console and select EC2 instances
  • select your region, (i.e. US-WEST N. California) region in top right menu
  • click on « Spot Request » on the leftmost menu and click « Request Spot Instances »
  • select community AMIs and search for ubuntu-14.04-hvm-deeplearning-paris
  • on the Choose instance Type tab, select GPU instances g2.2xlarge
  • bid a price larger than current price (e.g. $0.10, if it fails check the spot pricing history for that instance type)
  • in configure security group click Add Rule, and add a Custom TCP Rule with port Range 8888-8889 and from Anywhere
  • Review and launch, save the mykey.pem file

Once your machine is up (status : running in the online console), note the address to your instance : INSTANCE_ID.compute.amazonaws.com

Note: other regions with access to the deeplearning-paris image: Singapore, Ireland, North Virginia

Start using your instance

Using the notebooks

By default an IPython notebook server and an iTorch notebook server should be running on port 8888 and 8889 respectively. You need to open those ports in the Security Group of your instance if you have not done so yet.

To start using your instance, simply open the following URLs in your favorite browser:

SSH Connection to your instance

Once the instance is up, you might need to access directly your instance via SSH:

  • change the file mykey.pem accessibility:
chmod 400 mykey.pem
  • ssh to your instance
ssh -i mykey.pem ubuntu@INSTANCE_ID.compute.amazonaws.com

Deep Learning Tutorials

The amazon instance comes with a Deep Learning tutorial made by Gabriel Synnaeve.

Prerequisites

  • Strong background in Python
  • Good Mathematics (linear algebra and gradient derivation) knowledge
  • Knowledge of Machine Learning concepts

Tutorials

Connect to your IPython notebook and browse to the DL4H repository. Two tutorials are available:

  • DNN_notebook.ipynb
  • from_logistic_regression_to_deep_nets.ipynb

 

Further reading

The way we build the amazon image is by running two handmade scripts (located in the scripts directory of the dl-machine github):
ubuntu-14.04-cuda-6.5.sh installs CUDA (tested on a Ubuntu 14.04 amazon instance)
install-deeplearning-libraries.sh installs OpenBLAS, a python virtual environment with Numpy, Scipy, IPython, Matplotlib, Scikit-learn, Pandas, Theano & Torch.

Thanks to Olivier Grisel and Gabriel Synnaeve for their precious help and insights !

2 réponses à “Setting up your Deep Learning Amazon Machine

  1. Thanks for the instructions!
    If you want to make you spot instance persistant make sure to uncheck « Delete on Termination » that is checked by default. After you’re done with the spot, create an image and use it the next time you create a new spot request.

    J'aime

  2. Thanks for the instructions!
    If you want to make you spot instance persistant make sure to uncheck « Delete on Termination » that is checked by default. After you’re done with the spot, create an image and use it the next time you create a new spot request.

    J'aime

Laisser un commentaire

Entrez vos coordonnées ci-dessous ou cliquez sur une icône pour vous connecter:

Logo WordPress.com

Vous commentez à l'aide de votre compte WordPress.com. Déconnexion / Changer )

Image Twitter

Vous commentez à l'aide de votre compte Twitter. Déconnexion / Changer )

Photo Facebook

Vous commentez à l'aide de votre compte Facebook. Déconnexion / Changer )

Photo Google+

Vous commentez à l'aide de votre compte Google+. Déconnexion / Changer )

Connexion à %s