anton/mlpractical
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #8 from pswietojanski/master

Browse Source 
some more clarifications
This commit is contained in:
Pawel Swietojanski 2015-09-28 17:10:14 +01:00
commit 8f36eb66ed
1 changed files with 58 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
00_Introduction.ipynb
View File

@ -36,19 +36,39 @@
"\n", "\n",
"## Instructions for DICE:\n", "## Instructions for DICE:\n",
"\n", "\n",
"### Configuring virtual environment (the generic way)\n", "### Directory structure and getting things organised\n",
"\n",
"To get things somehow standarized between people, and make life of everyone easier, we propse to organise your DICE setup in the following directory structure:\n",
"\n",
" * `~/mlpractical/` -- for a general course repository\n",
" * `~/mlpractical/repos-3rd` -- for stuff you download, build and install (numpy, OpenBlas, virtualenv)\n",
" * `~/mlpractical/repo-mlp` -- this is the actual course repository you clone from our website (do not create a dir for it yet!)\n",
" * `~/mlpractical/venv` -- this is where virutal envirnoment will make its dir (do not create a dir for it yet!)\n",
"\n",
"Create now repos-3rd directory (option -p in the below command will automatically create (non-existing) **p**arent directories (mlpractical):\n",
"\n",
" * `mkdir -p ~/mlpractical/repos-3rd`\n",
"\n",
"And now, let us set an MLP_WDIR environmental variable (MLP Working DIRectory) that denotes an absolute path of working dir pointing to `~/mlpractial`, **add the below line** to your `~/.bashrc` file (if it does not exists, create one!):\n",
"\n",
"```\n",
"export MLP_WDIR=~/mlpractical\n",
"```\n",
"\n",
"Now re-source `~/.bashrc` by typing (so the env variables get updated!): `source ~/.bashrc`\n",
"\n",
"Enter the `repos-3rd` directory by typing: `cd ~/mlpractical/repos-3rd` (or ```cd $MLP_WDIR/repos-3rd``` if you want)\n",
"\n",
"### Configuring virtual environment\n",
"\n",
"Make sure you are in `repos-3rd` directory and that MLP_WDIR variable has been exported (you may type export in the terminal and examine the list of availabe variables in the current session), then type:\n",
"\n", "\n",
" * `git clone https://github.com/pypa/virtualenv`\n", " * `git clone https://github.com/pypa/virtualenv`\n",
" * Enter the cloned repository and type `./virtualenv.py --python /usr/bin/python2.7 --no-site-packages ~/mlpractical`\n", " * Enter the cloned repository and type ```./virtualenv.py --python /usr/bin/python2.7 --no-site-packages $MLP_WDIR/venv```\n",
" * Activate the environment by typing `source ~/mlpractical/bin/activate` (to leave the virtual environment one may type `decativate`). Environments need to be activated every time ones start the new session (unless you do this explicitly in the shell starting scripts, i.e. `~/.bashrc`).\n", " * Activate the environment by typing `source ~/mlpractical/venv/bin/activate` (to leave the virtual environment one may type `decativate`)\n",
"\n", " * Environments need to be activated every time ones start the new session so add this command to your `~/.bashrc` script, by typing the below command (note, MLP_WDIR export needs to preceed this command and `.` is actually important):\n",
"### Configuring virtual environment (more comfy DICE wrapper; ignore if you did the generic way above)\n", " \n",
"\n", " ```. $MLP_WDIR/venv/bin/activate``` \n",
"DICE comes with a handy virtual environment wrapper, called $\\texttt{mkvirtualenv}$, which allows to simplify a bit the above process, to use it:\n",
"\n",
" * `source /usr/bin/virtualenvwrapper.sh` (add this also to `~/.bashrc` script so it is available automatically every time you ssh to the grid)\n",
" * Then type `mkvirtualenv mlpractical --python /usr/bin/python2.7` (this will create an environment under ~/.virtualenvs/mlpractical)\n",
" * To activate the environment you can use `workon` script that comes with the wrapper. Simply type: `workon mlpractical`\n",
"\n", "\n",
"### Install packages\n", "### Install packages\n",
"\n", "\n",
@ -76,10 +96,10 @@
"\n", "\n",
"#### OpenBlas\n", "#### OpenBlas\n",
"\n", "\n",
"To install OpenBlas library run (replace first line to be the path to the directory where you want to install OpenBlas):\n", "Enter again repos-3rd directory (`cd ~/mlpractical/repos-3rd) and copy into terminal the following commands (one at the time):\n",
"\n", "\n",
"```\n", "```\n",
"OBDir=~/OpenBlas\n", "OBDir=$MLP_WDIR/repos-3rd/OpenBLAS\n",
"git clone git://github.com/xianyi/OpenBLAS\n", "git clone git://github.com/xianyi/OpenBLAS\n",
"cd OpenBLAS\n", "cd OpenBLAS\n",
"make\n", "make\n",
@ -89,24 +109,26 @@
"Once OpenBLAS is finished compiling we need to ensure the compiled shared library files in the `lib` subdirectory are available to the shared library loader. This can be done by appending the absolute path to the `lib` subdirectory to the `LD_LIBRARY_PATH` environment variable. To ensure this changes persist we will change the bash start up file `~/.bashrc` by opening it in a text editor (e.g. by running `gedit ~/.bashrc`) and adding the following line\n", "Once OpenBLAS is finished compiling we need to ensure the compiled shared library files in the `lib` subdirectory are available to the shared library loader. This can be done by appending the absolute path to the `lib` subdirectory to the `LD_LIBRARY_PATH` environment variable. To ensure this changes persist we will change the bash start up file `~/.bashrc` by opening it in a text editor (e.g. by running `gedit ~/.bashrc`) and adding the following line\n",
"\n", "\n",
"```\n", "```\n",
"export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OBDir/lib\n", "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MLP_WDIR/repos-3rd/OpenBLAS/lib\n",
"```\n", "```\n",
"\n", "\n",
"Note, we again are using MLP_WDIR here, so the above line needs to be placed after you set MLP_WDIR.\n",
"\n",
"After you have edited `.bashrc` run\n", "After you have edited `.bashrc` run\n",
"\n", "\n",
"```\n", "```\n",
"source ~/.bashrc\n", "source ~/.bashrc\n",
"source ~/mlpractical/bin/activate\n", "source ~/mlpractical/venv/bin/activate\n",
"```\n", "```\n",
"\n", "\n",
"to rerun the bash start up script make sure the new environment variable is available in the current shell and then reactivate the virtual environment.\n", "to rerun the bash start up script make sure the new environment variable is available in the current shell and then reactivate the virtual environment.\n",
"\n", "\n",
"#### Numpy\n", "#### Numpy\n",
"\n", "\n",
"To install `numpy` linked against the OpenBLAS libraries we just compiled, first run the following\n", "To install `numpy` linked against the OpenBLAS libraries we just compiled, first run the following commands (one at a time)\n",
"\n", "\n",
"```\n", "```\n",
"cd ~/Downloads\n", "cd ~/mlpractical/repos-3rd/\n",
"wget http://downloads.sourceforge.net/project/numpy/NumPy/1.9.2/numpy-1.9.2.zip\n", "wget http://downloads.sourceforge.net/project/numpy/NumPy/1.9.2/numpy-1.9.2.zip\n",
"unzip numpy-1.9.2.zip\n", "unzip numpy-1.9.2.zip\n",
"cd numpy-1.9.2\n", "cd numpy-1.9.2\n",
@ -116,67 +138,39 @@
"python setup.py build --fcompiler=gnu95\n", "python setup.py build --fcompiler=gnu95\n",
"```\n", "```\n",
"\n", "\n",
"Assuming the virtual environment is activated, the below command will install numpy in a desired space (`~/.virtualenvs/mlpractical/...`):\n", "Assuming the virtual environment is activated, the below command will install numpy in a desired space (`~/mlpractical/venv/...`):\n",
"\n", "\n",
"```\n", "```\n",
"python setup.py install\n", "python setup.py install\n",
"```\n", "```\n",
"\n", "\n",
"Now use pip to install remaining packages: `scipy`, `matplotlib`, `argparse`, `nose`,\n",
"\n", "\n",
"### Installing remaining packages and running tests\n", "### Getting the mlpractical repository\n",
"\n", "\n",
"Use pip to install remaining packages: `scipy`, `matplotlib`, `argparse`, `nose`, and check if they pass the tests. An example for numpy is given below." "Clone the course repository from the github, by navigating to `~/mlpractical` directory and typing:\n",
"\n",
"```\n",
"git clone https://github.com/CSTR-Edinburgh/mlpractical.git repo-mlp\n",
"```\n",
"\n",
"When download is ready, enter the repo-mlp directory (`cd repo-mlp`) and start the actual interactive notebook session by typing:\n",
"\n",
"```\n",
"ipython notebook\n",
"```\n",
"\n",
"This should start a ipython server which opens a new browser window listing files in `repo-mlp` directory, including `00_Introduction.ipynb.`. Open it and run (from the browser interface) the following examples and exercies."
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": null,
"metadata": { "metadata": {
"collapsed": false, "collapsed": false,
"scrolled": true "scrolled": true
}, },
"outputs": [ "outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[H\u001b[2Jatlas_3_10_blas_threads_info:\n",
" libraries = ['tatlas']\n",
" library_dirs = ['/usr/lib64/atlas']\n",
" define_macros = [('HAVE_CBLAS', None), ('ATLAS_INFO', '\"\\\\\"3.10.1\\\\\"\"')]\n",
" language = c\n",
" include_dirs = ['/usr/include']\n",
"lapack_opt_info:\n",
" libraries = ['tatlas', 'tatlas', 'tatlas']\n",
" library_dirs = ['/usr/lib64/atlas']\n",
" define_macros = [('ATLAS_INFO', '\"\\\\\"3.10.1\\\\\"\"')]\n",
" language = f77\n",
" include_dirs = ['/usr/include']\n",
"blas_opt_info:\n",
" libraries = ['tatlas']\n",
" library_dirs = ['/usr/lib64/atlas']\n",
" define_macros = [('HAVE_CBLAS', None), ('ATLAS_INFO', '\"\\\\\"3.10.1\\\\\"\"')]\n",
" language = c\n",
" include_dirs = ['/usr/include']\n",
"openblas_info:\n",
" NOT AVAILABLE\n",
"openblas_lapack_info:\n",
" NOT AVAILABLE\n",
"atlas_3_10_threads_info:\n",
" libraries = ['tatlas', 'tatlas', 'tatlas']\n",
" library_dirs = ['/usr/lib64/atlas']\n",
" define_macros = [('ATLAS_INFO', '\"\\\\\"3.10.1\\\\\"\"')]\n",
" language = f77\n",
" include_dirs = ['/usr/include']\n",
"lapack_mkl_info:\n",
" NOT AVAILABLE\n",
"blas_mkl_info:\n",
" NOT AVAILABLE\n",
"mkl_info:\n",
" NOT AVAILABLE\n"
]
}
],
"source": [ "source": [
"%clear\n", "%clear\n",
"import numpy\n", "import numpy\n",
@ -383,7 +377,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython2", "pygments_lexer": "ipython2",
"version": "2.7.5" "version": "2.7.9"
} }
}, },
"nbformat": 4, "nbformat": 4,