Skip to content
Snippets Groups Projects
Commit 32e39ea7 authored by Michael Blaschek's avatar Michael Blaschek :bicyclist:
Browse files

conda activate function revised!

parent d5baffd4
No related branches found
No related tags found
No related merge requests found
%% Cell type:markdown id: tags:
# Setting up a user environment
Topics
* Setup a user environment [using conda](#using-conda)
- [Add user environment to notebook kernels](#add-the-new-environment-to-your-active-kernels)
- [Activate your conda environment](#activate-your-conda-environment)
* Setup a user environment [using venv](#using-virtual-environment)
* Include `PATH` in [Notebook Kernel](#add-path-to-your-user-environment)
%% Cell type:code id: tags:
``` python
!module list
```
%% Output
Currently Loaded Modulefiles:
1) miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 
[?1l>
%% Cell type:markdown id: tags:
## Using conda
%% Cell type:code id: tags:
``` python
!conda info
```
%% Output
active environment : None
user config file : /jetfs/home/mblaschek/.condarc
populated config files : /jetfs/home/mblaschek/.condarc
conda version : 4.8.5
conda-build version : not installed
python version : 3.8.1.final.0
virtual packages : __glibc=2.28
base environment : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv (read only)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/pkgs
/jetfs/home/mblaschek/.conda/pkgs
envs directories : /jetfs/home/mblaschek/.conda/envs
/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/envs
platform : linux-64
user-agent : conda/4.8.5 requests/2.22.0 CPython/3.8.1 Linux/4.18.0-147.el8.x86_64 centos/8.2.2004 glibc/2.28
UID:GID : 54212:100
netrc file : None
offline mode : False
%% Cell type:code id: tags:
``` python
# Create a environment with conda
!conda create -y -n myenv python=3.7 numpy ipykernel
```
%% Output
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.8.5
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: /jetfs/home/mblaschek/.conda/envs/myenv
added / updated specs:
- ipykernel
- numpy
- python=3.7
The following packages will be downloaded:
package | build
---------------------------|-----------------
certifi-2020.11.8 | py37h06a4308_0 148 KB
ipython-7.19.0 | py37hb070fc8_0 978 KB
jupyter_core-4.7.0 | py37h06a4308_0 68 KB
numpy-1.19.2 | py37h54aff64_0 22 KB
numpy-base-1.19.2 | py37hfa32c7d_0 4.1 MB
pexpect-4.8.0 | pyhd3eb1b0_3 53 KB
pip-20.2.4 | py37h06a4308_0 1.7 MB
ptyprocess-0.6.0 | pyhd3eb1b0_2 20 KB
pygments-2.7.2 | pyhd3eb1b0_0 674 KB
setuptools-50.3.1 | py37h06a4308_1 711 KB
six-1.15.0 | py37h06a4308_0 27 KB
wheel-0.35.1 | pyhd3eb1b0_0 38 KB
------------------------------------------------------------
Total: 8.5 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
backcall pkgs/main/noarch::backcall-0.2.0-py_0
blas pkgs/main/linux-64::blas-1.0-mkl
ca-certificates pkgs/main/linux-64::ca-certificates-2020.10.14-0
certifi pkgs/main/linux-64::certifi-2020.11.8-py37h06a4308_0
decorator pkgs/main/noarch::decorator-4.4.2-py_0
intel-openmp pkgs/main/linux-64::intel-openmp-2020.2-254
ipykernel pkgs/main/linux-64::ipykernel-5.3.4-py37h5ca1d4c_0
ipython pkgs/main/linux-64::ipython-7.19.0-py37hb070fc8_0
ipython_genutils pkgs/main/linux-64::ipython_genutils-0.2.0-py37_0
jedi pkgs/main/linux-64::jedi-0.17.2-py37_0
jupyter_client pkgs/main/noarch::jupyter_client-6.1.7-py_0
jupyter_core pkgs/main/linux-64::jupyter_core-4.7.0-py37h06a4308_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
libedit pkgs/main/linux-64::libedit-3.1.20191231-h14c3975_1
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
libsodium pkgs/main/linux-64::libsodium-1.0.18-h7b6447c_0
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
mkl pkgs/main/linux-64::mkl-2020.2-256
mkl-service pkgs/main/linux-64::mkl-service-2.3.0-py37he904b0f_0
mkl_fft pkgs/main/linux-64::mkl_fft-1.2.0-py37h23d657b_0
mkl_random pkgs/main/linux-64::mkl_random-1.1.1-py37h0573a6f_0
ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1
numpy pkgs/main/linux-64::numpy-1.19.2-py37h54aff64_0
numpy-base pkgs/main/linux-64::numpy-base-1.19.2-py37hfa32c7d_0
openssl pkgs/main/linux-64::openssl-1.1.1h-h7b6447c_0
parso pkgs/main/noarch::parso-0.7.0-py_0
pexpect pkgs/main/noarch::pexpect-4.8.0-pyhd3eb1b0_3
pickleshare pkgs/main/linux-64::pickleshare-0.7.5-py37_1001
pip pkgs/main/linux-64::pip-20.2.4-py37h06a4308_0
prompt-toolkit pkgs/main/noarch::prompt-toolkit-3.0.8-py_0
ptyprocess pkgs/main/noarch::ptyprocess-0.6.0-pyhd3eb1b0_2
pygments pkgs/main/noarch::pygments-2.7.2-pyhd3eb1b0_0
python pkgs/main/linux-64::python-3.7.9-h7579374_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.1-py_0
pyzmq pkgs/main/linux-64::pyzmq-19.0.2-py37he6710b0_1
readline pkgs/main/linux-64::readline-8.0-h7b6447c_0
setuptools pkgs/main/linux-64::setuptools-50.3.1-py37h06a4308_1
six pkgs/main/linux-64::six-1.15.0-py37h06a4308_0
sqlite pkgs/main/linux-64::sqlite-3.33.0-h62c20be_0
tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0
tornado pkgs/main/linux-64::tornado-6.0.4-py37h7b6447c_1
traitlets pkgs/main/noarch::traitlets-5.0.5-py_0
wcwidth pkgs/main/noarch::wcwidth-0.2.5-py_0
wheel pkgs/main/noarch::wheel-0.35.1-pyhd3eb1b0_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
zeromq pkgs/main/linux-64::zeromq-4.3.3-he6710b0_3
zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Downloading and Extracting Packages
numpy-1.19.2 | 22 KB | ##################################### | 100%
pip-20.2.4 | 1.7 MB | ##################################### | 100%
numpy-base-1.19.2 | 4.1 MB | ##################################### | 100%
certifi-2020.11.8 | 148 KB | ##################################### | 100%
wheel-0.35.1 | 38 KB | ##################################### | 100%
ipython-7.19.0 | 978 KB | ##################################### | 100%
pexpect-4.8.0 | 53 KB | ##################################### | 100%
pygments-2.7.2 | 674 KB | ##################################### | 100%
ptyprocess-0.6.0 | 20 KB | ##################################### | 100%
jupyter_core-4.7.0 | 68 KB | ##################################### | 100%
setuptools-50.3.1 | 711 KB | ##################################### | 100%
six-1.15.0 | 27 KB | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate myenv
#
# To deactivate an active environment, use
#
# $ conda deactivate
%% Cell type:markdown id: tags:
### Add the new environment to your active kernels
This way you can choose the kernel from the Jupyterlab launcher.
%% Cell type:code id: tags:
``` python
!~/.conda/envs/myenv/bin/python -m ipykernel install --user --name MYENV --display-name "Py3.7"
```
%% Output
Installed kernelspec MYENV in /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
%% Cell type:markdown id: tags:
### Activate your conda environment
Conda Environments need to be added to the `PATH` to work well. So there is an `activate` function for these conda environments.
However, running
```bash
conda init bash
```
will edit your `.bashrc` and that is not nice, especially when you might move to another module.
**Therefore this is not recommended to do.**
In order to use your environment in a shell script or on a slurm job, try the following:
This script is used by `conda init bash` to set the configuration, we simply source (load) it and get the same functionality as we would be using `conda init bash`. **(Do not use `conda init bash`)**
Load the shell setup script located in the conda source tree (`${CONDA_ROOT}/etc/profile.d/conda.sh`) and activate your environment:
```bash
# Load from the current conda module the BASH functionality (as in activate)
source $(which conda)/../etc/profile.d/conda.sh
source $(dirname $(which conda))/../etc/profile.d/conda.sh
# now activating the environment works
conda activate myenv
```
#### bash conda_activate function
optionaly you can add this bash function to your `.bashrc` for convenience:
```bash
conda_activate(){
if ! command -v conda &> /dev/null; then
echo "Load ana/miniconda module first"
else
if [ $# -ne 1 ]; then
echo "usage: conda_activate [env-name]"
else
source $(dirname $(which conda))/../etc/profile.d/conda.sh
conda activate $1
fi
fi
}
```
After your `.bashrc` has been loaded (e.g. logout/login again) you should be able to use
```bash
module load anaconda3
conda_activate myenv
```
%% Cell type:code id: tags:
``` python
# List available Kernels
!jupyter kernelspec list
```
%% Output
Available kernels:
myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3
%% Cell type:markdown id: tags:
## Using virtual environment
%% Cell type:code id: tags:
``` python
# Setup a virtual Environment for your Python in the directoy: myfancyenv
!python -m venv myfancyenv
```
%% Cell type:code id: tags:
``` python
# Creates a directory with new environment
!ls myfancyenv/
```
%% Output
bin include lib lib64 pyvenv.cfg
%% Cell type:code id: tags:
``` python
# Install some package into that environment
# important is ipykernel to use it within jupyterlab/notebook
!./myfancyenv/bin/pip -q install numpy scipy pandas ipykernel
```
%% Output
WARNING: You are using pip version 19.2.3, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
%% Cell type:code id: tags:
``` python
# Allow this environment to be available on the LAUNCHERS (+) sign on the left
!./myfancyenv/bin/python -m ipykernel install --user --name fancy --display-name "My Python"
```
%% Output
Installed kernelspec fancy in /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy
%% Cell type:markdown id: tags:
Wait a bit and the new python environment should be available on your launchers.
%% Cell type:code id: tags:
``` python
!jupyter kernelspec list
```
%% Output
Available kernels:
fancy /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy
myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3
%% Cell type:markdown id: tags:
## Add PATH to your user environment
In order to get full access to all tools installed in your local environment, add it to your `PATH`.
Open your `kernel.json` file located `/jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv/kernel.json`:
```json
{
"argv": [
"/jetfs/home/mblaschek/.conda/envs/myenv/bin/python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"env" : {
"PATH": "/jetfs/home/mblaschek/.conda/envs/myenv/bin/:${PATH}",
"LD_LIBRARY_PATH": "/jetfs/home/mblaschek/.conda/envs/myenv/lib:${LD_LIBRARY_PATH}"
},
"display_name": "MYENV",
"language": "python"
}
```
%% Cell type:markdown id: tags:
### Finished
......
%% Cell type:markdown id: tags:
# Welcome to Jet
<img src='https://homepage.univie.ac.at/michael.blaschek/openfiles/rocket.png' width='100px'><img src='https://homepage.univie.ac.at/michael.blaschek/openfiles/uni.jpg' width='400px'>
You have done it! Your Notebook Server is running on the Jet Cluster.
Welcome, please skip this, if your know where you are, as this is meant as an introduction. However, not complete as it is.
If you need more infos about Notebooks have a look at [my Homepage](https://homepage.univie.ac.at/michael.blaschek/#python) and there are numerous Tutorials on the Internet for any advice, but feel free to ask me and give me feedback.
Help can be found here:
* Goto Help -> JupyterLab Reference
* Goto Help -> Jupyter Reference
* [Tour](https://jupyterlab.readthedocs.io/en/stable/user/interface.html)
In the following I will show you some useful things.
First: This Notebook is in your Home directory, but also on `/jetfs/scratch` for any reason.
%% Cell type:markdown id: tags:
## Check your current Host
%% Cell type:code id: tags:
``` python
!hostnamectl
```
%% Output
Static hostname: localhost.localdomain
Transient hostname: jet03.jet.local
Icon name: computer-server
Chassis: server
Operating System: ]8;;https://www.centos.org/CentOS Linux 8 (Core)]8;;
CPE OS Name: cpe:/o:centos:centos:8
Kernel: Linux 4.18.0-147.el8.x86_64
Architecture: x86-64
%% Cell type:markdown id: tags:
## Who is running this process
%% Cell type:code id: tags:
``` python
!whoami
```
%% Output
mblaschek
%% Cell type:code id: tags:
``` python
# Where are you
!pwd
```
%% Output
/jetfs/home/mblaschek/Documents
%% Cell type:code id: tags:
``` python
# Show my Current Jobs in the Queue
!squeue -u $(id -u) -l
```
%% Output
Fri Oct 16 23:10:58 2020
JOBID PARTITION NAME USER STATE TIME TIME_LIMI NODES NODELIST(REASON)
235 compute jupyters mblasche RUNNING 6:29:07 8:00:00 1 jet03
%% Cell type:markdown id: tags:
There you can see your `TIME_LIMIT` of 8 hours again and how much time has passed on your current jobs
%% Cell type:code id: tags:
``` python
# Import sys and check the executable (so what python you are using)
import sys
print(sys.executable)
```
%% Output
/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/bin/python
%% Cell type:markdown id: tags:
**Notice the funny path?**
Yes this is SPACK your nice scientific package manager.
Allows to build custom libraries and applications tailored to our needs!
Please not that software currently installed in /opt on JET01 is not available here.
Software under /opt is deprecated and will be removed after a full transition
to the current libraries
More information on [SPACK](https://spack.io/)
## Check your environmental PATH Variable, could be useful to know
%% Cell type:code id: tags:
``` python
%env PATH
```
%% Output
'/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/bin:/bin:/jetfs/spack/bin:/jetfs/userservices:/opt/jupyterhub/bin:/opt/slurm/bin:/sbin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/share/Modules/bin::/jetfs/home/mblaschek/bin:/jetfs/home/mblaschek/.local/bin'
%% Cell type:markdown id: tags:
## Check what modules are currently loaded in this Notebook session
%% Cell type:code id: tags:
``` python
!module list --no-pager
```
%% Output
Currently Loaded Modulefiles:
1) miniconda3/4.8.2-gcc-8.3.1-3m7b6t2
%% Cell type:markdown id: tags:
Put this file `load_modules_into_jupyter.conf` into your Home directory and your Notebook environment will have these modules loaded.
It is not possible to load these modules via magic (python magic ;).
**It needs the empty line at the end...**
%% Cell type:code id: tags:
``` python
%%writefile ~/load_modules_into_jupyter.conf
eccodes/2.18.0-gcc-8.3.1-s7clum3
```
%% Output
Writing /jetfs/home/mblaschek/load_modules_into_jupyter.conf
%% Cell type:code id: tags:
``` python
!cat ~/load_modules_into_jupyter.conf
```
%% Output
eccodes/2.18.0-gcc-8.3.1-s7clum3
%% Cell type:markdown id: tags:
**Restarting the kernel will show now loaded modules**
%% Cell type:code id: tags:
``` python
# after restarting
!module list --no-pager
```
%% Output
Currently Loaded Modulefiles:
1) miniconda3/4.8.2-gcc-8.3.1-3m7b6t2
2) openmpi/4.0.5-gcc-8.3.1-773ztsv
3) hdf5/1.12.0-gcc-8.3.1-awl4atl
4) parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
5) netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
6) eccodes/2.18.0-gcc-8.3.1-s7clum3
%% Cell type:markdown id: tags:
## Currently available Modules
%% Cell type:code id: tags:
``` python
# --no-pager is needed only inside notebook
!module avail --no-pager
```
%% Output
--------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ----------
anaconda2/2019.10-gcc-8.3.1-5pou6ji
anaconda3/2019.10-gcc-8.3.1-tmy5mgp
anaconda3/2020.07-gcc-8.3.1-weugqkf
cdo/1.9.8-gcc-8.3.1-ipgvzeh
eccodes/2.18.0-gcc-8.3.1-s7clum3
hdf5/1.12.0-gcc-8.3.1-awl4atl
intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw
miniconda2/4.7.12.1-gcc-8.3.1-zduqggv
miniconda3/4.8.2-gcc-8.3.1-3m7b6t2
netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj
openmpi/3.1.6-gcc-8.3.1-rk5av53
openmpi/4.0.5-gcc-8.3.1-773ztsv
parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu
parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
--------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ----------
anaconda2/2019.10-gcc-8.3.1-5pou6ji
anaconda3/2019.10-gcc-8.3.1-tmy5mgp
anaconda3/2020.07-gcc-8.3.1-weugqkf
cdo/1.9.8-gcc-8.3.1-ipgvzeh
eccodes/2.18.0-gcc-8.3.1-s7clum3
hdf5/1.12.0-gcc-8.3.1-awl4atl
intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw
miniconda2/4.7.12.1-gcc-8.3.1-zduqggv
miniconda3/4.8.2-gcc-8.3.1-3m7b6t2
netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj
openmpi/3.1.6-gcc-8.3.1-rk5av53
openmpi/4.0.5-gcc-8.3.1-773ztsv
parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu
parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
%% Cell type:markdown id: tags:
# Software on JET
Currently:
- [x] Anaconda (2,3)
- [x] Miniconda (2,3)
- [x] OPENMPI (4.0.5, 3.6.1) optimized for JET
- [x] HDF5 (1.12) optimized with OPENMPI
- [x] NetCDF (Parallel (NC3), NC4->HDF5)
- [x] Eccodes (2.18)
- [x] CDO
- [x] Intel compiled Libraries (License)
- [x] Emoslib
- [x] Ecaccess
- [ ] RTTOV
%% Cell type:markdown id: tags:
## Currently there are a few choices:
1. Python (2, 3)
* Anaconda
* Miniconda
2. Julia (not yet)
3. R (not yet)
%% Cell type:markdown id: tags:
## Install Python packages
%% Cell type:code id: tags:
``` python
# Please use the --user flag to install the packages into .local/lib/python...
%pip install --user numpy scipy
```
%% Output
Requirement already satisfied: numpy in /jetfs/home/mblaschek/.local/lib/python3.8/site-packages (1.19.2)
Requirement already satisfied: scipy in /jetfs/home/mblaschek/.local/lib/python3.8/site-packages (1.5.2)
Note: you may need to restart the kernel to use updated packages.
%% Cell type:code id: tags:
``` python
import os
import sys
import site
# Need to update search Path / only necessary here for the first time
# sys.path.append(os.path.expandvars("$HOME/.local/lib/python3.8/site-packages"))
sys.path.append(site.getusersitepackages())
```
%% Cell type:code id: tags:
``` python
import numpy as np
import scipy as sp
# WORKS! ;)
```
%% Cell type:markdown id: tags:
## Using conda
%% Cell type:code id: tags:
``` python
!conda info
```
%% Output
active environment : None
user config file : /jetfs/home/mblaschek/.condarc
populated config files : /jetfs/home/mblaschek/.condarc
conda version : 4.8.5
conda-build version : not installed
python version : 3.8.1.final.0
virtual packages : __glibc=2.28
base environment : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv (read only)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/pkgs
/jetfs/home/mblaschek/.conda/pkgs
envs directories : /jetfs/home/mblaschek/.conda/envs
/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/envs
platform : linux-64
user-agent : conda/4.8.5 requests/2.22.0 CPython/3.8.1 Linux/4.18.0-147.el8.x86_64 centos/8.2.2004 glibc/2.28
UID:GID : 54212:100
netrc file : None
offline mode : False
%% Cell type:code id: tags:
``` python
# Create a environment with conda
!conda create -y -n myenv python=3.7 numpy ipykernel
```
%% Output
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /jetfs/home/mblaschek/.conda/envs/myenv
added / updated specs:
- ipykernel
- numpy
- python=3.7
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
backcall pkgs/main/noarch::backcall-0.2.0-py_0
blas pkgs/main/linux-64::blas-1.0-mkl
ca-certificates pkgs/main/linux-64::ca-certificates-2020.10.14-0
certifi pkgs/main/linux-64::certifi-2020.6.20-py37_0
decorator pkgs/main/noarch::decorator-4.4.2-py_0
intel-openmp pkgs/main/linux-64::intel-openmp-2020.2-254
ipykernel pkgs/main/linux-64::ipykernel-5.3.4-py37h5ca1d4c_0
ipython pkgs/main/linux-64::ipython-7.18.1-py37h5ca1d4c_0
ipython_genutils pkgs/main/linux-64::ipython_genutils-0.2.0-py37_0
jedi pkgs/main/linux-64::jedi-0.17.2-py37_0
jupyter_client pkgs/main/noarch::jupyter_client-6.1.7-py_0
jupyter_core pkgs/main/linux-64::jupyter_core-4.6.3-py37_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
libedit pkgs/main/linux-64::libedit-3.1.20191231-h14c3975_1
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
libsodium pkgs/main/linux-64::libsodium-1.0.18-h7b6447c_0
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
mkl pkgs/main/linux-64::mkl-2020.2-256
mkl-service pkgs/main/linux-64::mkl-service-2.3.0-py37he904b0f_0
mkl_fft pkgs/main/linux-64::mkl_fft-1.2.0-py37h23d657b_0
mkl_random pkgs/main/linux-64::mkl_random-1.1.1-py37h0573a6f_0
ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1
numpy pkgs/main/linux-64::numpy-1.19.1-py37hbc911f0_0
numpy-base pkgs/main/linux-64::numpy-base-1.19.1-py37hfa32c7d_0
openssl pkgs/main/linux-64::openssl-1.1.1h-h7b6447c_0
parso pkgs/main/noarch::parso-0.7.0-py_0
pexpect pkgs/main/linux-64::pexpect-4.8.0-py37_1
pickleshare pkgs/main/linux-64::pickleshare-0.7.5-py37_1001
pip pkgs/main/linux-64::pip-20.2.3-py37_0
prompt-toolkit pkgs/main/noarch::prompt-toolkit-3.0.8-py_0
ptyprocess pkgs/main/linux-64::ptyprocess-0.6.0-py37_0
pygments pkgs/main/noarch::pygments-2.7.1-py_0
python pkgs/main/linux-64::python-3.7.9-h7579374_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.1-py_0
pyzmq pkgs/main/linux-64::pyzmq-19.0.2-py37he6710b0_1
readline pkgs/main/linux-64::readline-8.0-h7b6447c_0
setuptools pkgs/main/linux-64::setuptools-50.3.0-py37hb0f4dca_1
six pkgs/main/noarch::six-1.15.0-py_0
sqlite pkgs/main/linux-64::sqlite-3.33.0-h62c20be_0
tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0
tornado pkgs/main/linux-64::tornado-6.0.4-py37h7b6447c_1
traitlets pkgs/main/noarch::traitlets-5.0.5-py_0
wcwidth pkgs/main/noarch::wcwidth-0.2.5-py_0
wheel pkgs/main/noarch::wheel-0.35.1-py_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
zeromq pkgs/main/linux-64::zeromq-4.3.3-he6710b0_3
zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate myenv
#
# To deactivate an active environment, use
#
# $ conda deactivate
%% Cell type:markdown id: tags:
## Activating a conda environment
It is not recommended to use `conda init bash`, because it will write that specific version into your `.bashrc` file and will result in errors when you move to another module or newer version.
Hint: Checkout the guides on [Gitlab](https://gitlab.phaidra.org/imgw/computer-resources/-/blob/master/Python/QA-003-Conda-Environment.ipynb#Activate-your-conda-environment)
### Load from the current conda module the BASH functionality (as in activate)
### Load bash conda configuration
This script is used by `conda init bash` to set the configuration, we simply source (load) it and get the same functionality as we would be using `conda init bash`. **(Do not use `conda init bash`)**
```sh
source $(which conda)/../etc/profile.d/conda.sh
```bash
source $(dirname $(which conda))/../etc/profile.d/conda.sh
# now activating the environment works
conda activate myenv
```
optionaly you can add this bash function to your `.bashrc` for convenience:
```bash
conda_activate(){
if ! command -v conda &> /dev/null; then
echo "Load ana/miniconda module first"
else
if [ $# -ne 1 ]; then
echo "usage: conda_activate [env-name]"
else
source $(dirname $(which conda))/../etc/profile.d/conda.sh
conda activate $1
fi
fi
}
```
After your `.bashrc` has been loaded (e.g. logout/login again) you should be able to use
```bash
module load anaconda3
conda_activate myenv
```
### Activate your kernel in your notebooks
Please run the follwoing command and install the ipykernel, if it is not yet installed:
%% Cell type:code id: tags:
``` python
!~/.conda/envs/myenv/bin/python -m ipykernel install --user --name MYENV --display-name "MYCONDA"
```
%% Output
Installed kernelspec MYENV in /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
%% Cell type:code id: tags:
``` python
!jupyter kernelspec list
```
%% Output
Available kernels:
myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3
%% Cell type:markdown id: tags:
## Custom Environment Python Environment
%% Cell type:code id: tags:
``` python
# Setup a virtual Environment for your Python in the directoy: myfancyenv
!python -m venv myfancyenv
```
%% Cell type:code id: tags:
``` python
# Install some package into that environment
# important is ipykernel to use it within jupyterlab/notebook
!./myfancyenv/bin/pip -q install numpy scipy pandas ipykernel
```
%% Output
WARNING: You are using pip version 19.2.3, however version 20.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
%% Cell type:code id: tags:
``` python
# Allow this environment to be available on the LAUNCHERS (+) sign on the left
!./myfancyenv/bin/python -m ipykernel install --user --name fancy --display-name "My Python"
```
%% Output
Installed kernelspec fancy in /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy
%% Cell type:markdown id: tags:
Wait a bit and the new python environment should be available on your launchers.
%% Cell type:code id: tags:
``` python
!jupyter kernelspec list
```
%% Output
Available kernels:
fancy /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy
myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv
python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3
%% Cell type:markdown id: tags:
There you go. Your personal Environment.
# If you need help. Feel free to ask me. Enjoy!
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment