Skip to content
Snippets Groups Projects
Commit 2474487c authored by Marty Kandes's avatar Marty Kandes
Browse files

Create a separate MLNX_OFED definition file (layer) apart from MPI

Until now, all base OS + MPI naked-singularity definition files also
included the Mellanox OFED drivers, libraries, and other dependencies
required to run MPI-based applications from the containers over
Infiniband (IB) on SDSC HPC systems. In order to provide additional
flexability and modularity to the naked-singularity definition file
dependency chain, we will now seperate out the installation of the
MLNX_OFED distribution from any MPI distributioni moving forward. i.e.,
there will be a definition file (layer) for each OS + MLNX_OFED
combination we choose to support.
parent 8a24e734
No related branches found
No related tags found
No related merge requests found
...@@ -31,11 +31,14 @@ hosted at Singularity Hub. ...@@ -31,11 +31,14 @@ hosted at Singularity Hub.
singularity pull shub://mkandes/naked-singularity:ubuntu-18.04 singularity pull shub://mkandes/naked-singularity:ubuntu-18.04
``` ```
[Singularity Hub has been archived](https://vsoch.github.io/2021/singularity-hub-archive). At least for the time being, naked-singularity definition IMPORTANT: [Singularity Hub has been archived](https://vsoch.github.io/2021/singularity-hub-archive).
files that rely on containers that were built and hosted on Singularity For the time being, naked-singularity definition files that rely on
Hub prior to it being archived will continue to pull in these container containers that were built and hosted on Singularity Hub prior to it
dependencies and build properly. Alternative container build and hosting being archived will continue to pull in these container dependencies and
options for all future work are still under consideration. build properly. Note, however, new pre-built containers of the latest
naked-singularity definition files are currently being updated to be
hosted via the [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry).
## Status ## Status
...@@ -57,8 +60,8 @@ University of California, San Diego ...@@ -57,8 +60,8 @@ University of California, San Diego
## Version ## Version
1.7.9 1.8.1
## Last Updated ## Last Updated
Sunday, August 1st, 2021 Wednesday, August 4th, 2021
Bootstrap: oras
From: ghcr.io/mkandes/naked-singularity:ubuntu-18.04-mlnx-ofed-4.7-3.2.9.0-openmpi-3.1.6
%labels
APPLICATION_NAME OSU Micro-Benchmarks
APPLICATION_VERSION 5.7
APPLICATION_URL http://mvapich.cse.ohio-state.edu/benchmarks
AUTHOR_NAME Marty Kandes
AUTHOR_EMAIL mkandes@sdsc.edu
LAST_UPDATED 20210804
%setup
%environment
# Set OMB major and minor version numbers, root and
# installation directories
export OMB_ROOT_URL='http://mvapich.cse.ohio-state.edu/download/mvapich'
export OMB_MAJOR='5'
export OMB_MINOR='7'
export OMB_VERSION="${OMB_MAJOR}.${OMB_MINOR}"
export OMB_ROOT_DIR='/opt/omb'
export OMB_INSTALL_DIR="${OMB_ROOT_DIR}/${OMB_VERSION}"
# Set paths to OSU Microbenchmarks binaries
export PATH="${OMB_INSTALL_DIR}/collective:${PATH}"
export PATH="${OMB_INSTALL_DIR}/one-sided:${PATH}"
export PATH="${OMB_INSTALL_DIR}/pt2pt:${PATH}"
export PATH="${OMB_INSTALL_DIR}/startup:${PATH}"
%post -c /bin/bash
# Set operating system mirror URL
export MIRRORURL='http://us.archive.ubuntu.com/ubuntu'
# Set operating system version
export OSVERSION='bionic'
# Set system locale
export LC_ALL='C'
# Set debian frontend interface
export DEBIAN_FRONTEND='noninteractive'
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='3'
export OMPI_MINOR='1'
export OMPI_REVISION='6'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
# Set paths to OpenMPI binaries and libraries
export PATH="${OMPI_INSTALL_DIR}/bin:${PATH}"
export LD_LIBRARY_PATH="${OMPI_INSTALL_DIR}/lib:${LD_LIBRARY_PATH}"
# Upgrade all software packages to their latest versions
apt-get -y update && apt-get -y upgrade
cd /tmp
# Set OMB major and minor version numbers, root and
# installation directories
export OMB_ROOT_URL='http://mvapich.cse.ohio-state.edu/download/mvapich'
export OMB_MAJOR='5'
export OMB_MINOR='7'
export OMB_VERSION="${OMB_MAJOR}.${OMB_MINOR}"
export OMB_ROOT_DIR='/opt/omb'
export OMB_INSTALL_DIR="${OMB_ROOT_DIR}/${OMB_VERSION}"
# Download, build, and install OSU Microbenchmarks
wget "${OMB_ROOT_URL}/osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
tar -xf "osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
cd "osu-micro-benchmarks-${OMB_VERSION}"
./configure --prefix="${OMB_INSTALL_DIR}" CC='mpicc' CXX='mpicxx'
make
make install
# Reorganize OSU Microbenchmarks directory structure
cd "${OMB_INSTALL_DIR}"
mv libexec/osu-micro-benchmarks/mpi/collective ./
mv libexec/osu-micro-benchmarks/mpi/one-sided ./
mv libexec/osu-micro-benchmarks/mpi/pt2pt ./
mv libexec/osu-micro-benchmarks/mpi/startup ./
rm -rf libexec
cd /tmp
# Remove OMB build directory and source tarball
rm -rf "osu-micro-benchmarks-${OMB_VERSION}"
rm "osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
# Cleanup
apt-get -y autoremove --purge
apt-get -y clean
# Update database for mlocate
updatedb
%files
%runscript
%test
Bootstrap: oras
From: ghcr.io/mkandes/naked-singularity:ubuntu-18.04-mlnx-ofed-4.7-3.2.9.0-openmpi-4.0.5
%labels
APPLICATION_NAME OSU Micro-Benchmarks
APPLICATION_VERSION 5.7
APPLICATION_URL http://mvapich.cse.ohio-state.edu/benchmarks
AUTHOR_NAME Marty Kandes
AUTHOR_EMAIL mkandes@sdsc.edu
LAST_UPDATED 20210804
%setup
%environment
# Set OMB major and minor version numbers, root and
# installation directories
export OMB_ROOT_URL='http://mvapich.cse.ohio-state.edu/download/mvapich'
export OMB_MAJOR='5'
export OMB_MINOR='7'
export OMB_VERSION="${OMB_MAJOR}.${OMB_MINOR}"
export OMB_ROOT_DIR='/opt/omb'
export OMB_INSTALL_DIR="${OMB_ROOT_DIR}/${OMB_VERSION}"
# Set paths to OSU Microbenchmarks binaries
export PATH="${OMB_INSTALL_DIR}/collective:${PATH}"
export PATH="${OMB_INSTALL_DIR}/one-sided:${PATH}"
export PATH="${OMB_INSTALL_DIR}/pt2pt:${PATH}"
export PATH="${OMB_INSTALL_DIR}/startup:${PATH}"
%post -c /bin/bash
# Set operating system mirror URL
export MIRRORURL='http://us.archive.ubuntu.com/ubuntu'
# Set operating system version
export OSVERSION='bionic'
# Set system locale
export LC_ALL='C'
# Set debian frontend interface
export DEBIAN_FRONTEND='noninteractive'
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='4'
export OMPI_MINOR='0'
export OMPI_REVISION='5'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
# Set paths to OpenMPI binaries and libraries
export PATH="${OMPI_INSTALL_DIR}/bin:${PATH}"
export LD_LIBRARY_PATH="${OMPI_INSTALL_DIR}/lib:${LD_LIBRARY_PATH}"
# Upgrade all software packages to their latest versions
apt-get -y update && apt-get -y upgrade
cd /tmp
# Set OMB major and minor version numbers, root and
# installation directories
export OMB_ROOT_URL='http://mvapich.cse.ohio-state.edu/download/mvapich'
export OMB_MAJOR='5'
export OMB_MINOR='7'
export OMB_VERSION="${OMB_MAJOR}.${OMB_MINOR}"
export OMB_ROOT_DIR='/opt/omb'
export OMB_INSTALL_DIR="${OMB_ROOT_DIR}/${OMB_VERSION}"
# Download, build, and install OSU Microbenchmarks
wget "${OMB_ROOT_URL}/osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
tar -xf "osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
cd "osu-micro-benchmarks-${OMB_VERSION}"
./configure --prefix="${OMB_INSTALL_DIR}" CC='mpicc' CXX='mpicxx'
make
make install
# Reorganize OSU Microbenchmarks directory structure
cd "${OMB_INSTALL_DIR}"
mv libexec/osu-micro-benchmarks/mpi/collective ./
mv libexec/osu-micro-benchmarks/mpi/one-sided ./
mv libexec/osu-micro-benchmarks/mpi/pt2pt ./
mv libexec/osu-micro-benchmarks/mpi/startup ./
rm -rf libexec
cd /tmp
# Remove OMB build directory and source tarball
rm -rf "osu-micro-benchmarks-${OMB_VERSION}"
rm "osu-micro-benchmarks-${OMB_VERSION}.tar.gz"
# Cleanup
apt-get -y autoremove --purge
apt-get -y clean
# Update database for mlocate
updatedb
%files
%runscript
%test
Bootstrap: oras
From: ghcr.io/mkandes/naked-singularity:ubuntu-18.04
%labels
APPLICATION_NAME ubuntu + mlnx-ofed
APPLICATION_VERSION 18.04 + 4.7-3.2.9.0
APPLICATION_URL https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed
AUTHOR_NAME Marty Kandes
AUTHOR_EMAIL mkandes@sdsc.edu
LAST_UPDATED 20210804
%setup
%environment
# Set Mellanox OFED version, operating system, and hardware platform
export MLNX_ROOT_URL='http://www.mellanox.com/downloads/ofed'
export MLNX_OFED_VERSION='4.7-3.2.9.0'
export MLNX_OS_VERSION='ubuntu18.04'
export MLNX_PLATFORM='x86_64'
%post -c /bin/bash
# Set operating system mirror URL
export MIRRORURL='http://us.archive.ubuntu.com/ubuntu'
# Set operating system version
export OSVERSION='bionic'
# Set system locale
export LC_ALL='C'
# Set debian frontend interface
export DEBIAN_FRONTEND='noninteractive'
# Upgrade all software packages to their latest versions
apt-get -y update && apt-get -y upgrade
# Install all dependencies and/or prerequisites for Mellanox OFED
apt-get -y install bison
apt-get -y install chrpath
apt-get -y install debhelper
apt-get -y install dpatch
apt-get -y install flex
apt-get -y install graphviz
apt-get -y install libnl-3-dev
apt-get -y install libnl-route-3-200
apt-get -y install tcl-dev
apt-get -y install tk-dev
apt-get -y install swig
cd /tmp
# Set Mellanox OFED version, operating system, and hardware platform
export MLNX_ROOT_URL='http://www.mellanox.com/downloads/ofed'
export MLNX_OFED_VERSION='4.7-3.2.9.0'
export MLNX_OS_VERSION='ubuntu18.04'
export MLNX_PLATFORM='x86_64'
# Download and install Mellanox OFED drivers and supporting
# libraries for userspace access to Ethernet, RDMA, and Infiniband.
# https://docs.mellanox.com/pages/releaseview.action?pageId=15049785
wget "${MLNX_ROOT_URL}/MLNX_OFED-${MLNX_OFED_VERSION}/MLNX_OFED_LINUX-${MLNX_OFED_VERSION}-${MLNX_OS_VERSION}-${MLNX_PLATFORM}.tgz"
tar -xf "MLNX_OFED_LINUX-${MLNX_OFED_VERSION}-${MLNX_OS_VERSION}-${MLNX_PLATFORM}.tgz"
cd "MLNX_OFED_LINUX-${MLNX_OFED_VERSION}-${MLNX_OS_VERSION}-${MLNX_PLATFORM}"
./mlnxofedinstall --user-space-only --without-fw-update --force
cd /tmp
# Remove Mellanox OFED archive directory and tarball
rm -rf "MLNX_OFED_LINUX-${MLNX_OFED_VERSION}-${MLNX_OS_VERSION}-${MLNX_PLATFORM}"
rm "MLNX_OFED_LINUX-${MLNX_OFED_VERSION}-${MLNX_OS_VERSION}-${MLNX_PLATFORM}.tgz"
# Install additional tools
apt-get -y install numactl
apt-get -y install libnuma-dev
# Cleanup
apt-get -y autoremove --purge
apt-get -y clean
# Update database for mlocate
updatedb
%files
%runscript
%test
Bootstrap: oras
From: ghcr.io/mkandes/naked-singularity:ubuntu-18.04-mlnx-ofed-4.7-3.2.9.0
%labels
APPLICATION_NAME ubuntu + +mlnx-ofed + openmpi
APPLICATION_VERSION 18.04 + 4.7-3.2.9.0 + 3.1.6
APPLICATION_URL https://www.open-mpi.org
AUTHOR_NAME Marty Kandes
AUTHOR_EMAIL mkandes@sdsc.edu
LAST_UPDATED 20210804
%setup
%environment
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='3'
export OMPI_MINOR='1'
export OMPI_REVISION='6'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
# Set paths to OpenMPI binaries and libraries
export PATH="${OMPI_INSTALL_DIR}/bin:${PATH}"
export LD_LIBRARY_PATH="${OMPI_INSTALL_DIR}/lib:${LD_LIBRARY_PATH}"
%post -c /bin/bash
# Set operating system mirror URL
export MIRRORURL='http://us.archive.ubuntu.com/ubuntu'
# Set operating system version
export OSVERSION='bionic'
# Set system locale
export LC_ALL='C'
# Set debian frontend interface
export DEBIAN_FRONTEND='noninteractive'
# Upgrade all software packages to their latest versions
apt-get -y update && apt-get -y upgrade
# Install OpenMPI dependencies
apt-get -y install zlib1g-dev
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='3'
export OMPI_MINOR='1'
export OMPI_REVISION='6'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
cd /tmp
# Download, build, and install OpenMPI
wget "${OMPI_ROOT_URL}/v${OMPI_MAJOR}.${OMPI_MINOR}/openmpi-${OMPI_VERSION}.tar.gz"
tar -xf "openmpi-${OMPI_VERSION}.tar.gz"
cd "openmpi-${OMPI_VERSION}"
./configure --prefix="${OMPI_INSTALL_DIR}"
make all install
cd /tmp
# Remove OpenMPI build directory and source tarball
rm -rf "openmpi-${OMPI_VERSION}"
rm "openmpi-${OMPI_VERSION}.tar.gz"
# Cleanup
apt-get -y autoremove --purge
apt-get -y clean
# Update database for mlocate
updatedb
%files
%runscript
%test
Bootstrap: oras
From: ghcr.io/mkandes/naked-singularity:ubuntu-18.04-mlnx-ofed-4.7-3.2.9.0
%labels
APPLICATION_NAME ubuntu + mlnx-ofed + openmpi
APPLICATION_VERSION 18.04 + 4.7-3.2.9.0 + 4.0.5
APPLICATION_URL https://www.open-mpi.org
AUTHOR_NAME Marty Kandes
AUTHOR_EMAIL mkandes@sdsc.edu
LAST_UPDATED 20210804
%setup
%environment
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='4'
export OMPI_MINOR='0'
export OMPI_REVISION='5'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
# Set paths to OpenMPI binaries and libraries
export PATH="${OMPI_INSTALL_DIR}/bin:${PATH}"
export LD_LIBRARY_PATH="${OMPI_INSTALL_DIR}/lib:${LD_LIBRARY_PATH}"
%post -c /bin/bash
# Set operating system mirror URL
export MIRRORURL='http://us.archive.ubuntu.com/ubuntu'
# Set operating system version
export OSVERSION='bionic'
# Set system locale
export LC_ALL='C'
# Set debian frontend interface
export DEBIAN_FRONTEND='noninteractive'
# Upgrade all software packages to their latest versions
apt-get -y update && apt-get -y upgrade
# Install OpenMPI dependencies
apt-get -y install zlib1g-dev
# Set OpenMPI major, minor, and revision numbers, root and
# installation directories
export OMPI_ROOT_URL='https://download.open-mpi.org/release/open-mpi'
export OMPI_MAJOR='4'
export OMPI_MINOR='0'
export OMPI_REVISION='5'
export OMPI_VERSION="${OMPI_MAJOR}.${OMPI_MINOR}.${OMPI_REVISION}"
export OMPI_ROOT_DIR='/opt/openmpi'
export OMPI_INSTALL_DIR="${OMPI_ROOT_DIR}/${OMPI_VERSION}"
cd /tmp
# Download, build, and install OpenMPI
wget "${OMPI_ROOT_URL}/v${OMPI_MAJOR}.${OMPI_MINOR}/openmpi-${OMPI_VERSION}.tar.gz"
tar -xf "openmpi-${OMPI_VERSION}.tar.gz"
cd "openmpi-${OMPI_VERSION}"
./configure --prefix="${OMPI_INSTALL_DIR}" --without-verbs
make all install
cd /tmp
# Remove OpenMPI build directory and source tarball
rm -rf "openmpi-${OMPI_VERSION}"
rm "openmpi-${OMPI_VERSION}.tar.gz"
# Cleanup
apt-get -y autoremove --purge
apt-get -y clean
# Update database for mlocate
updatedb
%files
%runscript
%test
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment