Steps on Param Pravega

 Param Pravega uses SPACK for compilation and linking of modules. For more details on SPACK, refer to the SPACK web page , SPACK user guide , Param Pravega user guide, and official SPACK manual.

Example:

In order to find a installed library one can use “spack list” or “spack find”. In some cases you will find multiple options for loading in the libraries. For example, if you do

you will get ssgjscn as one of the options. Please make sure you load all the required compilers, e.g., icc or mpicc or gcc that you might need for the compilation of your code in the same manner.

In some codes you might need to include libraries during compilation such as in CUDA. For such cases the paths to the libraries can either be included in the .bashrc file or can be mentioned in the concerned makefiles used for the compilation. For example, when you want to use cuda@11.1.1 , you will need to specify the paths to /include and /lib. The paths can be figure out in the following manner.

First load the required package (here it is cuda@11.1.1 ) by typing

The paths to the /include and /lib can be found using the command

This lists the paths against the loaded libraries; for example the path to the cuda@11.1.1 library is listed below:

Paths to the respective include and lib64 libraries are:

There are two possibilities to proceed

• one is to include the paths in the respective makefiles

as

and include as

Finally include $(LIBS) in your compilation line in the Makefile. Similarly for the IDIR.

• Second is to add the path in your bashrc file;

and then execute1

Similar procedures can be adopted for the other softwares.

Some software like OpenFoam needs setting the path to the INTEL_LICENSE_FILE that is required for compilation of codes. This needs to be added to the .bashrc file. The path to the INTEL_LICENSE_FILE is : /opt/intel/serverlicenses/COM_L___G5NV-32584FFV.lic

The intel license is configured in 3 nodes (redundant servers) i.e login01, pravega0, mgmt01 and can be accessed with the port 27009.

These additional information can also be added to .bashrc, followed by source command.

Then compile your codes using the usual commands. e.g.,

Notes:

• If you want to use Intel OneAPI compilers:
      o >> spack load intel-oneapi-compilers@2021.4.0
      o >> spack load intel-mpi@2019.10.317 %intel@19.0.5.281
• If you want to use Intel MPI:
      o >> module load compiler/intel/oneapi/mpi/2021.5.
• If you want to use hdf5 libraries and h5pcc compiler
      o >> spack load hdf5 (Choose ssgjscn option).
• If you want to use h5pfc compiler wrapper, do
      o >> spack load hdf5 +fortran %intel
• For spack environment creation, use the following command
      o >>spack env create -d <path>
• For compiling GPU code on the PGI compiler:
      o >> spack load nvhpc
      o >>spack load gcc@11.2.0
• Some programs when loaded with spack may not have their include dirs appended to the CPATH environment variable (specifically zlib)
      o  Find the zlib install dir using
      o >>spack find –paths zlib
      o  Add the include dir to CPATH (bash shell)
      o >>export CPATH=”zlib_install_dir/include:$CPATH”
• To compile OpenMP with Slurm Option
      o  spack load openmpi+legacylaunchers fabrics=auto schedulers=auto
• spack automatically sets the CC, CXX, FC, MPICC, MPICXX, MPIF90 etc. environment variables. If need to change, set these environment variables to desired values.

For small data needs of your program, the data can be accessed from your home directory.

For large data needs, transfer your data to /mnt/scratch/<user_id>

Create a job script, e.g., job.sh. Param Pravega used SLURM job scheduler. Submit your job using the SLURM submission command.