The Jobscript job.ksh
¶
The jobscript is a Korn-shell script which will be created at runtime for each flex_extract
execution in the application modes remote and gateway.
It is based on the job.temp
template file which is stored in the Templates
directory.
This template is by itself generated in the installation process from a job.template
template file.
Flex_extract
uses the python package genshi to generate
the Korn-shell script from the template files by substituting the individual parameters.
These individual parameters are marked by a doubled $
sign in job.temp
.
The jobscript has a number of settings for the batch system which are fixed and differentiates between the ecgate and the cca/ccb server system to load the necessary modules for the environment when submitted to the batch queue.
The submission is done by the ECaccess
tool from within flex_extract
with the command ecaccess-job-submit
.
What does the jobscript do?¶
It sets necessary batch system parameters
It prepares the job environment at the ECMWF servers by loading the necessary library modules
It sets some environment variabels for the single session
It creates the directory structure in the users
$SCRATCH
file systemIt creates a CONTROL file on the ECMWF servers whith the parameters set before creating the
jobscript.ksh
.Flex_extract
has a set of parameters which are given to the jobscript with its default or the user defined values. It also sets theCONTROL
as an environment variable.
Flex_extract
is started from within thework
directory of the new directory structure by calling thesubmit.py
script. It sets new pathes for input and output directory and the recently generatedCONTROL
file.At the end it checks if the script returned an error or not and send the log file via email to the user.
Example jobscript.ksh
¶
#!/bin/ksh
# ON ECGB:
# start with ecaccess-job-submit -queueName ecgb NAME_OF_THIS_FILE on gateway server
# start with sbatch NAME_OF_THIS_FILE directly on machine
#SBATCH --workdir=/scratch/ms/at/km4a
#SBATCH --qos=normal
#SBATCH --job-name=flex_ecmwf
#SBATCH --output=flex_ecmwf.%j.out
#SBATCH --error=flex_ecmwf.%j.out
#SBATCH --mail-type=FAIL
#SBATCH --time=12:00:00
## CRAY specific batch requests
##PBS -N flex_ecmwf
##PBS -q np
##PBS -S /usr/bin/ksh
## -o /scratch/ms/at/km4a/flex_ecmwf.${PBS_JOBID}.out
## job output is in .ecaccess_DO_NOT_REMOVE
##PBS -j oe
##PBS -V
##PBS -l EC_threads_per_task=24
##PBS -l EC_memory_per_task=32000MB
set -x
export VERSION=7.1
case ${HOST} in
*ecg*)
module unload grib_api
module unload eccodes
module unload python
module unload emos
module load python3
module load eccodes/2.12.0
module load emos/455-r64
export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python
;;
*cca*)
module unload python
module switch PrgEnv-cray PrgEnv-intel
module load python3
module load eccodes/2.12.0
module load emos
export SCRATCH=${TMPDIR}
export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python
;;
esac
cd ${SCRATCH}
mkdir -p python$$
cd python$$
export CONTROL=CONTROL
cat >${CONTROL}<<EOF
accmaxstep 24
acctime 18
acctype FC
accuracy 24
addpar None
area 74.0/-24.0/10.0/60.0
basetime None
cds_api None
controlfile CONTROL_CERA
cwc 1
dataset None
date_chunk 3
debug 1
destination annep@genericSftp
doubleelda 0
dpdeta 1
dtime 3
ec_api None
ecfsdir ectmp:/${USER}/econdemand/
ecgid at
ecstorage 0
ectrans 1
ecuid km4a
end_date 20000809
eta 1
etadiff 0
etapar 77
expver 1
format GRIB1
gateway srvx8.img.univie.ac.at
gauss 0
gaussian
grib2flexpart 0
grid 1.0/1.0
inputdir /raid60/nas/tmc/Anne/Interpolation/flexextract/flex_extract_v7.1/run/workspace
install_target None
job_chunk 1
job_template job.temp
left -24.
level 91
levelist 1/to/91
logicals gauss omega omegadiff eta etadiff dpdeta cwc wrf grib2flexpart ecstorage ectrans debug oper request public purefc rrint doubleelda
lower 10.
mailfail ${USER}
mailops ${USER}
makefile Makefile.gfortran
marsclass EP
maxstep 0
number 000
omega 0
omegadiff 0
oper 0
outputdir /raid60/nas/tmc/Anne/Interpolation/flexextract/flex_extract_v7.1/run/workspace
prefix CE
public 0
purefc 0
queue ecgate
request 2
resol 159
right 60.
rrint 0
smooth 0
start_date 20000809
step 00 00 00 00 00 00 00 00
stream ENDA
time 00 03 06 09 12 15 18 21
type AN AN AN AN AN AN AN AN
upper 74.
wrf 0
EOF
submit.py --controlfile=${CONTROL} --inputdir=./work --outputdir=./work 1> prot 2>&1
if [ $? -eq 0 ] ; then
l=0
for muser in `grep -i MAILOPS ${CONTROL}`; do
if [ ${l} -gt 0 ] ; then
mail -s flex.${HOST}.$$ ${muser} <prot
fi
l=$((${l}+1))
done
else
l=0
for muser in `grep -i MAILFAIL ${CONTROL}`; do
if [ ${l} -gt 0 ] ; then
mail -s "ERROR! flex.${HOST}.$$" ${muser} <prot
fi
l=$((${l}+1))
done
fi