Documentation

Available online at https://wiki.sophia.mines-paristech.fr/cimlib

Launch a job

If you want to launch a job, run this command :

cimlib_run.sh -f cimlib.cfg

Configuration file cimlib.cfg must be filled . An example of this file can be found in :

/softs/cimlib/data/scripts/cimlib.cfg

Options of cimlib_run.sh script can be displayed by writting this command :

cimlib_run.sh -h

Options are :

-h -- display this help
-f -- parameters filename
-e -- executable full path
-p -- main filename
-n -- number of cores used
-o -- use oar scheduler
-r -- redirect output and error log in a file (if USE_OAR = false)
-w -- maximum duration of this job (if USE_OAR = true)
-a -- job identifier; launch this job after that another job (defined by job identifier) is finished (if USE_OAR = true)

Good to  know

Cimlib executable (cimlib_driver) and Cimlib cluster script (cimlib_run.sh) are automatically updated after each update of master branch of cimlib git project.
From any of your directory, you can launch theses commands :

cimlib_driver 

cimlib_run.sh

So it’s not necessary to download cimlib git project and compile it if you just want to use it.

Compilation

CimLib git project is hosted on https://git.sophia.mines-paristech.fr
If you want to access to this project, you have to send a request by email at support_cemef@mines-paristech.fr

Then you can download it by writting on linux console (ConEmu or Putty on Windows) :

git clone git@git.sophia.mines-paristech.fr:CIMLIB/cimlib.git
Requirements to compile cimlib
  • CMake 3.7.0 or higher
  • GIT 2.14 or higher
    • Windows : Check that environment variable “path” contains GIT binary path (c:\program files\git\bin)

    • Check if your SSH public/private keys have been setted up

      • Linux and MacOS : ~/.ssh/id_rsa.pub and ~/.ssh/id_rsa
      • Windows : c:/local/firstname.lastname/.ssh/id_rsa.pub and c:/local/firstname.lastname/.ssh/id_rsa 
      • Otherwise, generate them with the following command :
        • ssh-keygen -t rsa (press Enter for each question)
    • Check if your SSH public key has been put on your gitlab account server (git.sophia.mines-paristech.fr => Settings at top right of page) => SSH keys at middle left of page)
  • Compiler :
    • Linux and MacOS : OpenMPI 1.10.7 and gcc 4.9.3 or higher
    • Windows : Microsoft MPI V10.0 and Visual Studio 2015 or higher
  • MTC (Cemef mesher)
    •  Sources are protected, send a request by email at support_cemef@mines-paristech.fr to get headers and library.
    • Check or set MTC_TSV_DIR environment variable (path to MTC)
  • PetsC 3.8.4
    • Linux and MacOS
      • Compilation (assuming that $MPI_HOME={path_of_openmpi1.10.7}) : 
        • ./configure --with-mpi-shared=1 --with-x=0 --with-debugging=0 --with-mpi-dir=$MPI_HOME --download-f2cblaslapack PETSC_ARCH=linux PETSC_DIR=$PWD --with-shared-libraries COPTFLAGS="-O3 -march=native -mtune=native" CXXOPTFLAGS="-O3 -march=native -mtune=native" FOPTFLAGS="-O3 -march=native -mtune=native"
    • Windows
      • Compile PetsC with cygwin (https://www.mcs.anl.gov/petsc/documentation/installation.html)
    • Check or set PETSC_DIR and PETS_ARCH environment variables (path to PetsC and architecture used to compile PetsC)
  • BLAS LAPACK
    • Theses libraries could be installed with PetsC (–download-f2cblaslapack)
    • Check or set BLAS_LAPACK_DIR and BLAS_LAPACK_LIBRARIES environment variables (path to Lapack and libraries names, inside PetsC directory if installed with PetsC)
  • The following libraries are optionals. They are automatically downloaded and installed during first compilation of cimlib :
    •  FELICIA : Cemef git project (including another mesher) downloadable at git@git.sophia.mines-paristech.fr:CIMLIB/felicia.git => branch cimlib
    • VTK 8.2.0 : git project downloadable at https://gitlab.kitware.com/vtk/vtk.git => tag v8.2.0. Compiled with theses CMake arguments :
      -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DBUILD_TESTING=OFF -DVTK_Group_Rendering=OFF -DVTK_Group_StandAlone=OFF -DVTK_USE_X=OFF -DModule_vtkIOXML=ON -DCMAKE_USER_MAKE_RULES_OVERRIDE=compilerFlagsOverrides.cmake -DCMAKE_USER_MAKE_RULES_OVERRIDE_CXX=cimlibCompilerFlagsOverrides.cmake
    • HDF5 : git project downloadable at https://github.com/live-clones/hdf5.git => branch master). Compiled with theses CMake arguments :
      -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_EXECS=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DHDF5_BUILD_EXAMPLES=OFF -DHDF5_BUILD_TOOLS=OFF -DHDF5_BUILD_CPP_LIB=OFF -DHDF5_GENERATE_HEADERS=ON -DCMAKE_USER_MAKE_RULES_OVERRIDE=compilerFlagsOverrides.cmake -DCMAKE_USER_MAKE_RULES_OVERRIDE_CXX=cimlibCompilerFlagsOverrides.cmake

First compilation

  • Linux and MacOS
   cd cimlib
   ./build.sh
  • Windows
    • Open “Windows Explorer”
    • Change to {path_to_cimlib}
    • Double click on build.bat


Compilation (after first compilation)

  • Linux and MacOS
    cd cimlib/buildLinux
    cmake .. (optional, only if a new file has been added or if a cmake file has been modified)
    make -j4
  • Windows
    • Only if a new file has been added (*.cpp, *.h) or if a cmake file has been modified (*.cmake, CMakeLists.txt…)
      • Launch cmake-gui
      • Choose “{path_to_cimlib}” path as source (modifiy {path_to_cimlib})
      • Choose “{path_to_cimlib}/buildWindows” as destination
      • Configure (confirm creation of “buildWindows” directory, use default native compiler “Visual Studio 14 2015 Win64”)
      • Generate
    • Open buildWindows/cimlib.sln and generate solution (Release version)

Tools

  • Linux and MacOS
    • build/bin/cimlib_driver ⇒ cimlib executable
    • build/bin/cimlib_run.sh ⇒ script to launch cimlib executable in parrallel mode using OAR scheduler (launch cimlib_run.sh -h to get informations)
  • Windows
    • buildWindows/bin/cimlib_driver ⇒ cimlib executable
    • buildWindows/bin/cimlib_run.bat ⇒ script to launch cimlib executable in parrallel mode using MSMPI (open a console launch cimlib_run.bat -h to get informations)

Add a new test

  • Create directory containing your files in cimlib/tests
  • Required files that the directory must contain :
    • main.mtc (main file of test)
    • CMakeLists.txt, example : 
      # Number of cores used by MPI
      set(MPI_NUMBER_CORES_USED 2)
      # Main code of test
      include(cimlibTest)

    • ref_{output_filename} (all output files that you want to compare prefixed by ref_ keyword)

  • Reference files ref_{output_filename} can be defined in sub-directory (example results/ref_inc.vtf)
  • If .mtc files contain output filename or mesh filename, cimlib will search path of these files by priority order :
    1.  relative path of the file from output directory
    2. absolute path given by CIMLIB_INPUT_DIRECTORY environment variable
    3. absolute path given by CIMLIB_DATABASE environment variable
  • Examples of meshes and parameters files for partioner (*.t, partit.txt, remaille.txt, decale.txt, partit_ameliore.txt) are in database cimlib/data/meshes and cimlib/data/mesher_parameters_files

Launch tests

  • Linux and MacOS
    • Go to into build directory
    • Launch all tests in verbose mode :
ctest -V
    • Fifth test :

ctest -V -I 5,5

    • Fifth to tenth test :

ctest -V -I 5,10

    • Specify test using regular expression. For example, all tests containing solver substring :

ctest -V -R solver

  • Windows
    • Launch all tests in verbose mode :
      • generate RUN_ALL_TESTS solution under visual
    • One test
      • launch cmake-gui, select TEST you want, configure, generate
      • open buildWindows/cimlib.sln or update project and generate RUN_{test-name} solution

 

Run cimlib

  • A configuration file must be created (template of configuration file : cimlib/data/scripts/cimlib.cfg)
  • Linux and MacOs
    • cd cimlib/build/bin
    • cimlib_run.sh -f cimlib.cfg
  • Windows (with a cmd console or ConEmu)
    • cd cimlib/buildWindows/bin
    • cimlib_run.bat -f cimlib.cfg
#######################################
# Cimlib configuration file           #
# Usage : cimlib_run.sh -f cimlib.cfg #
#######################################

# Executable full path (default /softs/cimlib/bin/cimlib_driver)
EXECUTABLE /softs/cimlib/bin/cimlib_driver

# Main MTC filename (default main.mtc)
MAIN_FILENAME Lanceur.mtc

# Input path (default ./)
INPUT_DIRECTORY "./"

# Number of cores used (default 1)
NUMBER_CORES_USED 1

# Number of nodes used (default empty, means arbitrary nodes)
NUMBER_NODES_USED

# Name of MPI job (default CimLib_Test)
MPI_JOB_NAME CimLib_Test

# MPI wall time (seconds, default 3600)
MPI_WALL_TIME 3600

# Scheduler property (only avalaible on linux cluster, default empty)
# Examples of property on intel cluster :
# "host='n-in45' or host='n-in53'"
# "mem='128'"
# "nbcores='28'"
SCHEDULER_PROPERTY "nbcores='28'"

Debug cimlib

  • Compile cimlib in debug mode
    • Linux and MacOS
      • cd cimlib/build
      • ccmake .. => Press “t“, move to CMAKE_BUILD_TYPE, press “ENTER“, write Debug, press “ENTER“, press “c“, press “g“
      • make -j4
    • Windows
      • open buildWindows/cimlib.sln, select Debug before compiling
  • Check memory leaks and uninitialized variables (assuming that cimlib has been compiled in Debug mode)
    • Linux and MacOS
      • install valgrind tool (https://www.valgrind.org, already done on cemef cluster) with option with-mpicc={mpicc_path}
      • prepare your case (*.mtc and cimlib.cfg)
      • open cimlib.cfg and set theses lines (the second one is optional and available only on cemef cluster) :
COMMAND_BEFORE_EXECUTABLE "valgrind --leak-check=full --log-file=valgrind%p.log --show-reachable=yes --error-limit=no"
COMMAND_AFTER_JOB /usr/local/bin/oar_filter_valgrind.sh
      • run cimlib :
cimlib_run.sh -f cimlib.cfg
      • filter valgrind*.log files by using this command (on cemef cluster, this step is automatically done by adding COMMAND_AFTER_JOB in cimlib.cfg) :
cimlib_valgrind_filter "valgrind*.log" (double quotes mandatory to take account of regular expression)
      • open valgrindFiltered.log and parse issues