Benchmark procedures:
Instructions
for users and developpers: on installing
a new version of Chromalgema it is advised to run the benchmarks. It consists
in a serie of tests to run the program on a large scope of options and parameters
values, and different datasets comprising one, two or three components.
The benchmark directory tree-structure
is detailed below. It can be as deep as 6 levels and classifies the
tests according to the following order of the options/parameters:
- Number of components (1, 2 or 3)
- Type of computation. (Simple simulation
with given parameters, Evolutionary optimization, Gradient optimization,
Coupling of the 2 optimizations, Gradient computations.)
- Numerical scheme (Godunov, ENO, Godunov+ Multiresolution,
ENO+
Multiresolution
- In the case of gradient optimization, gradient computation
method (adjoint or direct derivation)
- In the case of gradient optimization, descent
algorithm (AGLS or CLS)
- Type of isotherm and choice of parameters to be identified
Each test directory contains a parameter file named
param.dat and possibly data files,
unless generic data files located in the parent directory (One/Two/ThreeComp)
are used. It also contains a subdirectory Ref which contains the reference output
files: colonne#.out and output.
When the script chtest.sh is run,
the benchmark directory is searched
for all sub directories containing param.dat
file and the corresponding test is run, the output files colonne#.out and output which are then created are compared
with their homonyms in the Ref directory
with a diff whose result is saved in a Diff
file. The number of lines exhibiting differences is summarized for
each test in the report file in
the benchmark directory.
For users:
some differences between the current tests and the reference ones are normal:
for instance the cpu time evaluations can be different from on computer to
the other. Also in the case of EO optimization, elapsed time for each generation
is output, which is also machine dependent. These differences will appear
in the output file. Differences in
the colonne#.out files should be
investigated.
For
developpers only: whenever changes are made to the program, the benchmark
procedure chtest.sh should
be run and differences should be explained and
justified. When the new version is accepted, the new reference
files should be created and stored in the Ref directories. This is automatized
using the script chtestgen.sh in the benchmark directory (remember to sent
the new Ref contents with CVS)
The scripts chtestgen.sh
and chtest.shin
the benchmark directory make use
of other scripts: chtestgenfile.sh, verifrep.sh, and chtestfile.sh, which are located
in the bin directory.
benchmark directory tree
structure
OneComp One
component data
- Simul Simple simulation with given parameters
- God Godunov Scheme
- LangmuirAP
Langmuir isotherm
- EO Evolutionary Optimization
- God Godunov Scheme
- LangmuirAP
Langmuir isotherm - K and
N* identified
- LangmuirAPP
Langmuir isotherm - K'=KN* and
N* identified
- LangmuirKO
Langmuir isotherm - only K
identified
- Gradient
Gradient optimization
- Adjoint
Ajoint method
for gradient computation
- Measure
Direct derivation
for gradient computation
- EO_Grad
Evolutionary optimization relayed
by gradient
- Measure
Direct derivation
for gradient computation
- TestGrad
Comparison of gradient computation
methods (Adjoint and derived)
- LangmuirAP
Langmuir isotherm - K and N*
identified
TwoComp Two components data
- Simul Simple simulation with given parameters
- God Godunov Scheme
- BiLangmuir
BiLangmuir isotherm
- EO Evolutionary Optimization
- BiLangmuir
BiLangmuir isotherm
- Gradient
Gradient optimization
- Adjoint
Ajoint method
for gradient computation
ThreeComp Three components data
- Simul Simple simulation with given parameters
- God Godunov Scheme
- LangmuirIgor
Langmuir isotherm with component
dependent N*
- Gradient
Gradient optimization
- Adjoint
Ajoint method
for gradient computation
MAJ: MP 05/10/04
