Installation¶
Installing via Spack¶
The HDF5 and ZFP libraries and the H5Z-ZFP plugin are all now part of the
Spack package manager. If you already have Spack installed, the easiest way to
install H5Z-ZFP is to simply use the Spack command spack install h5z-zfp
.
If you do not have Spack installed, it is very easy to install.
git clone https://github.com/llnl/spack.git
. spack/share/spack/setup-env.sh
spack install h5z-zfp
By default, H5Z-ZFP will attempt to build with Fortran support which requires a Fortran compiler. If you wish to exclude support for Fortran, use the command
spack install h5z-zfp~fortran
Note that these commands will build H5Z-ZFP and all of its dependencies including
the HDF5 library (as well as a number of other dependencies you may not initially
expect. Be patient and let the build complete). In addition, by default, Spack installs
packages to directory hashes within the cloned Spack repository’s directory tree,
$spack/opt/spack
. You can find the resulting installed HDF5 library with the command
spack find -vp hdf5
and your resulting H5Z-ZFP plugin installation with the command
spack find -vp h5z-zfp
. If you wish to exercise more control over where Spack
installs things, have a look at
configuring Spack
Manual Installation¶
If Spack is not an option for you, information on manually installing is provided here.
Prerequisites¶
- ZFP Library (or from Github)
- HDF5 Library
- H5Z-ZFP filter plugin
Compiling ZFP¶
- There is a
Config
file in top-level directory of the ZFP distribution that holdsmake
variables the ZFP Makefiles use. By default, this file is setup for a vanilla GNU compiler. If this is not the appropriate compiler, editConfig
as necessary to adjust the compiler and compilation flags. - An important flag you will need to adjust in order to use the ZFP library with this HDF5 filter is
the
BIT_STREAM_WORD_TYPE
CPP flag. To use ZFP with H5Z-ZFP, the ZFP library must be compiled withBIT_STREAM_WORD_TYPE
ofuint8
. Typically, this is achieved by including a line inConfig
of the formDEFS += -DBIT_STREAM_WORD_TYPE=uint8
. If you attempt to use this filter with a ZFP library compiled differently from this, the filter’scan_apply
method will always return false. This will result in silently ignoring an HDF5 client’s request to compress data with ZFP. Also, be sure to see Endian Issues. - After you have setup
Config
, simply runmake
and it will build the ZFP library placing the library in alib
sub-directory and the necessary include files ininc[lude]
sub-directory. - For more information and details, please see the ZFP README.
Compiling HDF5¶
- If you want to be able to run the fortran tests for this filter, HDF5 must be
configured with both the
--enable-fortran
and--enable-fortran2003
configuration switches. Otherwise, any vanilla installation of HDF5 is acceptable. - The Fortran interface to this filter requires a Fortran 2003 compiler because it uses ISO_C_BINDING to define the Fortran interface.
Compiling H5Z-ZFP¶
H5Z-ZFP is designed to be compiled both as a standalone HDF5 plugin and as a separate library an application can explicitly link. See Plugin vs. Library Operation.
Once you have installed the prerequisites, you can compile H5Z-ZFP using a command-line…
make [FC=<Fortran-compiler>] CC=<C-compiler>
ZFP_HOME=<path-to-zfp> HDF5_HOME=<path-to-hdf5>
PREFIX=<path-to-install>
where <path-to-zfp>
is a directory containing ZFP inc[lude]
and lib
dirs and
<path-to-hdf5>
is a directory containing HDF5 include
and lib
dirs.
If you don’t specify a C compiler, it will try to guess one from your path. Fortran
compilation is optional. If you do not specify a Fortran compiler, it will not attempt
to build the Fortran interface. However, if the variable FC
is already defined in
your enviornment (as in Spack for example), then H5Z-ZFP will attempt to build Fortran.
If this is not desired, the solution is to pass an empty FC
on the make command
line as in…
make FC= CC=<C-compiler>
ZFP_HOME=<path-to-zfp> HDF5_HOME=<path-to-hdf5>
PREFIX=<path-to-install>
The Makefile uses GNU Make syntax and is designed to work on OSX and Linux. The filter has been tested on gcc, clang, xlc, icc and pgcc compilers and checked with valgrind.
The command make help
will print useful information
about various make targets and variables. make check
will compile everything
and run a handful of tests.
If you don’t specify a PREFIX
, it will install to ./install
. The installed
package will look like…
$(PREFIX)/include/{H5Zzfp.h,H5Zzfp_plugin.h,H5Zzfp_props.h,H5Zzfp_lib.h}
$(PREFIX)/plugin/libh5zzfp.{so,dylib}
$(PREFIX)/lib/libh5zzfp.a
where $(PREFIX)
resolves to whatever the full path of the installation is.
To use the installed filter as an HDF5 plugin, you would specify, for example,
setenv HDF5_PLUGIN_PATH $(PREFIX)/plugin
H5Z-ZFP Source Code Organization¶
The source code is in two separate directories
src
includes the ZFP filter and a few header files
H5Zzfp_plugin.h
is an optional header file applications may wish to include because it contains several convenient macros for easily controlling various compression modes of the ZFP library (rate, precision, accuracy, expert) via the Generic Interface.H5Zzfp_props.h
is a header file that contains functions to control the filter using temporary Properties Interface. Fortran callers are required to use this interface.H5Zzfp_lib.h
is a header file for applications that wish to use the filter explicitly as a library rather than a plugin.H5Zzfp.h
is an all-of-the-above header file for applications that don’t care too much about separating out the above functionalities.
test
includes various tests. In particulartest_write.c
includes examples of using both the Generic Interface and Properties Interface. In addition, there is an example of how to use the filter from Fortran intest_rw_fortran.F90
.
Silo Integration¶
This filter is also built-in to the Silo library. In particular, the ZFP library itself is also embedded in Silo but is protected from appearing in Silo’s global namespace through a struct of function pointers (see Namespaces in C). If you happen to examine the source code for H5Z-ZFP, you will see some logic there that is specific to using this plugin within Silo and dealing with ZFP as an embedded library using this struct of function pointers wrapper. Just ignore this.