MPB Installation

From AbInitio

(Difference between revisions)
Jump to: navigation, search
Revision as of 20:20, 15 October 2005 (edit)
Stevenj (Talk | contribs)
(initial start)
← Previous diff
Current revision (18:05, 16 February 2017) (edit)
Ardavan (Talk | contribs)

 
Line 1: Line 1:
 +{{MPB}}
In this section, we outline the procedure for installing the MIT Photonic-Bands package. Mainly, this consists of downloading and installing various prerequisites. As much as possible, we have attempted to take advantage of existing packages such as BLAS, LAPACK, FFTW, and GNU Guile, in order to make our code smaller, more robust, faster, and more flexible. Unfortunately, this may make the installation of MPB more complicated if you do not already have these packages. In this section, we outline the procedure for installing the MIT Photonic-Bands package. Mainly, this consists of downloading and installing various prerequisites. As much as possible, we have attempted to take advantage of existing packages such as BLAS, LAPACK, FFTW, and GNU Guile, in order to make our code smaller, more robust, faster, and more flexible. Unfortunately, this may make the installation of MPB more complicated if you do not already have these packages.
Line 7: Line 8:
'''Note:''' It is important that you use the ''same Fortran compiler'' to compile Fortran libraries (like LAPACK) and for configuring MPB. Different Fortran compilers often have incompatible linking schemes. (The Fortran compiler for MPB can be set via the <code>F77</code> environment variable.) '''Note:''' It is important that you use the ''same Fortran compiler'' to compile Fortran libraries (like LAPACK) and for configuring MPB. Different Fortran compilers often have incompatible linking schemes. (The Fortran compiler for MPB can be set via the <code>F77</code> environment variable.)
-== Installation Paths ==+'''Note:''' The latest, pre-installed versions of MPB and Meep running on Ubuntu can also be accessed on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) as a free [https://aws.amazon.com/marketplace/pp/B01KHWH0AS Amazon Machine Image (AMI)]. To access this AMI, follow these [http://www.simpetuscloud.com/launchsims.html instructions].
-First, let's review some important information about installing software on Unix systems, especially in regards to installing software in non-standard locations. (None of these issues are specific to MPB, but they've caused a lot of confusion among our beloved users.)+== Installation on MacOS X ==
-Most of the software below, including MPB, installs under <code>/usr/local</code> [http://www.pathname.com/fhs/ by default]<nowiki>; that is, libraries go in </nowiki><code>/usr/local/lib</code>, programs in <code>/usr/local/bin</code>, etc. If you don't have <code>root</code> privileges on your machine, you may need to install somewhere else, e.g. under <code>$HOME/install</code> (the <code>install/</code> subdirectory of your home directory). Most of the programs below use a GNU-style <code>configure</code> script, which means that all you would do to install there would be:+See [[Meep Installation#Installation on MacOS X|Meep installation on OS X]] for the easiest way to do this.
- ./configure --prefix=$HOME/install+{{Unix installation|name=MPB}}
-when configuring the program; the directories <code>$HOME/install/lib</code> etc. are created automatically as needed.+==BLAS and LAPACK==
-=== Paths for Configuring ===+MPB requires the BLAS and LAPACK libraries for matrix computations.
-There are two further complications. First, if you install in a non-standard location (and <code>/usr/local</code> is considered non-standard by some proprietary compilers), you will need to tell the compilers where to find the libraries and header files that you dutifully installed. You do this by setting two environment variables:+{{Installing BLAS and LAPACK}}
- setenv LDFLAGS "-L/usr/local/lib"+== MPI ''(for parallel MPB)'' ==
- setenv CPPFLAGS "-I/usr/local/include"+
-Of course, substitute whatever installation directory you used. Do this ''before'' you run the <code>configure</code> scripts, etcetera. You may need to include multiple <code>-L</code> and <code>-I</code> flags (separated by spaces) if your machine has stuff installed in several non-standard locations. Bourne shell users (e.g. <code>bash</code> or <code>ksh</code>) should use the "<code>export FOO=bar</code>" syntax instead of <code>csh</code>'s "<code>setenv FOO bar</code>", of course.+Optionally, MPB is able to run on a distributed-memory parallel machine, and to do this we use the standard MPI message-passing interface. You can learn about MPI from the [http://www-unix.mcs.anl.gov/mpi/ MPI Home Page]. Most commercial supercomputers already have an MPI implementation installed; two free MPI implementations that you can install yourself are [http://www-unix.mcs.anl.gov/mpi/mpich/ MPICH] and [http://www.lam-mpi.org/ LAM]; a promising new implementation is [http://www.open-mpi.org/ Open MPI]. MPI is ''not required'' to compile the ordinary, uniprocessor version of our software.
-You might also need to update your <code>PATH</code> so that you can run the executables you installed (although <code>/usr/local/bin</code> is in the default <code>PATH</code> on many systems). e.g. if we installed in our home directory as described above, we would do:+In order for the MPI version of our software to run successfully, we have a slightly nonstandard requirement: each process must be able to read from the disk. (This way, Guile can boot for each process and they can all read your control file in parallel.) Many (most?) commercial supercomputers, Linux [http://www.beowulf.org Beowulf] clusters, etcetera, satisfy this requirement.
- setenv PATH "$HOME/install/bin:$PATH"+Also, in order to get good performance, you'll need fast interconnect hardware such as Myrinet; 100Mbps Ethernet probably won't cut it. The speed bottleneck comes from the FFT, which requires most of the communications in MPB. FFTW's MPI transforms ([[#FFTW|see below]]) come with benchmark programs that will give you a good idea of whether you can get speedups on your system. Of course, even with slow communications, you can still benefit from the memory savings per CPU for large problems.
-=== Paths for Running (Shared Libraries) ===+As described [[#MIT Photonic Bands|below]], when you configure MPB with MPI support (<code>--with-mpi</code>), it installs itself as <code>mpb-mpi</code>. See also the [user-ref.html#mpb-mpi user reference] section for information on using MPB on parallel machines. Normally, you should ''also'' install the serial version of MPB, if only to get the <code>mpb-data</code> utility, which is not installed with the MPI version.
- +
-Second, many of the packages installed below (e.g. Guile) are installed as shared libraries. You need to make sure that your runtime linker knows where to find these shared libraries. The bad news is that every operating system does this in a slightly different way. The good news is that, when you run <code>make install</code> for the packages involving shared libraries, the output includes the necessary instructions specific to your system, so pay close attention! It will say something like "<code>add LIBDIR to the <code>''foobar''</code> environment variable</code>," where <code>LIBDIR</code> will be your library installation directory (e.g. <code>/usr/local/lib</code>) and ''foobar'' is some environment variable specific to your system (e.g. <code>LD_LIBRARY_PATH</code> on some systems, including Linux). For example, you might do:+
- +
- setenv ''foobar'' "/usr/local/lib:$''foobar''"+
- +
-Note that we just add to the library path variable, and don't replace it in case it contains stuff already. If you use Linux and have <code>root</code> privileges, you can instead simply run <code>/sbin/ldconfig</code>, first making sure that a line "<code>/usr/local/lib</code>" (or whatever) is in <code>/etc/ld.so.conf</code>.+
- +
-If you don't want to type these commands every time you log in, you can put them in your <code>~/.cshrc</code> file (or <code>~/.profile</code>, or <code>~/.bash_profile</code>, depending on your shell).+
- +
-== Fun with Fortran ==+
- +
-MPB, and many of the libraries it calls, are written in C, but it also calls libraries such as BLAS and LAPACK (see below) that are usually compiled from Fortran. This can cause some added difficulty because of the various linking schemes used by Fortran compilers. MPB's <code>configure</code> script attempts to detect the Fortran linking scheme automatically, but in order for this to work ''you must use the same Fortran compiler and options with MPB as were used to compile BLAS/LAPACK''.+
- +
-By default, MPB looks for a vendor Fortran compiler first (<code>f77</code>, <code>xlf</code>, etcetera) and then looks for GNU <code>g77</code>. In order to manually specify a Fortran compiler <code>foobar</code> you would [#mpb configure MPB] with '<code>./configure F77=foobar ...</code>'.+
- +
-If, when you compiled BLAS/LAPACK, you used compiler options that alter the linking scheme (e.g. <code>g77</code>'s <code>-fcase-upper</code> or <code>-fno-underscoring</code>), you will need to pass the same flags to MPB via '<code>./configure FFLAGS="...''flags''..." ...</code>'.+
- +
-One final note: you may run into trouble if you use a C compiler by a different vendor than your Fortran compiler, due to incompatible libraries. By default, MPB looks for a vendor C compiler (<code>cc</code>) first, but you can specify a different C compiler with '<code>./configure CC=foobar ...</code>'.+
- +
-== BLAS ==+
- +
-The first thing you must have on your system is a BLAS implementation. "BLAS" stands for "Basic Linear Algebra Subroutines," and is a standard interface for operations like matrix multiplication. It is designed as a building-block for other linear-algebra applications, and is used both directly by our code and in LAPACK (see below). By using it, we can take advantage of many highly-optimized implementations of these operations that have been written to the BLAS interface. (Note that you will need implementations of BLAS levels 1-3.)+
- +
-You can find more BLAS information, as well as a basic implementation, on the [http://www.netlib.org/blas/ BLAS Homepage]. Once you get things working with the basic BLAS implementation, it might be a good idea to try and find a more optimized BLAS code for your hardware. Vendor-optimized BLAS implementations are available as part of the Compaq CXML, IBM ESSL, SGI sgimath, and other libraries. Recently, there has also been work on self-optimizing BLAS implementations that can achieve performance competitive with vendor-tuned codes; see the [http://math-atlas.sourceforge.net/ ATLAS] homepage (and also [http://www.icsi.berkeley.edu/~bilmes/phipac/ PhiPACK]). Links to more BLAS implementations can be found on [http://SAL.KachinaTech.COM/B/0/BLAS.html SAL]. I recommend ATLAS, but it does take some time to compile.+
- +
-Note that the generic BLAS does not come with a <code>Makefile</code><nowiki>; compile it with something like: </nowiki>+
- +
- mkdir blas &amp;&amp; cd blas # the BLAS archive does not create its own directory+
- ''get http://www.netlib.org/blas/blas.tgz''+
- gunzip blas.tgz+
- tar xf blas.tar+
- f77 -c -O3 *.f # compile all of the .f files to produce .o files+
- ar rv libblas.a *.o # combine the .o files into a library+
- su -c "cp libblas.a /usr/local/lib" # switch to root and install+
- +
-(Replace <code>-O3</code> with your favorite optimization options. On Linux, I use <code>g77 -O3 -fomit-frame-pointer -funroll-loops</code>, with <code>-malign-double -mcpu=i686</code> on a Pentium II.) Note that MPB looks for the standard BLAS library with <code>-lblas</code>, so the library file should be called <code>libblas.a</code> and reside in a standard directory like <code>/usr/local/lib</code>. (See also below for the <code>--with-blas=''lib''</code> option to MPB's <code>configure</code> script, to manually specify a library location.)+
- +
-== LAPACK ==+
- +
-LAPACK, the Linear Algebra PACKage, is a standard collection of routines, built on BLAS, for more-complicated (dense) linear algebra operations like matrix inversion and diagonalization. You can download LAPACK from the [http://www.netlib.org/lapack/ LAPACK Home Page]. More LAPACK links can be found on [http://SAL.KachinaTech.COM/B/0/LAPACK.html SAL].+
- +
-Note that MPB looks for LAPACK by linking with <code>-llapack</code>. This means that the library must be called <code>liblapack.a</code> and be installed in a standard directory like <code>/usr/local/lib</code> (alternatively, you can specify another directory via the <code>LDFLAGS</code> environment variable as described earlier). (See also below for the <code>--with-lapack=''lib''</code> option to MPB's <code>configure</code> script, to manually specify a library location.)+
- +
-== MPI ''(optional)'' ==+
- +
-Optionally, MPB is able to run on a distributed-memory parallel machine, and to do this we use the standard MPI message-passing interface. You can learn about MPI from the [http://www-unix.mcs.anl.gov/mpi/index.html MPI Home Page]. Most commercial supercomputers already have an MPI implementation installed; two free MPI implementations that you can install yourself are [http://www-unix.mcs.anl.gov/mpi/mpich/ MPICH] and [http://www.lam-mpi.org/ LAM]. MPI is ''not required'' to compile the ordinary, uniprocessor version of MPB.+
- +
-In order for the MPI version of MPB to run successfully, we have a slightly nonstandard requirement: each process must be able to read from the disk. (This way, Guile can boot for each process and they can all read your control file in parallel.) Many (most?) commercial supercomputers, Linux [http://www.beowulf.org Beowulf] clusters, etcetera, satisfy this requirement.+
- +
-Also, in order to get good performance, you'll need fast interconnect hardware such as Myrinet; 100Mbps Ethernet probably won't cut it. The speed bottleneck comes from the FFT, which requires most of the communications in MPB. FFTW's MPI transforms ([#fftw see below]) come with benchmark programs that will give you a good idea of whether you can get speedups on your system. Of course, even with slow communications, you can still benefit from the memory savings per CPU for large problems.+
- +
-As described [#mpb below], when you configure MPB with MPI support (<code>--with-mpi</code>), it installs itself as <code>mpb-mpi</code>. See also the [user-ref.html#mpb-mpi user reference] section for information on using MPB on parallel machines.+
MPI support in MPB is thanks to generous support from [http://www.clarendonphotonics.com/ Clarendon Photonics]. MPI support in MPB is thanks to generous support from [http://www.clarendonphotonics.com/ Clarendon Photonics].
Line 90: Line 38:
We require a portable, standard binary format for outputting the electromagnetic fields and similar volumetric data, and for this we use HDF. (If you don't have HDF5, you can still compile MPB, but you won't be able to output the fields or the dielectric function.) We require a portable, standard binary format for outputting the electromagnetic fields and similar volumetric data, and for this we use HDF. (If you don't have HDF5, you can still compile MPB, but you won't be able to output the fields or the dielectric function.)
-HDF is a widely-used, free, portable library and file format for multi-dimensional scientific data, developed in the National Center for Supercomputing Applications (NCSA) at the University of Illinois (UIUC, home of the [http://www.fightingillini.com/ Fighting Illini] and ''alma mater'' to many of this author's fine relatives).+{{Installing HDF5}}
- +
-There are two incompatible versions of HDF, HDF4 and HDF5 (no, not HDF1 and HDF2). We require the newer version, HDF5. Many visualization and data-analysis tools still use HDF4, but HDF5 includes an <code>h5toh4</code> conversion program that you can use if you need HDF4 files.+
- +
-HDF5 includes parallel I/O support under MPI, which can be enabled by configuring it with <code>--enable-parallel</code>. (You may also have to set the <code>CC</code> environment variable to <code>mpicc</code>. The resulting HDF5 library, however, may not work with the serial MPB.) This is ''not'' required to use the parallel version of MPB. MPB includes optional code to support this feature, and it may result in faster file I/O in the parallel MPB, but it is currently untested. (Let me know if you try it out; the worst that can happen is that MPB crashes or outputs garbage fields.)+
- +
-You can get HDF and learn about it on the [http://hdf.ncsa.uiuc.edu HDF Home Page]. We have also collected some links to [http://ab-initio.mit.edu/mpb/hdf.html useful HDF software].+
== FFTW == == FFTW ==
FFTW is a self-optimizing, portable, high-performance FFT implementation, including both serial and parallel FFTs. You can download FFTW and find out more about it from the [http://www.fftw.org FFTW Home Page]. FFTW is a self-optimizing, portable, high-performance FFT implementation, including both serial and parallel FFTs. You can download FFTW and find out more about it from the [http://www.fftw.org FFTW Home Page].
- 
-'''Important:''' the current release of MPB requires FFTW 2.x, and will ''not'' compile with FFTW 3.x. 
If you want to use MPB on a parallel machine with MPI, you will also need to install the MPI FFTW libraries (this just means including <code>--enable-mpi</code> in the FFTW <code>configure</code> flags). If you want to use MPB on a parallel machine with MPI, you will also need to install the MPI FFTW libraries (this just means including <code>--enable-mpi</code> in the FFTW <code>configure</code> flags).
Line 112: Line 52:
== GNU Guile == == GNU Guile ==
-GNU Guile is an extension/scripting language implementation based on Scheme, and we use it to provide a rich, fully-programmable user interface with minimal effort. It's free, of course, and you can download it from the [http://www.gnu.org/software/guile/ Guile Home Page]. (Guile is typically included with GNU/Linux systems.)+{{Template:Installing Guile}}
- +
-''Important:'' If you are using an RPM-based Linux system with Guile pre-installed, please note that you must also install the <code>guile-devel</code> RPM (which should be on your CD, but is not installed by default).+
== GNU Autoconf ''(optional)'' == == GNU Autoconf ''(optional)'' ==
Line 122: Line 60:
== libctl == == libctl ==
-Instead of using Guile directly, we separated much of the user interface code into a package called libctl, in the hope that this might be more generally useful. libctl automatically handles the communication between the program and Guile, converting complicated data structures and so on, to make it even easier to use Guile to control scientific applications. Download libctl from the [http://ab-initio.mit.edu/libctl/ libctl home site], unpack it, and run the usual <code>configure</code>, <code>make</code>, <code>make install</code> sequence. You'll also want to browse its [http://ab-initio.mit.edu/libctl/doc/ manual], as this will give you a general overview of what the user interface will be like.+{{Template:Installing libctl}}
-If you are not the system administrator of your machine, and/or want to install libctl somewhere else (like your home directory), you can do so with the standard <code>--prefix=''dir''</code> option to <code>configure</code> (the default prefix is <code>/usr/local</code>). In this case, however, you'll need to specify the location of the libctl shared files for the MPB package, using the <code>--with-libctl=''dir''/share/libctl</code> option to the MPB <code>configure</code> script.+== MIT Photonic Bands ==
-== MIT Photonic-Bands ==+Okay, if you've made it all the way here, you're ready to install the MPB package and start cranking out eigenmodes. (You can download the latest version and read this manual at the [[MIT Photonic Bands|MIT Photonic-Bands Homepage]].) Once you've unpacked it, just run:
- +
-Okay, if you've made it all the way here, you're ready to install the MPB package and start cranking out eigenmodes. (You can download the latest version and read this manual at the [http://ab-initio.mit.edu/mpb/ MIT Photonic-Bands Homepage].) Once you've unpacked it, just run:+
./configure ./configure
make make
-to configure and compile the package (see below to install). Hopefully, the <code>configure</code> script will correctly detect the BLAS, FFTW, etcetera libraries that you've dutifully installed, as well as the C compiler and so on, and the <code>make</code> compilation will proceed without a hitch. If not, it's a [http://www.catb.org/~esr/jargon/html/entry/SMOP.html Simple Matter of Programming] to correct the problem. <code>configure</code> accepts several flags to help control its behavior. Some of these are standard, like <code>--prefix=''dir''</code> to specify and installation directory prefix, and some of them are specific to the MPB package (<code>./configure --help</code> for more info). The <code>configure</code> flags specific to MPB are:+to configure and compile the package (see below to install). Hopefully, the <code>configure</code> script will correctly detect the BLAS, FFTW, etcetera libraries that you've dutifully installed, as well as the C compiler and so on, and the <code>make</code> compilation will proceed without a hitch. If not, it's a [http://www.catb.org/jargon/html/S/SMOP.html Simple Matter of Programming] to correct the problem. <code>configure</code> accepts several flags to help control its behavior. Some of these are standard, like <code>--prefix=''dir''</code> to specify and installation directory prefix, and some of them are specific to the MPB package (<code>./configure --help</code> for more info). The <code>configure</code> flags specific to MPB are:
; <code>--with-inv-symmetry</code> ; <code>--with-inv-symmetry</code>
Line 152: Line 88:
; <code>--with-mpi</code> ; <code>--with-mpi</code>
: Attempt to compile a [user-ref.html#mpb-mpi parallel version of MPB] using MPI; the resulting program will be installed as <code>mpb-mpi</code>. Requires [#mpi MPI] and [#fftw MPI FFTW] libraries to be installed, as described above. : Attempt to compile a [user-ref.html#mpb-mpi parallel version of MPB] using MPI; the resulting program will be installed as <code>mpb-mpi</code>. Requires [#mpi MPI] and [#fftw MPI FFTW] libraries to be installed, as described above.
-Does ''not'' compile the serial MPB, or <code>mpb-data</code><nowiki>; if you want those, you should </nowiki><code>make distclean</code> and compile/install them separately.+: Does ''not'' compile the serial MPB, or <code>mpb-data</code><nowiki>; if you want those, you should </nowiki><code>make distclean</code> and compile/install them separately.
-<code>--with-mpi</code> ''can'' be used along with <code>--with-inv-symmetry</code>, in which case the program is installed as <code>mpbi-mpi</code> (try typing that five times quickly).+:<code>--with-mpi</code> ''can'' be used along with <code>--with-inv-symmetry</code>, in which case the program is installed as <code>mpbi-mpi</code> (try typing that five times quickly).
 +; <code>--with-openmp</code>
 +: Attempt to compile a shared-memory parallel version of MPB using OpenMP; the resulting program will be installed as <code>mpb</code>, and FFTs will use OpenMP parallelism. Requires OpenMP FFTW libraries to be installed.
; <code>--with-libctl=''dir''</code> ; <code>--with-libctl=''dir''</code>
-: If libctl was installed in a nonstandard location (i.e. neither <code>/usr</code> nor <code>/usr/local</code>), you need to specify the location of the libctl directory, <code>''dir''</code>. This is either <code>''prefix''/share/libctl</code>, where <code>''prefix''<code> is the installation prefix of libctl, or the original libctl source code directory. +: If libctl was installed in a nonstandard location (i.e. neither <code>/usr</code> nor <code>/usr/local</code>), you need to specify the location of the libctl directory, <code>''dir''</code>. This is either <code>''prefix''/share/libctl</code>, where <code>''prefix''</code> is the installation prefix of libctl, or the original libctl source code directory.
-</code></code>+
; <code>--with-blas=''lib''</code> ; <code>--with-blas=''lib''</code>
: The <code>configure</code> script automatically attempts to detect accelerated BLAS libraries, like DXML (DEC/Alpha), SCSL and SGIMATH (SGI/MIPS), ESSL (IBM/PowerPC), ATLAS, and PHiPACK. You can, however, force a specific library name to try via <code>--with-blas=''lib''</code>. : The <code>configure</code> script automatically attempts to detect accelerated BLAS libraries, like DXML (DEC/Alpha), SCSL and SGIMATH (SGI/MIPS), ESSL (IBM/PowerPC), ATLAS, and PHiPACK. You can, however, force a specific library name to try via <code>--with-blas=''lib''</code>.
Line 189: Line 126:
make distclean make distclean
 +
 +[[Category:MPB]]

Current revision

MIT Photonic Bands
Download
Release notes
MPB manual
Introduction
Installation
User Tutorial
Data Analysis Tutorial
User Reference
Developer Information
Acknowledgements
License and Copyright

In this section, we outline the procedure for installing the MIT Photonic-Bands package. Mainly, this consists of downloading and installing various prerequisites. As much as possible, we have attempted to take advantage of existing packages such as BLAS, LAPACK, FFTW, and GNU Guile, in order to make our code smaller, more robust, faster, and more flexible. Unfortunately, this may make the installation of MPB more complicated if you do not already have these packages.

You will also need an ANSI C compiler, of course (gcc is fine), and installation will be easiest on a UNIX-like system (Linux is fine). In the following list, some of the packages are dependent upon packages listed earlier, so you should install them in more-or-less the order given.

Note: Many of these libraries may be available in precompiled binary form, especially for GNU/Linux systems. Be aware, however, that library binary packages often come in two parts, library and library-dev, and both are required to compile programs using it.

Note: It is important that you use the same Fortran compiler to compile Fortran libraries (like LAPACK) and for configuring MPB. Different Fortran compilers often have incompatible linking schemes. (The Fortran compiler for MPB can be set via the F77 environment variable.)

Note: The latest, pre-installed versions of MPB and Meep running on Ubuntu can also be accessed on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) as a free Amazon Machine Image (AMI). To access this AMI, follow these instructions.

Contents

Installation on MacOS X

See Meep installation on OS X for the easiest way to do this.

Unix Installation Basics

Installation Paths

First, let's review some important information about installing software on Unix systems, especially in regards to installing software in non-standard locations. (None of these issues are specific to this software, but they've caused a lot of confusion among our beloved users.)

Most of the software below, including this software, installs under /usr/local by default; that is, libraries go in /usr/local/lib, programs in /usr/local/bin, etc. If you don't have root privileges on your machine, you may need to install somewhere else, e.g. under $HOME/install (the install/ subdirectory of your home directory). Most of the programs below use a GNU-style configure script, which means that all you would do to install there would be:

./configure --prefix=$HOME/install

when configuring the program; the directories $HOME/install/lib etc. are created automatically as needed.

Paths for Configuring

There are two further complications. First, if you install in a non-standard location (and /usr/local is considered non-standard by some proprietary compilers), you will need to tell the compilers where to find the libraries and header files that you dutifully installed. You do this by setting two environment variables:

setenv LDFLAGS "-L/usr/local/lib"
setenv CPPFLAGS "-I/usr/local/include"

Of course, substitute whatever installation directory you used. Do this before you run the configure scripts, etcetera. You may need to include multiple -L and -I flags (separated by spaces) if your machine has stuff installed in several non-standard locations. Bourne shell users (e.g. bash or ksh) should use the "export FOO=bar" syntax instead of csh's "setenv FOO bar", of course.

You might also need to update your PATH so that you can run the executables you installed (although /usr/local/bin is in the default PATH on many systems). e.g. if we installed in our home directory as described above, we would do:

setenv PATH "$HOME/install/bin:$PATH"

Paths for Running (Shared Libraries)

Second, many of the packages installed below (e.g. Guile) are installed as shared libraries. You need to make sure that your runtime linker knows where to find these shared libraries. The bad news is that every operating system does this in a slightly different way. The good news is that, when you run make install for the packages involving shared libraries, the output includes the necessary instructions specific to your system, so pay close attention! It will say something like "add LIBDIR to the foobar environment variable," where LIBDIR will be your library installation directory (e.g. /usr/local/lib) and foobar is some environment variable specific to your system (e.g. LD_LIBRARY_PATH on some systems, including Linux). For example, you might do:

setenv LD_LIBRARY_PATH "/usr/local/lib:$LD_LIBRARY_PATH"

Note that we just add to the library path variable, and don't replace it in case it contains stuff already. If you use Linux and have root privileges, you can instead simply run /sbin/ldconfig, first making sure that a line "/usr/local/lib" (or whatever) is in /etc/ld.so.conf.

If you don't want to type these commands every time you log in, you can put them in your ~/.cshrc file (or ~/.profile, or ~/.bash_profile, depending on your shell).

Fun with Fortran

Our software, along with many of the libraries it calls, is written in C or C++, but it also calls libraries such as BLAS and LAPACK (see below) that are usually compiled from Fortran. This can cause some added difficulty because of the various linking schemes used by Fortran compilers. Our configure script attempts to detect the Fortran linking scheme automatically, but in order for this to work you must use the same Fortran compiler and options with our software as were used to compile BLAS/LAPACK.

By default, our software looks for a vendor Fortran compiler first (f77, xlf, etcetera) and then looks for GNU g77. In order to manually specify a Fortran compiler foobar you would configure it with './configure F77=foobar ...'.

If, when you compiled BLAS/LAPACK, you used compiler options that alter the linking scheme (e.g. g77's -fcase-upper or -fno-underscoring), you will need to pass the same flags to our software via './configure FFLAGS="...flags..." ...'.

Picking a compiler

It is often important to be consistent about which compiler you employ. This is especially true for C++ software. To specify a particular C compiler foo, configure with ./configure CC=foo; to specify a particular C++ compiler foo++, configure with ./configure CXX=foo++; to specify a particular Fortran compiler foo90, configure with ./configure F77=foo90.

GNU/Linux and BSD binary packages

If you are installing on your personal GNU/Linux or BSD machine, then precompiled binary packages are likely to be available for many of these packages, and may even have been included with your system. On Debian systems, the packages are in .deb format and the built-in apt-get program can fetch them from a central repository. On Red Hat, SuSE, and most other Linux-based systems, binary packages are in RPM format and can be fetched (if they did not come with your system) from places like rpmfind.net. OpenBSD has its "ports" system, and so on.

Do not compile something from source if an official binary package is available. For one thing, you're just creating pain for yourself. Worse, the binary package may already be installed, in which case installing a different version from source will just cause trouble.

One thing to watch out for is that libraries like LAPACK, Guile, HDF5, etcetera, will often come split up into two (or more) packages: e.g. a guile package and a guile-devel package. You need to install both of these to compile software using the library.

BLAS and LAPACK

MPB requires the BLAS and LAPACK libraries for matrix computations.

BLAS

The first thing you must have on your system is a BLAS implementation. "BLAS" stands for "Basic Linear Algebra Subroutines," and is a standard interface for operations like matrix multiplication. It is designed as a building-block for other linear-algebra applications, and is used both directly by our code and in LAPACK (see below). By using it, we can take advantage of many highly-optimized implementations of these operations that have been written to the BLAS interface. (Note that you will need implementations of BLAS levels 1-3.)

You can find more BLAS information, as well as a basic implementation, on the BLAS Homepage. Once you get things working with the basic BLAS implementation, it might be a good idea to try and find a more optimized BLAS code for your hardware. Vendor-optimized BLAS implementations are available as part of the Intel MKL, HP CXML, IBM ESSL, SGI sgimath, and other libraries. An excellent, high-performance, free-software BLAS implementation is OpenBLAS; another is ATLAS.

Note that the generic BLAS does not come with a Makefile; compile it with something like:

get http://www.netlib.org/blas/blas.tgz
gunzip blas.tgz
tar xf blas.tar
cd BLAS
f77 -c -O3 *.f   # compile all of the .f files to produce .o files
ar rv libblas.a *.o    #  combine the .o files into a library
su -c "cp libblas.a /usr/local/lib"   # switch to root and install

(Replace -O3 with your favorite optimization options. On Linux, I use g77 -O3 -fomit-frame-pointer -funroll-loops, with -malign-double -mcpu=i686 on a Pentium II.) Note that MPB looks for the standard BLAS library with -lblas, so the library file should be called libblas.a and reside in a standard directory like /usr/local/lib. (See also below for the --with-blas=lib option to MPB's configure script, to manually specify a library location.)

LAPACK

LAPACK, the Linear Algebra PACKage, is a standard collection of routines, built on BLAS, for more-complicated (dense) linear algebra operations like matrix inversion and diagonalization. You can download LAPACK from the LAPACK Home Page.

Note that our software looks for LAPACK by linking with -llapack. This means that the library must be called liblapack.a and be installed in a standard directory like /usr/local/lib (alternatively, you can specify another directory via the LDFLAGS environment variable as described earlier). (See also below for the --with-lapack=lib option to our configure script, to manually specify a library location.)

I currently recommend installing OpenBLAS, which includes LAPACK so you do not need to install it separately.

MPI (for parallel MPB)

Optionally, MPB is able to run on a distributed-memory parallel machine, and to do this we use the standard MPI message-passing interface. You can learn about MPI from the MPI Home Page. Most commercial supercomputers already have an MPI implementation installed; two free MPI implementations that you can install yourself are MPICH and LAM; a promising new implementation is Open MPI. MPI is not required to compile the ordinary, uniprocessor version of our software.

In order for the MPI version of our software to run successfully, we have a slightly nonstandard requirement: each process must be able to read from the disk. (This way, Guile can boot for each process and they can all read your control file in parallel.) Many (most?) commercial supercomputers, Linux Beowulf clusters, etcetera, satisfy this requirement.

Also, in order to get good performance, you'll need fast interconnect hardware such as Myrinet; 100Mbps Ethernet probably won't cut it. The speed bottleneck comes from the FFT, which requires most of the communications in MPB. FFTW's MPI transforms (see below) come with benchmark programs that will give you a good idea of whether you can get speedups on your system. Of course, even with slow communications, you can still benefit from the memory savings per CPU for large problems.

As described below, when you configure MPB with MPI support (--with-mpi), it installs itself as mpb-mpi. See also the [user-ref.html#mpb-mpi user reference] section for information on using MPB on parallel machines. Normally, you should also install the serial version of MPB, if only to get the mpb-data utility, which is not installed with the MPI version.

MPI support in MPB is thanks to generous support from Clarendon Photonics.

HDF5 (optional, strongly advised)

We require a portable, standard binary format for outputting the electromagnetic fields and similar volumetric data, and for this we use HDF. (If you don't have HDF5, you can still compile MPB, but you won't be able to output the fields or the dielectric function.)

HDF is a widely-used, free, portable library and file format for multi-dimensional scientific data, developed in the National Center for Supercomputing Applications (NCSA) at the University of Illinois (UIUC, home of the Fighting Illini and alma mater to many of this author's fine relatives). You can get HDF and learn about it on the HDF Home Page.

There are two incompatible versions of HDF, HDF4 and HDF5 (no, not HDF1 and HDF2). We require the newer version, HDF5, which is supported by a number scientific of visualization tools, including our own h5utils utilities.

HDF5 includes parallel I/O support under MPI, which can be enabled by configuring it with --enable-parallel. (You may also have to set the CC environment variable to mpicc.) Unfortunately, the parallel HDF5 library then does not work with serial code, so you have may have to choose one or the other.

We have some hacks in our MPB and Meep software so that they can do parallel I/O even with the serial HDF5 library; these hacks work okay when you are using a small number of processors, but on large supercomputers we strongly recommend using the parallel HDF5.

Note: If you have a version of HDF5 compiled with MPI parallel I/O support, then you need to use the MPI compilers to link to it, even when you are compiling the serial versions of Meep or MPB. Just use ./configure CC=mpicc CXX=mpic++ (or whatever your MPI compilers are) when configuring.

Note: version 1.8 of HDF5 changed the programming interface, breaking older programs like MPB (versions 1.4.x and earlier) and Meep (versions 0.11.x and earlier). As a workaround, if you have HDF5 1.8 or later and would like to compile these versions of MPB and/or Meep, include -DH5_USE_16_API=1 in the CPPFLAGS (see below).

FFTW

FFTW is a self-optimizing, portable, high-performance FFT implementation, including both serial and parallel FFTs. You can download FFTW and find out more about it from the FFTW Home Page.

If you want to use MPB on a parallel machine with MPI, you will also need to install the MPI FFTW libraries (this just means including --enable-mpi in the FFTW configure flags).

GNU Readline (optional)

GNU Readline is a library to provide command-line history, tab-completion, emacs keybindings, and other shell-like niceties to command-line programs. This is an optional package, but one that can be used by Guile (see below) if it is installed; we recommend getting it. You can download Readline from the GNU ftp site. (Readline is typically preinstalled on GNU/Linux systems).

GNU Guile

GNU Guile is an extension/scripting language implementation based on Scheme, and we use it to provide a rich, fully-programmable user interface with minimal effort. It's free, of course, and you can download it from the Guile Home Page. (Guile is typically included with GNU/Linux systems.)

  • Important: Most Linux distributions come with Guile already installed (check by seeing whether you can run guile --version from the command line). In that case, do not install your own version of Guile from source — having two versions of Guile on the same system will cause problems. However, by default most distributions install only the Guile libraries and not the programming headers — to compile libctl and MPB, you should install the "guile-devel" or "guile-dev" package (e.g. with RPM) that came with your system.

GNU Autoconf (optional)

If you want to be a developer of the MPB package (as opposed to merely a user), you will also need the GNU Autoconf program. Autoconf is a portability tool that generates configure scripts to automatically detect the capabilities of a system and configure a package accordingly. You can find out more at the Autoconf Home Page (autoconf is typically installed by default on Linux systems). In order to install Autoconf, you will also need the GNU m4 program if you do not already have it (see the GNU m4 Home Page).

libctl

Instead of using Guile directly, we separated much of the user interface code into a package called libctl, in the hope that this might be more generally useful. libctl automatically handles the communication between the program and Guile, converting complicated data structures and so on, to make it even easier to use Guile to control scientific applications. Download libctl from the libctl page, unpack it, and run the usual configure, make, make install sequence. You'll also want to browse the libctl manual, as this will give you a general overview of what the user interface will be like.

If you are not the system administrator of your machine, and/or want to install libctl somewhere else (like your home directory), you can do so with the standard --prefix=dir option to configure (the default prefix is /usr/local). In this case, however, you'll need to specify the location of the libctl shared files for the MPB or Meep package, using the --with-libctl=dir/share/libctl option to our configure script.

MIT Photonic Bands

Okay, if you've made it all the way here, you're ready to install the MPB package and start cranking out eigenmodes. (You can download the latest version and read this manual at the MIT Photonic-Bands Homepage.) Once you've unpacked it, just run:

./configure
make

to configure and compile the package (see below to install). Hopefully, the configure script will correctly detect the BLAS, FFTW, etcetera libraries that you've dutifully installed, as well as the C compiler and so on, and the make compilation will proceed without a hitch. If not, it's a Simple Matter of Programming to correct the problem. configure accepts several flags to help control its behavior. Some of these are standard, like --prefix=dir to specify and installation directory prefix, and some of them are specific to the MPB package (./configure --help for more info). The configure flags specific to MPB are:

--with-inv-symmetry
Assume [user-ref.html#inv-symmetry inversion symmetry] in the dielectric function, allowing us to use real fields (in Fourier space) instead of complex fields. This gives a factor of 2 benefit in speed and memory. In this case, the MPB program will be installed as mpbi instead of mpb, so that you can have versions both with and without inversion symmetry installed at the same time. To install both mpb and mpbi, you should do:
./configure
make
su -c "make install"
make distclean
./configure --with-inv-symmetry
make
su -c "make install"
--with-hermitian-eps
Support the use of [user-ref.html#dielectric-anisotropic complex-hermitian dielectric tensors] (corresponding to magnetic materials, which break inversion symmetry).
--enable-single
Use single precision (C float) instead of the default double precision (C double) for computations. (Not recommended.)
--without-hdf5
Don't use the HDF5 library for field and dielectric function output. (In which case, no field output is possible.)
--with-mpi
Attempt to compile a [user-ref.html#mpb-mpi parallel version of MPB] using MPI; the resulting program will be installed as mpb-mpi. Requires [#mpi MPI] and [#fftw MPI FFTW] libraries to be installed, as described above.
Does not compile the serial MPB, or mpb-data; if you want those, you should make distclean and compile/install them separately.
--with-mpi can be used along with --with-inv-symmetry, in which case the program is installed as mpbi-mpi (try typing that five times quickly).
--with-openmp
Attempt to compile a shared-memory parallel version of MPB using OpenMP; the resulting program will be installed as mpb, and FFTs will use OpenMP parallelism. Requires OpenMP FFTW libraries to be installed.
--with-libctl=dir
If libctl was installed in a nonstandard location (i.e. neither /usr nor /usr/local), you need to specify the location of the libctl directory, dir. This is either prefix/share/libctl, where prefix is the installation prefix of libctl, or the original libctl source code directory.
--with-blas=lib
The configure script automatically attempts to detect accelerated BLAS libraries, like DXML (DEC/Alpha), SCSL and SGIMATH (SGI/MIPS), ESSL (IBM/PowerPC), ATLAS, and PHiPACK. You can, however, force a specific library name to try via --with-blas=lib.
--with-lapack=lib
Cause the configure script to look for a LAPACK library called lib (the default is to use -llapack).
--disable-checks
Disable runtime checks. (Not recommended; the disabled checks shouldn't take up a significant amount of time anyway.)
--enable-prof
Compile for performance profiling.
--enable-debug
Compile for debugging, adding extra runtime checks and so on.
--enable-debug-malloc
Use special memory-allocation routines for extra debugging (to check for array overwrites, memory leaks, etcetera).
--with-efence
More debugging: use the Electric Fence library, if available, for extra runtime array bounds-checking.

You can further control configure by setting various environment variables, such as:

  • CC: the C compiler command
  • CFLAGS: the C compiler flags (defaults to -O3).
  • CPPFLAGS: -Idir flags to tell the C compiler additional places to look for header files.
  • LDFLAGS: -Ldir flags to tell the linker additional places to look for libraries.
  • LIBS: additional libraries to link against.

Once compiled, the main program (as opposed to various test programs) resides in the mpb-ctl/ subdirectory, and is called mpb. You can install this program under /usr/local (or elsewhere, if you used the --prefix flag for configure), by running:

su -c "make install"

The "su" command is to switch to root for installation into system directories. You can just do make install if you are installing into your home directory instead.

If you make a mistake (e.g. you forget to specify a needed -Ldir flag) or in general want to start over from a clean slate, you can restore MPB to a pristine state by running:

make distclean
Personal tools