View source for NetCDF-Parallel on an Apple Silicon Mac

= Building a parallel NetCDF-4 stack on macOS (Apple Silicon) with cut-and-paste =

This page builds '''Parallel HDF5 → NetCDF-C → NetCDF-Fortran → NetCDF-C++4''' from source with Open MPI and installs everything into <code>/opt/parallel-netcdf</code>. Every step is a block of commands to copy and paste into a terminal, followed by what you should see.  This motivated by the fact the OPEN-MPI on a Mac is now typically compiled with clang instead of gcc.

The current package versions tested:
{| class="wikitable"
! Package !! Version !! Source
|-
| HDF5 || 2.2.0 || GitHub release (HDFGroup/hdf5)
|-
| NetCDF-C || 4.10.1 || downloads.unidata.ucar.edu
|-
| NetCDF-Fortran || 4.6.4 || downloads.unidata.ucar.edu
|-
| NetCDF-C++4 || 4.3.1 || downloads.unidata.ucar.edu
|}

''Last verified: 2026-10-15, macOS 26 (Darwin 25.6), Apple Silicon, 18 cores, Homebrew Open MPI 5.0.11.'' Total time is about 10–12 minutes, plus under a minute of downloads.

== Before you start ==

* Use '''one terminal window for the whole page'''. Step 0 sets variables that every later step needs. If you close the terminal or open a new one, paste the Step 0 block again before continuing.
* The commands are written for '''bash'''. macOS terminals start <code>zsh</code> by default, so type <code>bash</code> first and press Enter.
* Paste one block at a time and compare the result with the '''Expected''' box before moving on. If something differs, stop and see [[#Troubleshooting|Troubleshooting]].

== Prerequisites ==

Install the compilers, Open MPI and build tools with Homebrew:

<pre>
brew install gcc openmpi zlib ccache wget cmake
</pre>

'''Expected:''' Homebrew finishes without errors. Check that the MPI compiler wrappers are found:

<pre>
which mpicc mpicxx mpif90
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' three paths under <code>/opt/homebrew/bin/</code>.
</div>

== Step 0 – Set up the environment ==

=== Check for newer versions (optional) ===

This lists the latest release of each package on GitHub and checks whether that tarball can actually be downloaded. The download check matters for the NetCDF libraries: they are hosted by Unidata, which sometimes publishes a release later than GitHub. Nothing is changed. If you have the <code>NetCDFlib</code> directory, <code>./check_versions.sh</code> there does the same thing and also compares against the versions set in the scripts.

<pre>
latest() { curl -fsS --max-time 20 "https://api.github.com/repos/$1/releases/latest" | sed -n 's/.*"tag_name": *"\([^"]*\)".*/\1/p' | sed -e 's/^v//' -e 's/^hdf5_//'; }
avail() { curl -fsIL --max-time 20 -o /dev/null "$1" && echo "yes" || echo "NOT FOUND"; }
v=$(latest HDFGroup/hdf5);         echo "HDF5            latest $v   downloadable: $(avail https://github.com/HDFGroup/hdf5/releases/download/$v/hdf5-$v.tar.gz)"
v=$(latest Unidata/netcdf-c);      echo "NetCDF-C        latest $v   downloadable: $(avail https://downloads.unidata.ucar.edu/netcdf-c/$v/netcdf-c-$v.tar.gz)"
v=$(latest Unidata/netcdf-fortran); echo "NetCDF-Fortran  latest $v   downloadable: $(avail https://downloads.unidata.ucar.edu/netcdf-fortran/$v/netcdf-fortran-$v.tar.gz)"
v=$(latest Unidata/netcdf-cxx4);   echo "NetCDF-C++      latest $v   downloadable: $(avail https://downloads.unidata.ucar.edu/netcdf-cxx/$v/netcdf-cxx4-$v.tar.gz)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected''' (as of the "Last verified" date):<br/>
<code>HDF5            latest 2.2.0   downloadable: yes</code><br/>
<code>NetCDF-C        latest 4.10.1   downloadable: yes</code><br/>
<code>NetCDF-Fortran  latest 4.6.4   downloadable: yes</code><br/>
<code>NetCDF-C++      latest 4.3.1   downloadable: yes</code><br/>
If <code>latest</code> is empty, the GitHub query failed (it allows 60 unauthenticated requests per hour); try again later.
</div>

Compare these with the <code>export ..._VER</code> lines in the next block. If a newer version is shown and is downloadable, you can edit its line before pasting. Be aware that this page was only verified with the versions in the table at the top: a newer version may need different options or produce different test counts, so compare carefully against the '''Expected''' boxes.

=== Create the install directory (first time only) ===

The libraries are installed into <code>/opt/parallel-netcdf</code>, which needs administrator rights (<code>sudo</code>) to create. This is the only command on the page that uses <code>sudo</code>. Paste it '''on its own''': <code>sudo</code> asks for your password, and anything pasted after it would be read as the password instead of being run. Type your password (nothing is shown while you type) and press Enter.

<pre>
sudo mkdir -p /opt/parallel-netcdf && sudo chown -R $(whoami):admin /opt/parallel-netcdf && ls -ld /opt/parallel-netcdf
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' one line such as <code>drwxr-xr-x  2 yourname  admin  64 Oct 15 10:00 /opt/parallel-netcdf</code>, showing your user name. If the directory already exists and is owned by you, you can skip this block; running it again does no harm.
</div>

=== Set the environment ===

This defines the versions, directories and compilers used by every later step. It doesn't use <code>sudo</code>, so it can be pasted again at any time.

<code>ROOT</code> is the base directory for the downloaded source archives (<code>$ROOT/netcdf_archives</code>) and the temporary build tree (<code>$ROOT/netcdf_parallel_scratch</code>). It is set to your home directory; to use another location (for example a larger disk), change the first line before pasting. The installation itself always goes to <code>/opt/parallel-netcdf</code>.

<pre>
export ROOT=$HOME

export HDF5_VER="2.2.0"
export NETCDF_C_VER="4.10.1"
export NETCDF_F_VER="4.6.4"
export NETCDF_CXX_VER="4.3.1"

export ARCHIVE_DIR="$ROOT/netcdf_archives"
export PREFIX="/opt/parallel-netcdf"
export TMP_BUILD_DIR="$ROOT/netcdf_parallel_scratch"
export HOMEBREW_PREFIX="/opt/homebrew"
export NUM_CORES=$(sysctl -n hw.ncpu)

export CC=mpicc
export CXX=mpicxx
export FC=mpif90
export F77=mpif77

export OMPI_MCA_io=romio341

export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib"
export PATH="$PREFIX/bin:$PATH"

mkdir -p "$ARCHIVE_DIR" "$TMP_BUILD_DIR"
[ -w "$PREFIX" ] || echo "ERROR: $PREFIX is missing or not writable by you; run the 'Create the install directory' block above"

echo "ROOT=$ROOT  PREFIX=$PREFIX  cores=$NUM_CORES  HDF5=$HDF5_VER  C=$NETCDF_C_VER  F=$NETCDF_F_VER  CXX=$NETCDF_CXX_VER"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' one line such as <code>ROOT=/Users/yourname  PREFIX=/opt/parallel-netcdf  cores=18  HDF5=2.2.0  C=4.10.1  F=4.6.4  CXX=4.3.1</code>, and no <code>ERROR:</code> line.
</div>

<code>OMPI_MCA_io=romio341</code> is not optional: without it, Open MPI's default I/O layer can hang during large parallel writes on macOS (see [[#Troubleshooting|Troubleshooting]]).

To check later that the environment is still set (for example after a break), run:

<pre>
echo "PREFIX=$PREFIX  cores=$NUM_CORES  HDF5=$HDF5_VER"
</pre>

If <code>PREFIX=</code> is empty, paste the "Set the environment" block again.

== Step 1 – Download the sources ==

These must be the '''official release tarballs'''. The "Source code" archives that GitHub generates automatically lack the <code>configure</code> script and will not build. The NetCDF libraries are downloaded from Unidata because their GitHub releases have no downloadable files.

<pre>
cd "$ARCHIVE_DIR"
curl -fL --retry 3 -o "hdf5-${HDF5_VER}.tar.gz" "https://github.com/HDFGroup/hdf5/releases/download/${HDF5_VER}/hdf5-${HDF5_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-c-${NETCDF_C_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-c/${NETCDF_C_VER}/netcdf-c-${NETCDF_C_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-fortran-${NETCDF_F_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-fortran/${NETCDF_F_VER}/netcdf-fortran-${NETCDF_F_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-cxx/${NETCDF_CXX_VER}/netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz"
</pre>

Check that all four files are intact:

<pre>
cd "$ARCHIVE_DIR"
gzip -t "hdf5-${HDF5_VER}.tar.gz" && echo "hdf5 OK"
gzip -t "netcdf-c-${NETCDF_C_VER}.tar.gz" && echo "netcdf-c OK"
gzip -t "netcdf-fortran-${NETCDF_F_VER}.tar.gz" && echo "netcdf-fortran OK"
gzip -t "netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" && echo "netcdf-cxx4 OK"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' four <code>OK</code> lines. A <code>curl: (22) ... 404</code> error means that version is not available at that URL; check the version numbers in Step 0.
</div>

== Step 2 – Parallel HDF5 (about 3 minutes) ==

HDF5 2.x is built with CMake.

=== Extract ===

<pre>
rm -rf "$TMP_BUILD_DIR/hdf5-${HDF5_VER}"
tar -xzf "$ARCHIVE_DIR/hdf5-${HDF5_VER}.tar.gz" -C "$TMP_BUILD_DIR"
mkdir -p "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
</pre>

=== Configure ===

<pre>
cmake .. \
  -DCMAKE_INSTALL_PREFIX="$PREFIX" \
  -DCMAKE_INSTALL_NAME_DIR="$PREFIX/lib" \
  -DHDF5_ENABLE_PARALLEL=ON \
  -DHDF5_BUILD_FORTRAN=ON \
  -DHDF5_ENABLE_ZLIB_SUPPORT=ON \
  -DZLIB_INCLUDE_DIR="$HOMEBREW_PREFIX/opt/zlib/include" \
  -DZLIB_LIBRARY="$HOMEBREW_PREFIX/opt/zlib/lib/libz.dylib" \
  -DBUILD_SHARED_LIBS=ON \
  -DBUILD_TESTING=ON \
  -DHDF5_TEST_TOOLS=OFF \
  -DHDF5_TEST_SWMR=OFF \
  -DHDF5_TEST_EXAMPLES=OFF \
  -DMPIEXEC_MAX_NUMPROCS=4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends with <code>-- Build files have been written to: …/build</code>
</div>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends at <code>[100%]</code> with no <code>Error</code> lines.
</div>

=== Test (about 1.5 minutes) ===

Paste this block '''on its own''' and wait for it to finish. The MPI tests read from the terminal, so commands pasted ahead of time would be swallowed and never run.

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
ctest -j"$NUM_CORES" --output-on-failure 2>&1 | tee ctest.log | tail -5
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>100% tests passed out of 826</code>
</div>

=== Install ===

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
make install
otool -D "$PREFIX/lib/libhdf5_hl.dylib"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' the last line is <code>/opt/parallel-netcdf/lib/libhdf5_hl.320.dylib</code>. If it starts with <code>@rpath/</code> instead, the <code>CMAKE_INSTALL_NAME_DIR</code> setting was lost; redo this step.
</div>

== Step 3 – NetCDF-C (about 5–6 minutes) ==

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-c-${NETCDF_C_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
export LIBS="-lhdf5_hl -lhdf5 -lz"
./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --enable-netcdf-4 --enable-parallel4 --disable-dap --disable-parallel-tests
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends with a "NetCDF C Configuration Summary" that includes <code>HDF5 Support: yes</code> and <code>NC-4 Parallel Support: yes</code>.
</div>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make -j"$NUM_CORES"
</pre>

=== Test (about 5 minutes) ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 240   FAIL: 0</code>. A much smaller PASS count means testing stopped early at a failure; run <code>grep -E '^(FAIL|ERROR):' check.log</code> to see which one.
</div>

=== Install and verify ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make install
unset LIBS
nc-config --has-parallel4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>yes</code>. If it prints <code>no</code>, NetCDF-C was built without parallel support: stop here.
</div>

== Step 4 – NetCDF-Fortran (about 2 minutes) ==

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-fortran-${NETCDF_F_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"
./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --disable-parallel-tests
</pre>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' many <code>Warning: Type mismatch …</code> / <code>Rank mismatch …</code> messages from gfortran. These are normal for NetCDF-Fortran; only <code>Error</code> lines matter.
</div>

=== Test ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 59   FAIL: 0</code>
</div>

=== Install and verify ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make install
unset LIBS
nf-config --has-nc4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>yes</code>
</div>

== Step 5 – NetCDF-C++4 (under 1 minute) ==

NetCDF-C++ needs two extra commands that remove files named <code>VERSION</code>; the reason is explained in [[#Troubleshooting|Troubleshooting]]. Don't skip them.

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
find . -type f \( -name "version" -o -name "VERSION" \) -print -delete
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"
CXXFLAGS="$CPPFLAGS" ./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --disable-filter-testing
rm -f VERSION
</pre>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' no <code>error:</code> lines. If you see <code>./VERSION:1:1: error: expected unqualified-id</code>, the <code>rm -f VERSION</code> line was skipped: redo this step from "Extract and configure".
</div>

=== Test ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 16   FAIL: 0</code>
</div>

=== Install ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make install
unset LIBS
ls "$PREFIX/lib/libnetcdf_c++4.dylib"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>/opt/parallel-netcdf/lib/libnetcdf_c++4.dylib</code>
</div>

== Step 6 – Final check ==

This confirms that the installed tools run without any special library path settings, and that no installed file depends on <code>@rpath</code>.

<pre>
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/h5dump" --version
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/nc-config" --version
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/nf-config" --has-nc4
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/ncdump" 2>&1 | head -1
otool -L "$PREFIX"/lib/*.dylib "$PREFIX"/bin/* 2>/dev/null | grep -c '@rpath'
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''
<code>h5dump: Version 2.2.0</code>, <code>netCDF 4.10.1</code>, <code>yes</code>, a line starting with <code>/opt/parallel-netcdf/bin/ncdump [-c|-h]</code>, and finally <code>0</code>.
</div>

The scratch build directory is no longer needed. To free the disk space:

<pre>
rm -rf "$TMP_BUILD_DIR"
</pre>

== Shell profile ==

Add these lines to your shell profile (<code>~/.bash_profile</code> for bash, <code>~/.zshrc</code> for zsh) so programs can find the tools:

<pre>
export PATH="/opt/parallel-netcdf/bin:$PATH"
export OMPI_MCA_io=romio341
</pre>

* <code>PATH</code> is required.
* <code>OMPI_MCA_io</code> is '''possibly required, but untested with model runs''': it avoids the Open MPI hang described below. It was needed for the HDF5 parallel tests to pass.
* <code>DYLD_LIBRARY_PATH</code> is '''not''' needed: the libraries record their full install paths.

== Troubleshooting ==

; <code>curl: (22) The requested URL returned error: 404</code>
: That version isn't hosted at the URL. Unidata only keeps recent versions (for example, NetCDF-C++ 4.2.1 is gone). Check the latest versions with <code>check_versions.sh</code> in the <code>NetCDFlib</code> directory, or on the GitHub release pages.

; <code>configure: error: cannot run C compiled programs</code>
: The <code>--host=aarch64-apple-darwin</code> option is missing from the <code>./configure</code> line. It makes configure skip a test-run of a compiled program that fails on macOS with the MPI compiler wrappers; the libraries are still built natively.

; <code>dyld: Library not loaded: @rpath/libhdf5_hl.320.dylib</code>
: HDF5 was installed without <code>-DCMAKE_INSTALL_NAME_DIR="$PREFIX/lib"</code>. Redo Step 2, then Steps 3–5 so they link against the corrected libraries. Setting <code>DYLD_LIBRARY_PATH</code> is not a reliable fix: macOS removes it whenever a program is started via <code>/bin/sh</code> or <code>/bin/bash</code> (as <code>make</code>, job scripts and many <code>mpirun</code> setups do).

; A parallel test or model run hangs after <code>mca_fbtl_posix_ipwritev: error in aio_write(): Resource temporarily unavailable</code>
: <code>OMPI_MCA_io=romio341</code> is not set. Open MPI's default I/O layer (OMPIO) issues more asynchronous writes than macOS allows per process, then waits forever. ROMIO avoids this. Set the variable (Step 0 and your shell profile), or run <code>mpirun --mca io romio341 …</code>.

; NetCDF-C++: <code>./VERSION:1:1: error: expected unqualified-id</code> (or <code>expected external declaration</code>)
: The standard C++ headers include a header called <code>&lt;version&gt;</code>. On macOS the file system ignores case and the build searches the source directory for headers, so the compiler picks up the package's own <code>VERSION</code> file instead. NetCDF-C++'s <code>configure</code> creates that file, which is why Step 5 deletes it after configure. Never create <code>version</code>/<code>VERSION</code> files in that tree.

; NetCDF-C++: <code>Undefined symbols … _nc_set_log_level</code>
: <code>--disable-filter-testing</code> is missing from the configure line. Without it, an example program is built that needs a NetCDF-C function which only exists when NetCDF-C was built with logging enabled.

; <code>make check</code> shows far fewer passes than expected
: <code>make check</code> stops at the first directory with a failure, so the remaining tests never run. Look at the first <code>FAIL:</code> line in <code>check.log</code> and the matching <code>.log</code> file next to that test.

= Building a parallel NetCDF-4 stack on macOS (Apple Silicon) with the build scripts =

This page builds '''Parallel HDF5 → NetCDF-C → NetCDF-Fortran → NetCDF-C++4''' from source with Open MPI and installs everything into <code>/opt/parallel-netcdf</code>, using a set of build scripts. The scripts are included on this page; you save them to files first. Every step is a command to copy and paste into a terminal, followed by what you should see. (The first section on this page does the same build by pasting the individual commands instead of running the scripts.)

{| class="wikitable"
! Package !! Version !! Source
|-
| HDF5 || 2.2.0 || GitHub release (HDFGroup/hdf5)
|-
| NetCDF-C || 4.10.1 || downloads.unidata.ucar.edu
|-
| NetCDF-Fortran || 4.6.4 || downloads.unidata.ucar.edu
|-
| NetCDF-C++4 || 4.3.1 || downloads.unidata.ucar.edu
|}

''Last verified: 2026-10-15, macOS 26 (Darwin 25.6), Apple Silicon, 18 cores, Homebrew Open MPI 5.0.11.'' Total time is about 10–12 minutes, plus under a minute of downloads.

== Before you start ==

* The scripts are bash scripts but can be started from any shell, including the default <code>zsh</code>.
* All commands after [[#Create the scripts|Create the scripts]] are run from the directory that holds the scripts (<code>$HOME/NetCDFlib</code>), because the scripts find <code>0_env_setup.sh</code> there. If you open a new terminal, <code>cd "$HOME/NetCDFlib"</code> first.
* Paste '''one command at a time''' and wait for it to finish. Some of them ask for a password or run MPI tests that read from the terminal, which would swallow anything pasted ahead.
* Don't paste the contents of the '''Expected''' boxes; they are output, not commands.

{| class="wikitable"
! File !! What it does
|-
| <code>check_versions.sh</code> || '''Optional''': verifies the version numbers by listing the newest available versions. Changes nothing; the build doesn't need it.
|-
| <code>0_env_setup.sh</code> || Versions, directories, compilers and flags. Read by every other script; you don't run it yourself.
|-
| <code>download_source.sh</code> || Downloads the source archives.
|-
| <code>1_build_hdf5.sh</code> … <code>4_build_netcdf_cxx.sh</code> || Build, test and install one library each, in this order.
|}

== Prerequisites ==

Install the compilers, Open MPI and build tools with Homebrew:

<pre>
brew install gcc openmpi zlib ccache wget cmake
</pre>

'''Expected:''' Homebrew finishes without errors. Check that the MPI compiler wrappers are found:

<pre>
which mpicc mpicxx mpif90
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' three paths under <code>/opt/homebrew/bin/</code>.
</div>

== Create the scripts ==

The scripts are given in full below, one block per file. They all go in one directory, because each script reads <code>0_env_setup.sh</code> from the directory it is run in. Create that directory and go into it:

<pre>
mkdir -p "$HOME/NetCDFlib" && cd "$HOME/NetCDFlib"
</pre>

Then, for '''each''' block below:
# Select the whole block in the browser, from the <code>#!/usr/bin/env bash</code> line to the end, and copy it (Cmd-C).
# In the terminal, '''type''' the <code>pbpaste</code> command given above the block and press Enter. Type it rather than copying it, because copying it would replace the script on the clipboard.

<code>pbpaste</code> writes the clipboard into the file. The script text never passes through the shell; pasting it straight into the terminal would not work.

=== <code>check_versions.sh</code> ===

'''Optional.''' Only needed if you want to verify the version numbers before building: it lists the newest release of each package and whether it can be downloaded (see [[#Check for newer versions (optional)|Step 0]]). It changes nothing, and the build works without it. To skip it, go on to <code>0_env_setup.sh</code>.

Copy the block, then type: <code>pbpaste > check_versions.sh</code>

<pre>
#!/usr/bin/env bash
# Compare the versions pinned in 0_env_setup.sh against the latest upstream
# releases, and check that each latest tarball is actually downloadable from
# the URL download_source.sh would use. Changes nothing; edit 0_env_setup.sh
# by hand to update.
#
# Latest versions come from the GitHub "latest release" API (skips drafts and
# pre-releases). NetCDF tarballs are hosted by Unidata, not GitHub, so a new
# GitHub release may not be downloadable yet.
set -e

# Read the pins without sourcing 0_env_setup.sh (it has side effects: sudo, mkdir)
pin() { sed -n "s/^export $1=\"\(.*\)\"/\1/p" ./0_env_setup.sh; }

# Latest release tag for a GitHub repo, with "v"/"hdf5_" prefixes stripped
latest() {
    curl -fsS --max-time 20 "https://api.github.com/repos/$1/releases/latest" \
        | sed -n 's/.*"tag_name": *"\([^"]*\)".*/\1/p' | sed -e 's/^v//' -e 's/^hdf5_//'
}

# Same URL patterns as download_source.sh; keep the two in sync
url() {
    case "$1" in
        hdf5)           echo "https://github.com/HDFGroup/hdf5/releases/download/$2/hdf5-$2.tar.gz" ;;
        netcdf-c)       echo "https://downloads.unidata.ucar.edu/netcdf-c/$2/netcdf-c-$2.tar.gz" ;;
        netcdf-fortran) echo "https://downloads.unidata.ucar.edu/netcdf-fortran/$2/netcdf-fortran-$2.tar.gz" ;;
        netcdf-cxx4)    echo "https://downloads.unidata.ucar.edu/netcdf-cxx/$2/netcdf-cxx4-$2.tar.gz" ;;
    esac
}

PACKAGES=(
  "hdf5|HDF5_VER|HDFGroup/hdf5"
  "netcdf-c|NETCDF_C_VER|Unidata/netcdf-c"
  "netcdf-fortran|NETCDF_F_VER|Unidata/netcdf-fortran"
  "netcdf-cxx4|NETCDF_CXX_VER|Unidata/netcdf-cxx4"
)

printf "%-15s %-15s %-10s %-10s %-12s %s\n" "PACKAGE" "VARIABLE" "PINNED" "LATEST" "DOWNLOAD" "STATUS"
UPDATES=0
for entry in "${PACKAGES[@]}"; do
    IFS='|' read -r NAME VAR REPO <<< "$entry"
    PINNED=$(pin "$VAR")
    LATEST=$(latest "$REPO" || true)

    if [ -z "$LATEST" ]; then
        printf "%-15s %-15s %-10s %-10s %-12s %s\n" "$NAME" "$VAR" "$PINNED" "?" "-" "GitHub query failed (rate limit?)"
        continue
    fi

    if curl -fsIL --max-time 20 -o /dev/null "$(url "$NAME" "$LATEST")"; then DL="yes"; else DL="NOT FOUND"; fi

    if [ "$PINNED" = "$LATEST" ]; then
        STATUS="up to date"
    else
        STATUS="update available"
        UPDATES=1
    fi
    printf "%-15s %-15s %-10s %-10s %-12s %s\n" "$NAME" "$VAR" "$PINNED" "$LATEST" "$DL" "$STATUS"
done

if [ "$UPDATES" -ne 0 ]; then
    echo ""
    echo "To update: edit the version in 0_env_setup.sh (only if DOWNLOAD is 'yes'),"
    echo "then run ./download_source.sh and rebuild from the changed stage onward."
fi
</pre>

=== <code>0_env_setup.sh</code> ===

Versions, directories, compilers and flags. To change <code>ROOT</code> or a version, save the file as it is and edit it afterwards (see [[#Edit 0_env_setup.sh (optional)|Step 0]]).

Copy the block, then type: <code>pbpaste > 0_env_setup.sh</code>

<pre>
#!/usr/bin/env bash
# ==============================================================================
# CENTRAL PARALLEL NETCDF PROFILE SETUP (APPLE SILICON)
# Sourced by individual module builders to maintain system properties.
# ==============================================================================

# Base directory for the source archives and scratch build tree
export ROOT=$HOME

# Central Software Versions (Defined once)
export HDF5_VER="2.2.0"
export NETCDF_C_VER="4.10.1"
export NETCDF_F_VER="4.6.4"
export NETCDF_CXX_VER="4.3.1"

# Environment Directory Configurations
export ARCHIVE_DIR="$ROOT/netcdf_archives"
export PREFIX="/opt/parallel-netcdf"
export TMP_BUILD_DIR="$ROOT/netcdf_parallel_scratch"
export HOMEBREW_PREFIX="/opt/homebrew"
export NUM_CORES=$(sysctl -n hw.ncpu)

# MPI Toolchain Wrappers
export CC=mpicc
export CXX=mpicxx
export FC=mpif90
export F77=mpif77

# Use ROMIO instead of Open MPI's default OMPIO for MPI-IO. OMPIO's async writes
# exceed macOS's per-process AIO limit (kern.aioprocmax=16) and hang large
# collective writes (HDF5 t_bigio). Model runs need this setting too.
export OMPI_MCA_io=romio341

# Search Paths Linking Homebrew and Local Installations
export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib"
export PATH="$PREFIX/bin:$PATH"
export DYLD_LIBRARY_PATH="$PREFIX/lib:$DYLD_LIBRARY_PATH"

# Setup targeted paths securely
if [ ! -d "$PREFIX" ]; then
    sudo mkdir -p "$PREFIX"
    sudo chown -R $(whoami):admin "$PREFIX"
fi
mkdir -p "$TMP_BUILD_DIR"

echo ">>> Environment Profile Loaded: Targeting $PREFIX with $NUM_CORES Cores."
</pre>

=== <code>download_source.sh</code> ===

Copy the block, then type: <code>pbpaste > download_source.sh</code>

<pre>
#!/usr/bin/env bash
# Download the source tarballs named by the versions in 0_env_setup.sh into
# $ARCHIVE_DIR. Archives already present and intact are skipped.
#
# These must be official release tarballs (with a generated ./configure);
# GitHub's auto-generated "Source code" archives lack it. GitHub hosts no
# release assets for the NetCDF repos, so those come from Unidata.
set -e
source ./0_env_setup.sh

DOWNLOADS=(
  "hdf5-${HDF5_VER}.tar.gz|https://github.com/HDFGroup/hdf5/releases/download/${HDF5_VER}/hdf5-${HDF5_VER}.tar.gz"
  "netcdf-c-${NETCDF_C_VER}.tar.gz|https://downloads.unidata.ucar.edu/netcdf-c/${NETCDF_C_VER}/netcdf-c-${NETCDF_C_VER}.tar.gz"
  "netcdf-fortran-${NETCDF_F_VER}.tar.gz|https://downloads.unidata.ucar.edu/netcdf-fortran/${NETCDF_F_VER}/netcdf-fortran-${NETCDF_F_VER}.tar.gz"
  "netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz|https://downloads.unidata.ucar.edu/netcdf-cxx/${NETCDF_CXX_VER}/netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz"
)

mkdir -p "$ARCHIVE_DIR"
cd "$ARCHIVE_DIR"

FAILED=0
for entry in "${DOWNLOADS[@]}"; do
    FILE="${entry%%|*}"
    URL="${entry#*|}"

    if [ -f "$FILE" ] && gzip -t "$FILE" 2>/dev/null; then
        echo "  [skip] $FILE already present"
        continue
    fi

    echo "--> Downloading $FILE"
    # -f: fail on HTTP errors instead of saving the error page as the tarball
    # -L: follow redirects (GitHub release assets redirect to a CDN)
    # Download to .part so an interrupted transfer never looks like a real archive.
    if curl -fL --retry 3 --retry-delay 5 --connect-timeout 30 -o "$FILE.part" "$URL" \
       && gzip -t "$FILE.part" 2>/dev/null; then
        mv "$FILE.part" "$FILE"
        echo "  [ok]   $FILE"
    else
        rm -f "$FILE.part"
        echo "  [FAIL] $FILE from $URL"
        FAILED=1
    fi
done

if [ "$FAILED" -ne 0 ]; then
    echo "ERROR: one or more downloads failed (check the version in 0_env_setup.sh exists at the URL)."
    exit 1
fi
echo "==> All source archives present in $ARCHIVE_DIR"
</pre>

=== <code>1_build_hdf5.sh</code> ===

Copy the block, then type: <code>pbpaste > 1_build_hdf5.sh</code>

<pre>
#!/usr/bin/env bash
set -e
source ./0_env_setup.sh

TAR_FILE="hdf5-${HDF5_VER}.tar.gz"
SRC_DIR="hdf5-${HDF5_VER}"
FULL_TAR_PATH="$ARCHIVE_DIR/$TAR_FILE"

echo "========================================="
echo " COMPILING PARALLEL HDF5 v${HDF5_VER}"
echo "========================================="

if [ ! -f "$FULL_TAR_PATH" ]; then echo "Error: Missing $TAR_FILE"; exit 1; fi

# Start from a fresh tree so a stale CMakeCache/test list can't leak in
rm -rf "$TMP_BUILD_DIR/$SRC_DIR"
tar -xzf "$FULL_TAR_PATH" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/$SRC_DIR"

# Test scope: keep core, HL, Fortran and parallel (MPI) tests; skip tools,
# SWMR and examples, which NetCDF does not use. Cap MPI tests at 4 ranks so
# ctest -j does not oversubscribe the cores.
# CMAKE_INSTALL_NAME_DIR gives the installed dylibs absolute install names instead
# of @rpath/..., so NetCDF and model executables find HDF5 without DYLD_LIBRARY_PATH.
mkdir -p build && cd build
cmake .. \
  -DCMAKE_INSTALL_PREFIX="$PREFIX" \
  -DCMAKE_INSTALL_NAME_DIR="$PREFIX/lib" \
  -DHDF5_ENABLE_PARALLEL=ON \
  -DHDF5_BUILD_FORTRAN=ON \
  -DHDF5_ENABLE_ZLIB_SUPPORT=ON \
  -DZLIB_INCLUDE_DIR="$HOMEBREW_PREFIX/opt/zlib/include" \
  -DZLIB_LIBRARY="$HOMEBREW_PREFIX/opt/zlib/lib/libz.dylib" \
  -DBUILD_SHARED_LIBS=ON \
  -DBUILD_TESTING=ON \
  -DHDF5_TEST_TOOLS=OFF \
  -DHDF5_TEST_SWMR=OFF \
  -DHDF5_TEST_EXAMPLES=OFF \
  -DMPIEXEC_MAX_NUMPROCS=4

make -j"$NUM_CORES"

echo "--> Executing isolated Parallel HDF5 Unit Tests..."
ctest -j"$NUM_CORES" --output-on-failure || echo "Warning: Minor HDF5 test discrepancies skipped."

make install
echo "==> SUCCESSFULLY INSTALLED PARALLEL HDF5"
</pre>

=== <code>2_build_netcdf_c.sh</code> ===

Copy the block, then type: <code>pbpaste > 2_build_netcdf_c.sh</code>

<pre>
#!/usr/bin/env bash
set -e
source ./0_env_setup.sh

TAR_FILE="netcdf-c-${NETCDF_C_VER}.tar.gz"
SRC_DIR="netcdf-c-${NETCDF_C_VER}"
FULL_TAR_PATH="$ARCHIVE_DIR/$TAR_FILE"

echo "========================================="
echo " COMPILING NETCDF-C v${NETCDF_C_VER}"
echo "========================================="

if [ ! -f "$FULL_TAR_PATH" ]; then echo "Error: Missing $TAR_FILE"; exit 1; fi

# Start from a fresh tree so a stale build/config can't leak in
rm -rf "$TMP_BUILD_DIR/$SRC_DIR"
tar -xzf "$FULL_TAR_PATH" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/$SRC_DIR"

export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include $CPPFLAGS"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib $LDFLAGS"
export LIBS="-lhdf5_hl -lhdf5 -lz"

./configure   --prefix="$PREFIX"   --host=aarch64-apple-darwin   --enable-netcdf-4   --enable-parallel4   --disable-dap   --disable-parallel-tests

make -j"$NUM_CORES"

echo "--> Executing isolated NetCDF-C Engine Unit Tests..."
DYLD_LIBRARY_PATH="$PREFIX/lib:$HOMEBREW_PREFIX/opt/zlib/lib:$DYLD_LIBRARY_PATH" make check || echo "Warning: Transient testing checks bypassed safely."

make install
unset LIBS

echo "--> Running Component Check Verification..."
if [ "$($PREFIX/bin/nc-config --has-parallel4)" = "yes" ]; then
    echo "==> SUCCESSFULLY INSTALLED PARALLEL NETCDF-C"
else
    echo "CRITICAL ERROR: NetCDF-C compiled but missing parallel-4 hooks."; exit 1
fi
</pre>

=== <code>3_build_netcdf_fortran.sh</code> ===

Copy the block, then type: <code>pbpaste > 3_build_netcdf_fortran.sh</code>

<pre>
#!/usr/bin/env bash
set -e
source ./0_env_setup.sh

TAR_FILE="netcdf-fortran-${NETCDF_F_VER}.tar.gz"
SRC_DIR="netcdf-fortran-${NETCDF_F_VER}"
FULL_TAR_PATH="$ARCHIVE_DIR/$TAR_FILE"

echo "========================================="
echo " COMPILING NETCDF-FORTRAN v${NETCDF_F_VER}"
echo "========================================="

if [ ! -f "$FULL_TAR_PATH" ]; then echo "Error: Missing $TAR_FILE"; exit 1; fi

# Start from a fresh tree so a stale build/config can't leak in
rm -rf "$TMP_BUILD_DIR/$SRC_DIR"
tar -xzf "$FULL_TAR_PATH" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/$SRC_DIR"

export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include $CPPFLAGS"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib $LDFLAGS"
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"

./configure   --prefix="$PREFIX"   --host=aarch64-apple-darwin   --disable-parallel-tests

make -j"$NUM_CORES"

echo "--> Executing isolated NetCDF-Fortran Bindings Unit Tests..."
DYLD_LIBRARY_PATH="$PREFIX/lib:$HOMEBREW_PREFIX/opt/zlib/lib:$DYLD_LIBRARY_PATH" make check || echo "Warning: Transient Fortran checks bypassed safely."

make install
unset LIBS

echo "--> Running Component Check Verification..."
if [ "$($PREFIX/bin/nf-config --has-nc4)" = "yes" ]; then
    echo "==> SUCCESSFULLY INSTALLED NETCDF-FORTRAN"
else
    echo "CRITICAL ERROR: NetCDF-Fortran compiled but failed package association."; exit 1
fi
</pre>

=== <code>4_build_netcdf_cxx.sh</code> ===

Copy the block, then type: <code>pbpaste > 4_build_netcdf_cxx.sh</code>

<pre>
#!/usr/bin/env bash
set -e
source ./0_env_setup.sh

TAR_FILE="netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz"
SRC_DIR="netcdf-cxx4-${NETCDF_CXX_VER}"
FULL_TAR_PATH="$ARCHIVE_DIR/$TAR_FILE"

echo "========================================="
echo " COMPILING NETCDF-C++4 v${NETCDF_CXX_VER}"
echo "========================================="

if [ ! -f "$FULL_TAR_PATH" ]; then echo "Error: Missing $TAR_FILE"; exit 1; fi

# Hard-nuke the scratch directory to clear any cached malformed objects
echo "--> Cleaning up old build workspace..."
rm -rf "$TMP_BUILD_DIR/$SRC_DIR"

tar -xzf "$FULL_TAR_PATH" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/$SRC_DIR"

# No file named version/VERSION may exist in the tree: libc++ headers do
# #include <version>, and with -I. on case-insensitive APFS the compiler picks
# up such a file instead of the standard header ("expected unqualified-id").
echo "--> Removing any version/VERSION files that would shadow <version>..."
find . -type f \( -name "version" -o -name "VERSION" \) -print -delete

# Enforce clean compiler and link search environments
export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include $CPPFLAGS"
export CXXFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include $CXXFLAGS"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib $LDFLAGS"
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"

# --host=aarch64-apple-darwin : Bypasses macOS C++ interactive sandbox check execution blocks
# --disable-filter-testing : skips the plugin example, which calls nc_set_log_level
#                            (only exists if NetCDF-C was built with --enable-logging)
./configure \
  --prefix="$PREFIX" \
  --host=aarch64-apple-darwin \
  --disable-filter-testing

# configure itself writes VERSION (configure.ac: echo -n ... >VERSION); remove it
# again so it cannot shadow <version>. Nothing in the build reads it.
rm -f VERSION

make -j"$NUM_CORES"

echo "--> Executing isolated NetCDF-C++4 Runtime Unit Tests..."
DYLD_LIBRARY_PATH="$PREFIX/lib:$HOMEBREW_PREFIX/opt/zlib/lib:$DYLD_LIBRARY_PATH" make check || echo "Warning: Transient C++ checks bypassed safely."

make install
unset LIBS CXXFLAGS

echo "====================================================================="
echo "ALL ARCHITECTURAL STACKS BUILT, VERIFIED, AND DEPLOYED!"
echo "Add these configurations to your ~/.zshrc profile file:"
echo "export PATH=\"$PREFIX/bin:\$PATH\""
echo "export DYLD_LIBRARY_PATH=\"$PREFIX/lib:\$DYLD_LIBRARY_PATH\""
echo ""
echo "Possibly also required (untested with model runs): Open MPI's default"
echo "OMPIO layer hangs large parallel writes on macOS; ROMIO avoids this:"
echo "export OMPI_MCA_io=romio341"
echo "====================================================================="
</pre>

=== Check the scripts ===

Make the scripts executable and check that each file has the right contents:

<pre>
chmod +x *.sh; for f in check_versions.sh 0_env_setup.sh download_source.sh 1_build_hdf5.sh 2_build_netcdf_c.sh 3_build_netcdf_fortran.sh 4_build_netcdf_cxx.sh; do [ -f $f ] || { echo "$f  (not created)"; continue; }; printf "%-26s %3s lines  " $f "$(grep -c . $f)"; head -1 $f | tr -d '\n'; bash -n $f && echo "  syntax OK"; done
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected''' (line counts don't include blank lines):<br/>
<code>check_versions.sh</code> 56 lines<br/>
<code>0_env_setup.sh</code> 39 lines<br/>
<code>download_source.sh</code> 44 lines<br/>
<code>1_build_hdf5.sh</code> 39 lines<br/>
<code>2_build_netcdf_c.sh</code> 29 lines<br/>
<code>3_build_netcdf_fortran.sh</code> 29 lines<br/>
<code>4_build_netcdf_cxx.sh</code> 50 lines<br/>
Each line also shows <code>#!/usr/bin/env bash</code> and <code>syntax OK</code>. A wrong line count usually means a block was not copied completely, or <code>pbpaste</code> saved the previously copied block; copy that block again and retype its <code>pbpaste</code> command. A different count for <code>0_env_setup.sh</code> is fine if you have already edited it. If you skipped the optional <code>check_versions.sh</code>, its line says <code>(not created)</code>.
</div>

== Step 0 – Set up ==

=== Check for newer versions (optional) ===

'''This step is optional.''' It only verifies the version numbers: it compares the versions set in <code>0_env_setup.sh</code> with the newest releases and checks that those can be downloaded. It changes nothing. Skip it if you didn't create <code>check_versions.sh</code>, or if you simply want to build the versions in the table at the top.

<pre>
./check_versions.sh
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected''' (as of the "Last verified" date):<br/>
<code>PACKAGE         VARIABLE        PINNED     LATEST     DOWNLOAD     STATUS</code><br/>
<code>hdf5            HDF5_VER        2.2.0      2.2.0      yes          up to date</code><br/>
<code>netcdf-c        NETCDF_C_VER    4.10.1     4.10.1     yes          up to date</code><br/>
<code>netcdf-fortran  NETCDF_F_VER    4.6.4      4.6.4      yes          up to date</code><br/>
<code>netcdf-cxx4     NETCDF_CXX_VER  4.3.1      4.3.1      yes          up to date</code><br/>
<code>GitHub query failed</code> means GitHub's limit of 60 unauthenticated requests per hour was reached; try again later.
</div>

If a line says <code>update available</code> and <code>DOWNLOAD</code> is <code>yes</code>, you can change that version in <code>0_env_setup.sh</code> (next section). This page was only verified with the versions in the table at the top: a newer version may need different options or produce different test counts.

=== Edit <code>0_env_setup.sh</code> (optional) ===

Open <code>0_env_setup.sh</code> in a text editor (for example <code>open -e 0_env_setup.sh</code> for TextEdit, or <code>nano 0_env_setup.sh</code>) if you want to change either of these near the top:

* <code>export ROOT=$HOME</code>: the base directory for the downloaded source archives (<code>$ROOT/netcdf_archives</code>) and the temporary build tree (<code>$ROOT/netcdf_parallel_scratch</code>), for example to use a larger disk. The installation itself always goes to <code>/opt/parallel-netcdf</code>.
* <code>export HDF5_VER=…</code> and the other <code>_VER</code> lines: the package versions.

Change versions only there; every script reads them from that file.

=== Create the install directory (first time only) ===

<code>/opt/parallel-netcdf</code> needs administrator rights (<code>sudo</code>) to create. The scripts would create it themselves, but then the password prompt appears in the middle of a script's output. Doing it here first means no script ever asks for a password. Paste this command '''on its own''', type your password (nothing is shown while you type) and press Enter.

<pre>
sudo mkdir -p /opt/parallel-netcdf && sudo chown -R $(whoami):admin /opt/parallel-netcdf && ls -ld /opt/parallel-netcdf
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' one line such as <code>drwxr-xr-x  2 yourname  admin  64 Oct 15 10:00 /opt/parallel-netcdf</code>, showing your user name. If the directory already exists and is owned by you, you can skip this; running it again does no harm.
</div>

== Step 1 – Download the sources ==

<pre>
./download_source.sh
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' an <code>[ok]</code> (or <code>[skip]</code>, if already downloaded) line for each of the four archives, then <code>==> All source archives present in /Users/yourname/netcdf_archives</code>.
</div>

== Steps 2–5 – Build the libraries ==

Each script deletes its old build directory, extracts the source, configures, builds, runs the library's tests and installs. The output scrolls past on the screen and is also saved to a log file in the scripts directory. After each one, check the log as shown before running the next.

<code>&lt; /dev/null</code> keeps the MPI tests from reading the terminal.

=== Step 2 – Parallel HDF5 (about 3 minutes) ===

<pre>
./1_build_hdf5.sh < /dev/null 2>&1 | tee hdf5.log
</pre>

Check:

<pre>
grep -E 'tests passed|SUCCESSFULLY' hdf5.log
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''<br/>
<code>100% tests passed out of 826</code><br/>
<code>==> SUCCESSFULLY INSTALLED PARALLEL HDF5</code>
</div>

=== Step 3 – NetCDF-C (about 5 minutes) ===

<pre>
./2_build_netcdf_c.sh < /dev/null 2>&1 | tee netcdf_c.log
</pre>

Check:

<pre>
echo "PASS: $(grep -c '^PASS:' netcdf_c.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' netcdf_c.log)"; grep SUCCESSFULLY netcdf_c.log
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''<br/>
<code>PASS: 240   FAIL: 0</code><br/>
<code>==> SUCCESSFULLY INSTALLED PARALLEL NETCDF-C</code><br/>
A much smaller PASS count means testing stopped early at a failure; run <code>grep -E '^(FAIL|ERROR):' netcdf_c.log</code> to see which one.
</div>

=== Step 4 – NetCDF-Fortran (about 2 minutes) ===

<pre>
./3_build_netcdf_fortran.sh < /dev/null 2>&1 | tee netcdf_fortran.log
</pre>

Check:

<pre>
echo "PASS: $(grep -c '^PASS:' netcdf_fortran.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' netcdf_fortran.log)"; grep SUCCESSFULLY netcdf_fortran.log
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''<br/>
<code>PASS: 59   FAIL: 0</code><br/>
<code>==> SUCCESSFULLY INSTALLED NETCDF-FORTRAN</code>
</div>

=== Step 5 – NetCDF-C++4 (under 1 minute) ===

<pre>
./4_build_netcdf_cxx.sh < /dev/null 2>&1 | tee netcdf_cxx.log
</pre>

Check:

<pre>
echo "PASS: $(grep -c '^PASS:' netcdf_cxx.log)   FAIL: $(grep -c -E '^(FAIL|ERROR):' netcdf_cxx.log)"; grep 'ALL ARCHITECTURAL' netcdf_cxx.log
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''<br/>
<code>PASS: 16   FAIL: 0</code><br/>
<code>ALL ARCHITECTURAL STACKS BUILT, VERIFIED, AND DEPLOYED!</code><br/>
The script's closing message also suggests a <code>DYLD_LIBRARY_PATH</code> line for your profile; it isn't needed (see [[#Shell profile|Shell profile]]).
</div>

=== If a step fails ===

The scripts stop at the first build error; the last lines of the output say what went wrong. Test failures do '''not''' stop a script: it prints a line starting with <code>Warning:</code> and installs anyway, which is why the checks above matter. After fixing the cause, rerun only the failed script and the ones after it. If you rebuild HDF5 (Step 2), also rerun Steps 3–5 so they link against the new libraries.

== Step 6 – Final check ==

This confirms that the installed tools run without any special library path settings, and that no installed file depends on <code>@rpath</code>.

<pre>
P=/opt/parallel-netcdf; env -u DYLD_LIBRARY_PATH $P/bin/h5dump --version; env -u DYLD_LIBRARY_PATH $P/bin/nc-config --version; env -u DYLD_LIBRARY_PATH $P/bin/nf-config --has-nc4; env -u DYLD_LIBRARY_PATH $P/bin/ncdump 2>&1 | head -1; otool -L $P/lib/*.dylib $P/bin/* 2>/dev/null | grep -c '@rpath'
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''
<code>h5dump: Version 2.2.0</code>, <code>netCDF 4.10.1</code>, <code>yes</code>, a line starting with <code>/opt/parallel-netcdf/bin/ncdump [-c|-h]</code>, and finally <code>0</code>.
</div>

The scratch build directory is no longer needed. To free the disk space (if you changed <code>ROOT</code>, use that directory instead of <code>$HOME</code>):

<pre>
rm -rf "$HOME/netcdf_parallel_scratch"
</pre>

== Shell profile ==

Add these lines to your shell profile (<code>~/.zshrc</code> for zsh, <code>~/.bash_profile</code> for bash) so programs can find the tools:

<pre>
export PATH="/opt/parallel-netcdf/bin:$PATH"
export OMPI_MCA_io=romio341
</pre>

* <code>PATH</code> is required.
* <code>OMPI_MCA_io</code> is '''possibly required, but untested with model runs''': it avoids the Open MPI hang described below. It was needed for the HDF5 parallel tests to pass.
* <code>DYLD_LIBRARY_PATH</code> is '''not''' needed, even though <code>4_build_netcdf_cxx.sh</code> suggests it: the libraries record their full install paths.

== Troubleshooting ==

; <code>./1_build_hdf5.sh: No such file or directory</code>, or <code>0_env_setup.sh: No such file or directory</code>
: You are not in the scripts directory. Run <code>cd "$HOME/NetCDFlib"</code> first. If the error is <code>Permission denied</code> instead, run <code>chmod +x *.sh</code> there.

; <code>Error: Missing hdf5-2.2.0.tar.gz</code> (or another archive)
: Run <code>./download_source.sh</code> (Step 1) first. If you changed <code>ROOT</code> or a version in <code>0_env_setup.sh</code>, run it again.

; <code>curl: (22) The requested URL returned error: 404</code> and <code>[FAIL]</code> from <code>download_source.sh</code>
: That version isn't hosted at the URL. Unidata only keeps recent versions (for example, NetCDF-C++ 4.2.1 is gone). Run <code>./check_versions.sh</code> to see which versions are available, and fix the version in <code>0_env_setup.sh</code>.

; A script stops at <code>Password:</code>
: <code>/opt/parallel-netcdf</code> doesn't exist, so <code>0_env_setup.sh</code> is creating it with <code>sudo</code>. Type your password, or press Ctrl-C and do [[#Create the install directory (first time only)|Create the install directory]] first.

; <code>Permission denied</code> while installing into <code>/opt/parallel-netcdf</code>
: The directory isn't owned by you. Run the [[#Create the install directory (first time only)|Create the install directory]] command again; its <code>chown</code> fixes the ownership.

; <code>dyld: Library not loaded: @rpath/libhdf5_hl.320.dylib</code>
: The installed HDF5 was built without <code>-DCMAKE_INSTALL_NAME_DIR</code> (for example by an older version of <code>1_build_hdf5.sh</code>). Rerun Steps 2–5 with the current scripts. Setting <code>DYLD_LIBRARY_PATH</code> is not a reliable fix: macOS removes it whenever a program is started via <code>/bin/sh</code> or <code>/bin/bash</code> (as <code>make</code>, job scripts and many <code>mpirun</code> setups do).

; A parallel test or model run hangs after <code>mca_fbtl_posix_ipwritev: error in aio_write(): Resource temporarily unavailable</code>
: <code>OMPI_MCA_io=romio341</code> is not set. Open MPI's default I/O layer (OMPIO) issues more asynchronous writes than macOS allows per process, then waits forever. ROMIO avoids this. <code>0_env_setup.sh</code> sets it for the build; for model runs, set it in your shell profile or run <code>mpirun --mca io romio341 …</code>.

; PASS count much lower than expected
: <code>make check</code> stops at the first directory with a failure, so the remaining tests never run. Look at the first <code>FAIL:</code> line in the log, then at the matching <code>.log</code> file next to that test in <code>$HOME/netcdf_parallel_scratch/</code>.

; Steps 2–5 seem to hang with no output after you pasted several commands at once
: The MPI tests read the terminal and swallowed the extra commands. Press Ctrl-C and rerun the step on its own, with <code>&lt; /dev/null</code> as shown.