Planets - Recent changes [en]

Dummy PCM compilation on an Apple Silicon Mac

Mwolff — Fri, 16 Oct 2026 10:54:28 GMT

Created page with "= Compiling the Dummy PCM (LMDZ.COMMON, no physics) on Apple Silicon with XIOS = This page provides the LMDZ.COMMON arch files needed to build the dynamical core with no phys..."

New page

= Compiling the Dummy PCM (LMDZ.COMMON, no physics) on Apple Silicon with XIOS =

This page provides the LMDZ.COMMON arch files needed to build the dynamical core with no physics (<code>-p nophys</code>), MPI and XIOS output, on macOS with Homebrew gfortran and Homebrew Open MPI. It assumes XIOS has already been built with the <code>CLANG_MACOSX</code> arch files, as described on the XIOS compilation page ([[XIOS on an Apple Silicon Mac]].

The one setting that differs from the stock <code>arch-gfortran</code> files is the C++ library in <code>XIOS_LIB</code> and discussed below for the interested reader - [[#Background: which C++ library libxios.a needs|the background section]].

== Tested environment ==

* macOS 26 (Darwin 25.6.0), Apple Silicon (arm64)
* GNU Fortran 16.2.0 (Homebrew), installed as <code>gfortran</code>
* Apple clang 21 (also what <code>/usr/bin/gcc</code> and <code>/usr/bin/cpp</code> run)
* Open MPI 5.0.11 (Homebrew)
* netCDF-C and netCDF-Fortran compiled from source against Open MPI, installed in <code>/opt/parallel-netcdf</code>
* XIOS built with <code>--arch CLANG_MACOSX</code> (C++ compiled by clang against libc++)

== Directory layout ==

The <code>.path</code> file finds IOIPSL and XIOS relative to the LMDZ.COMMON directory, so the three directories must sit side by side:

<pre>
trunk/
IOIPSL/
LMDZ.COMMON/
XIOS/
</pre>

== Arch files ==

Create these two files in <code>LMDZ.COMMON/arch/</code>. No <code>.env</code> file is needed: <code>makelmdz_fcm</code> uses an empty one if it is missing.

=== arch/arch-CLANG_MACOSX.fcm ===

<pre>
%COMPILER mpif90
%LINK mpif90
%AR ar
%MAKE make
%FPP_FLAGS -P -traditional
%CPP_FLAGS -P
%CPP_DEF LAPACK
%FPP_DEF NC_DOUBLE
%BASE_FFLAGS -c -fdefault-real-8 -fdefault-double-8 -ffree-line-length-none -fno-align-commons -fallow-argument-mismatch -fcray-pointer
%PROD_FFLAGS -O3
%DEV_FFLAGS -O
%DEBUG_FFLAGS -ffpe-trap=invalid,zero,overflow -fbounds-check -g3 -O0 -fstack-protector-all -finit-real=snan -fbacktrace
%C_COMPILER gcc
%C_OPTIM -O3
%MPI_FFLAGS
%OMP_FFLAGS
%BASE_LD
%MPI_LD
%OMP_LD
</pre>

=== arch/arch-CLANG_MACOSX.path ===

<code>nc-config</code> and <code>nf-config</code> must be on your <code>PATH</code>.

<pre>
ROOT=$PWD

NETCDF_INCDIR="$(nc-config --cflags) $(nf-config --fflags)"
NETCDF_LIBDIR="$(nc-config --libs) $(nf-config --flibs)"
NETCDF_LIB=""

IOIPSL_INCDIR="-I$ROOT/../IOIPSL/inc"
IOIPSL_LIBDIR="-L$ROOT/../IOIPSL/lib"
IOIPSL_LIB="-lioipsl"

XIOS_INCDIR="-I$ROOT/../XIOS/inc"
XIOS_LIBDIR="-L$ROOT/../XIOS/lib"
XIOS_LIB="-lxios -lc++"
</pre>

== Building ==

From the <code>LMDZ.COMMON</code> directory:

<pre>
./makelmdz_fcm -arch CLANG_MACOSX -parallel mpi -io xios -d 32x32x12 -p nophys gcm
</pre>

A successful build produces <code>bin/gcm_32x32x12_phynophys_para.e</code>.

== Background: which C++ library libxios.a needs ==

=== Symptom with the stock setting ===

The stock gfortran arch files use <code>XIOS_LIB="-lxios -lstdc++"</code>. With that setting, all Fortran sources compile, and then the final link fails with hundreds of undefined C++ symbols coming from <code>libxios.a</code>:

<pre>
Undefined symbols for architecture arm64:
"std::__1::regex_error::regex_error(std::__1::regex_constants::error_type)", referenced from:
... in libxios.a[366](server.o)
"vtable for std::__1::basic_ios<char, std::__1::char_traits<char>>", referenced from:
xios::CException::CException() in libxios.a[90](exception.o)
...
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
</pre>

=== Two compilers, two C++ standard libraries ===

Two different toolchains build the pieces of this model:

{| class="wikitable"
! Component !! Wrapper !! Underlying compiler !! C++ standard library
|-
| XIOS (C++ core) || <code>mpiCC</code> || Apple '''clang''' || '''libc++''' (LLVM), linked with <code>-lc++</code>
|-
| LMDZ (Fortran) || <code>mpif90</code> || Homebrew '''gfortran''' || none (GCC Fortran runtime)
|}

<code>mpiCC</code> is not a separate C++ wrapper here. macOS filesystems are case-insensitive by default, so <code>mpiCC</code> and <code>mpicc</code> cannot both exist. Homebrew's Open MPI makes <code>/opt/homebrew/bin/mpiCC</code> a symlink to the C wrapper <code>mpicc</code>, which calls Apple <code>clang</code>:

<pre>
$ mpiCC --showme
clang -I/opt/homebrew/Cellar/open-mpi/5.0.11/include -L/opt/homebrew/Cellar/open-mpi/5.0.11/lib -lmpi
</pre>

clang compiles the XIOS <code>.cpp</code> files as C++, because it picks the language from the file extension. Unless it is given <code>-stdlib=libstdc++</code>, it compiles them against '''libc++''', LLVM's C++ standard library. libc++ puts its symbols in the inline namespace <code>std::__1::</code>, which is why every missing symbol in the error starts with <code>std::__1::</code>.

GCC's <code>g++</code> uses a different library, '''libstdc++''' (<code>-lstdc++</code>), with its own symbol names such as <code>std::__cxx11::</code>. The two libraries are '''not interchangeable''' at link time: an object compiled against libc++ can only be linked with libc++.

The gcm is linked by <code>mpif90</code> (gfortran), and gfortran does not add any C++ library on its own. The C++ library that <code>libxios.a</code> needs must be named in <code>XIOS_LIB</code>. <code>-lstdc++</code> is right only when XIOS was compiled with <code>g++</code>. Here it was compiled with clang, so the linker looked for <code>std::__1::</code> symbols in libstdc++, found none, and failed.

The XIOS arch file already makes the same choice for the XIOS executables:

<pre>
%CCOMPILER mpiCC
%LINKER mpif90
%BASE_LD -lc++
</pre>

=== Fix ===

In <code>arch/arch-CLANG_MACOSX.path</code>:

<pre>
- XIOS_LIB="-lxios -lstdc++"
+ XIOS_LIB="-lxios -lc++"
</pre>

=== Checking which library your libxios.a needs ===

From <code>LMDZ.COMMON</code>:

<pre>
nm -C ../XIOS/lib/libxios.a | grep -m1 'std::__1::' # match -> libc++ (-lc++)
nm -C ../XIOS/lib/libxios.a | grep -m1 'std::__cxx11::' # match -> libstdc++ (-lstdc++)
</pre>

The rule: '''<code>XIOS_LIB</code> must name the same C++ library that compiled XIOS.''' If you rebuild XIOS with GCC's <code>g++</code> instead, switch back to <code>-lstdc++</code>. One way to do that rebuild is to set <code>%CCOMPILER mpicxx</code> in the XIOS arch file and <code>export OMPI_CXX=g++-16</code>. Setting <code>OMPI_CXX</code> has no effect on <code>mpiCC</code>, because <code>mpiCC</code> is really <code>mpicc</code>, which reads <code>OMPI_CC</code>.

=== Other notes ===

* '''Preprocessor.''' Unlike the XIOS build, this arch file does not set <code>%FPP</code>, so LMDZ.COMMON's <code>bld.cfg</code> falls back to plain <code>cpp</code>, which is Apple clang's preprocessor on macOS. With <code>-P -traditional</code> this worked for the no-physics build. The <code>cpp-16</code> workaround on the XIOS page is for the <code>##</code> macros in XIOS's own Fortran interface, not for the gcm.
* '''<code>%C_COMPILER gcc</code>''' runs <code>/usr/bin/gcc</code>, which on macOS is Apple clang, not GNU gcc.
* '''Harmless warning:''' <code>ld: warning: ignoring duplicate libraries: '-lnetcdf'</code>. It appears because <code>nc-config --libs</code> and <code>nf-config --flibs</code> both list <code>-lnetcdf</code>.

[[Category:LMDZ]]
[[Category:XIOS]]
[[Category:macOS]]

XIOS on an Apple Silicon Mac

Mwolff — Fri, 16 Oct 2026 10:37:41 GMT

‎Compiling XIOS on macOS (Apple Silicon) with openmpi and clang

XIOS on an Apple Silicon Mac

Mwolff — Fri, 16 Oct 2026 10:14:32 GMT

Created page with "= Compiling XIOS on macOS (Apple Silicon) with Open MPI and Clang = This page provides the three XIOS arch files needed to build XIOS on macOS with Homebrew gfortran and Home..."

New page

= Compiling XIOS on macOS (Apple Silicon) with Open MPI and Clang =

This page provides the three XIOS arch files needed to build XIOS on macOS with Homebrew gfortran and Homebrew Open MPI. In this setup the Open MPI C++ wrapper calls Apple '''clang''', not GCC. Because of that, two settings differ from the stock <code>arch-gfortran</code> files. They are explained in [[#Background: Open MPI uses Clang, not GCC|the background section]] below.

== Tested environment ==

* macOS 26 (Darwin 25.6.0), Apple Silicon (arm64)
* GNU Fortran 16.2.0 (Homebrew), installed as <code>gfortran</code>
* Apple clang 21, which provides <code>/usr/bin/cpp</code> and is called by <code>mpiCC</code>
* Open MPI 5.0.11 (Homebrew)
* netCDF-C 4.10.1 and netCDF-Fortran 4.6.4, compiled from source against Open MPI and installed in <code>/opt/parallel-netcdf</code>

== Arch files ==

Create these three files in the <code>arch/</code> directory of your XIOS source tree.

=== arch/arch-CLANG_MACOSX.env ===

Change <code>NETCDF_DIR</code> to match your netCDF installation.

<pre>
# Environment to compile XIOS on Mac OS X with NETCDF compiled with CLANG
#
# Tested successfully with :
# Mac OS X 25.6.0
# Prerequisite softwares installed with Homebrew
# gfortran : 16.2.0
# openmpi : 5.0.11
#
# Compiled from source with open-mpi
# netcdf : netCDF 4.10.1 netCDF-Fortran 4.6.4
#
#
export NETCDF_DIR=/opt/parallel-netcdf
export NETCDF_INC_DIR=${NETCDF_DIR}/include
export NETCDF_LIB_DIR=${NETCDF_DIR}/lib
</pre>

=== arch/arch-CLANG_MACOSX.fcm ===

<pre>
################################################################################
################### Projet XIOS ###################
################################################################################
#Build environment (2026-10-13):
#
#- macOS 26 (Darwin 25.6.0), arm64
#- GNU Fortran 16.2.0 (Homebrew), installed as `gfortran`
#- Apple clang 21, which provides `/usr/bin/cpp` and is wrapped by `mpiCC`
#- Open MPI 5.0.11 (Homebrew)
## Summary

%CCOMPILER mpiCC
%FCOMPILER mpif90
%LINKER mpif90

%BASE_CFLAGS -w -std=gnu++11
%PROD_CFLAGS -O3 -D BOOST_DISABLE_ASSERTS
%DEV_CFLAGS -g
%DEBUG_CFLAGS -g -O2

%BASE_FFLAGS -D__NONE__ -ffree-line-length-none
%PROD_FFLAGS -O3
%DEV_FFLAGS -g -O2
%DEBUG_FFLAGS -g

%BASE_INC -D__NONE__
%BASE_LD -lc++

%CPP cpp -P -undef
%FPP cpp-16 -P
%MAKE make
</pre>

=== arch/arch-CLANG_MACOSX.path ===

This file gets the netCDF flags from <code>nc-config</code> and <code>nf-config</code>, so both must be on your <code>PATH</code>.

<pre>
NETCDF_INCDIR="$(nc-config --cflags) $(nf-config --fflags)"
NETCDF_LIBDIR="$(nc-config --libs) $(nf-config --flibs)"
NETCDF_LIB=""
</pre>

== Building ==

From the top of the XIOS source tree:

<pre>
./make_xios --arch CLANG_MACOSX --job 8 --full > make_xios.out 2>&1
</pre>

Use <code>--full</code> to rebuild from scratch. <code>make_xios</code> has no <code>--clean</code> option, and it ignores unknown options without any warning. So <code>./make_xios --clean</code> cleans nothing. FCM then keeps reusing old preprocessed files in <code>ppsrc/</code>, and a fix to the preprocessor settings does not take effect.

A successful build produces:
* <code>lib/libxios.a</code>
* <code>bin/xios_server.exe</code>
* <code>bin/generic_testcase.exe</code>

== Background: Open MPI uses Clang, not GCC ==

On Linux, <code>mpif90</code>, <code>mpicc</code> and <code>mpicxx</code> usually all call the GNU compilers. With Homebrew on macOS, Open MPI is built with Apple's toolchain. Only the Fortran wrapper calls GCC:

{| class="wikitable"
! Component !! Wrapper !! Underlying compiler !! C++ standard library
|-
| XIOS (C++ core) || <code>mpiCC</code> || Apple '''clang''' || '''libc++''' (LLVM), linked with <code>-lc++</code>
|-
| XIOS Fortran interface, and the model (e.g. LMDZ / PCM) || <code>mpif90</code> || Homebrew '''gfortran''' || none (GCC Fortran runtime)
|}

This mix causes two problems: one with the preprocessor and one at link time.

=== Why mpiCC is really clang ===

macOS filesystems are case-insensitive by default, so the files <code>mpiCC</code> and <code>mpicc</code> cannot both exist. Homebrew's Open MPI makes <code>/opt/homebrew/bin/mpiCC</code> a symlink to the C wrapper <code>mpicc</code>, which calls Apple <code>clang</code>:

<pre>
$ mpiCC --showme
clang -I/opt/homebrew/Cellar/open-mpi/5.0.11/include -L/opt/homebrew/Cellar/open-mpi/5.0.11/lib -lmpi
</pre>

clang still compiles the XIOS <code>.cpp</code> files as C++, because it picks the language from the file extension. But it compiles them against '''libc++''', LLVM's C++ standard library, not GCC's '''libstdc++'''.

A side effect: <code>OMPI_CXX</code> has no effect on <code>mpiCC</code>, because <code>mpiCC</code> is really <code>mpicc</code>, and <code>mpicc</code> reads <code>OMPI_CC</code>.

=== 1. Link library: <code>-lc++</code> instead of <code>-lstdc++</code> ===

The stock <code>arch-gfortran.fcm</code> sets <code>%BASE_LD -lstdc++</code>. With clang-compiled C++ objects, the link of <code>xios_server.exe</code> fails with errors like:

<pre>
Undefined symbols for architecture arm64:
"std::__1::basic_string<...>::copy(...) const", referenced from: ...
"std::__1::regex_error::regex_error(...)", referenced from: ...
ld: symbol(s) not found for architecture arm64
</pre>

The <code>std::__1::</code> prefix is libc++'s inline namespace. libstdc++ uses different names, such as <code>std::__cxx11::</code>. The two libraries are '''not interchangeable''' at link time: an object compiled against libc++ can only be linked with libc++.

The final link is done by <code>mpif90</code> (gfortran), and gfortran does not add any C++ library on its own. The arch file must name the right one, so <code>%BASE_LD</code> is set to <code>-lc++</code>.

'''This also applies to every program that links against this <code>libxios.a</code>.''' For example, the LMDZ / PCM arch <code>.path</code> file must use:

<pre>
XIOS_LIB="-lxios -lc++"
</pre>

and not <code>-lstdc++</code>. Otherwise the model build compiles all its sources and then fails at the final link with hundreds of undefined <code>std::__1::</code> symbols coming from <code>libxios.a</code>.

To check which C++ library an existing <code>libxios.a</code> needs:

<pre>
nm -C lib/libxios.a | grep -m1 'std::__1::' # match -> libc++ (-lc++)
nm -C lib/libxios.a | grep -m1 'std::__cxx11::' # match -> libstdc++ (-lstdc++)
</pre>

The rule: '''the C++ library named at link time must be the same one that compiled XIOS.''' If you rebuild XIOS with GCC's <code>g++</code> instead (for example <code>%CCOMPILER mpicxx</code> with <code>export OMPI_CXX=g++-16</code>), switch back to <code>-lstdc++</code> in both the XIOS and the model arch files.

=== 2. Fortran preprocessor: <code>cpp-16</code> instead of <code>cpp</code> ===

The XIOS Fortran interface builds its names with a token-pasting macro, defined in <code>src/interface/fortran/xios_fortran_prefix.hpp</code>:

<pre>
#define xios(arg) xios_##arg
#define txios(arg) xios_##arg
</pre>

On macOS, <code>/usr/bin/cpp</code> is Apple clang. It does not expand <code>##</code> in these files, so <code>txios(duration)</code> comes out as <code>xios_##duration</code> instead of <code>xios_duration</code>. Compilation then fails, for example in <code>duration_interface.f90</code>:

<pre>
IMPORT :: xios_##duration
Error: Syntax error in IMPORT statement at (1)
TYPE(xios_##duration), VALUE :: dur
Error: Malformed type-spec at (1)
</pre>

The fix is to use the GNU C preprocessor that comes with Homebrew GCC for Fortran files: <code>%FPP cpp-16 -P</code>. The <code>-16</code> suffix follows the major version of Homebrew GCC. If GCC is upgraded, update this line to match. Check the installed version with:

<pre>
ls /opt/homebrew/bin/cpp-*
</pre>

<code>gfortran -E -P -cpp</code> may also work as the Fortran preprocessor, but it has not been tested.

=== Harmless warning ===

The XIOS build and the model builds may print:

<pre>
ld: warning: ignoring duplicate libraries: '-lnetcdf'
</pre>

It appears because <code>nc-config --libs</code> and <code>nf-config --flibs</code> both list <code>-lnetcdf</code>. You can ignore it.

[[Category:XIOS]]
[[Category:macOS]]

NetCDF-Parallel on an Apple Silicon Mac

Mwolff — Thu, 15 Oct 2026 19:47:20 GMT

‎Building a parallel NetCDF-4 stack on macOS (Apple Silicon) with the build scripts

Show changes

NetCDF-Parallel on an Apple Silicon Mac

Mwolff — Thu, 15 Oct 2026 16:45:05 GMT

Created page with "= Building a parallel NetCDF-4 stack on macOS (Apple Silicon) = This page builds '''Parallel HDF5 → NetCDF-C → NetCDF-Fortran → NetCDF-C++4''' from source with Open MPI..."

New page

= Building a parallel NetCDF-4 stack on macOS (Apple Silicon) =

This page builds '''Parallel HDF5 → NetCDF-C → NetCDF-Fortran → NetCDF-C++4''' from source with Open MPI and installs everything into <code>/opt/parallel-netcdf</code>. Every step is a block of commands to copy and paste into a terminal, followed by what you should see.

It reproduces the build scripts <code>0_env_setup.sh</code> … <code>4_build_netcdf_cxx.sh</code> in the <code>NetCDFlib</code> directory. If the two ever disagree, the scripts are the reference.

{| class="wikitable"
! Package !! Version !! Source
|-
| HDF5 || 2.2.0 || GitHub release (HDFGroup/hdf5)
|-
| NetCDF-C || 4.10.1 || downloads.unidata.ucar.edu
|-
| NetCDF-Fortran || 4.6.4 || downloads.unidata.ucar.edu
|-
| NetCDF-C++4 || 4.3.1 || downloads.unidata.ucar.edu
|}

''Last verified: 2026-10-15, macOS 26 (Darwin 25.6), Apple Silicon, 18 cores, Homebrew Open MPI 5.0.11.'' Total time is about 10–12 minutes, plus under a minute of downloads.

== Before you start ==

* Use '''one terminal window for the whole page'''. Step 0 sets variables that every later step needs. If you close the terminal or open a new one, paste the Step 0 block again before continuing.
* The commands are written for '''bash'''. macOS terminals start <code>zsh</code> by default, so type <code>bash</code> first and press Enter.
* Paste one block at a time and compare the result with the '''Expected''' box before moving on. If something differs, stop and see [[#Troubleshooting|Troubleshooting]].
* Don't paste the contents of the '''Expected''' boxes; they are output, not commands.

== Prerequisites ==

Install the compilers, Open MPI and build tools with Homebrew:

<pre>
brew install gcc openmpi zlib ccache wget cmake
</pre>

'''Expected:''' Homebrew finishes without errors. Check that the MPI compiler wrappers are found:

<pre>
which mpicc mpicxx mpif90
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' three paths under <code>/opt/homebrew/bin/</code>.
</div>

== Step 0 – Set up the environment ==

This defines the versions, directories and compilers used by every later step. It creates <code>/opt/parallel-netcdf</code> the first time, which asks for your password (<code>sudo</code>).

<code>ROOT</code> is the base directory for the downloaded source archives (<code>$ROOT/netcdf_archives</code>) and the temporary build tree (<code>$ROOT/netcdf_parallel_scratch</code>). It is set to your home directory; to use another location (for example a larger disk), change the first line before pasting. The installation itself always goes to <code>/opt/parallel-netcdf</code>.

<pre>
export ROOT=$HOME

export HDF5_VER="2.2.0"
export NETCDF_C_VER="4.10.1"
export NETCDF_F_VER="4.6.4"
export NETCDF_CXX_VER="4.3.1"

export ARCHIVE_DIR="$ROOT/netcdf_archives"
export PREFIX="/opt/parallel-netcdf"
export TMP_BUILD_DIR="$ROOT/netcdf_parallel_scratch"
export HOMEBREW_PREFIX="/opt/homebrew"
export NUM_CORES=$(sysctl -n hw.ncpu)

export CC=mpicc
export CXX=mpicxx
export FC=mpif90
export F77=mpif77

export OMPI_MCA_io=romio341

export CPPFLAGS="-I$PREFIX/include -I$HOMEBREW_PREFIX/opt/zlib/include"
export LDFLAGS="-L$PREFIX/lib -L$HOMEBREW_PREFIX/opt/zlib/lib"
export PATH="$PREFIX/bin:$PATH"

if [ ! -d "$PREFIX" ]; then
sudo mkdir -p "$PREFIX"
sudo chown -R $(whoami):admin "$PREFIX"
fi
mkdir -p "$ARCHIVE_DIR" "$TMP_BUILD_DIR"

echo "ROOT=$ROOT PREFIX=$PREFIX cores=$NUM_CORES HDF5=$HDF5_VER C=$NETCDF_C_VER F=$NETCDF_F_VER CXX=$NETCDF_CXX_VER"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' one line such as <code>ROOT=/Users/yourname PREFIX=/opt/parallel-netcdf cores=18 HDF5=2.2.0 C=4.10.1 F=4.6.4 CXX=4.3.1</code>
</div>

<code>OMPI_MCA_io=romio341</code> is not optional: without it, Open MPI's default I/O layer can hang during large parallel writes on macOS (see [[#Troubleshooting|Troubleshooting]]).

To check later that the environment is still set (for example after a break), run:

<pre>
echo "PREFIX=$PREFIX cores=$NUM_CORES HDF5=$HDF5_VER"
</pre>

If <code>PREFIX=</code> is empty, paste the Step 0 block again.

== Step 1 – Download the sources ==

These must be the '''official release tarballs'''. The "Source code" archives that GitHub generates automatically lack the <code>configure</code> script and will not build. The NetCDF libraries are downloaded from Unidata because their GitHub releases have no downloadable files.

<pre>
cd "$ARCHIVE_DIR"
curl -fL --retry 3 -o "hdf5-${HDF5_VER}.tar.gz" "https://github.com/HDFGroup/hdf5/releases/download/${HDF5_VER}/hdf5-${HDF5_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-c-${NETCDF_C_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-c/${NETCDF_C_VER}/netcdf-c-${NETCDF_C_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-fortran-${NETCDF_F_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-fortran/${NETCDF_F_VER}/netcdf-fortran-${NETCDF_F_VER}.tar.gz"
curl -fL --retry 3 -o "netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" "https://downloads.unidata.ucar.edu/netcdf-cxx/${NETCDF_CXX_VER}/netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz"
</pre>

Check that all four files are intact:

<pre>
cd "$ARCHIVE_DIR"
gzip -t "hdf5-${HDF5_VER}.tar.gz" && echo "hdf5 OK"
gzip -t "netcdf-c-${NETCDF_C_VER}.tar.gz" && echo "netcdf-c OK"
gzip -t "netcdf-fortran-${NETCDF_F_VER}.tar.gz" && echo "netcdf-fortran OK"
gzip -t "netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" && echo "netcdf-cxx4 OK"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' four <code>OK</code> lines. A <code>curl: (22) ... 404</code> error means that version is not available at that URL; check the version numbers in Step 0.
</div>

== Step 2 – Parallel HDF5 (about 3 minutes) ==

HDF5 2.x is built with CMake.

=== Extract ===

<pre>
rm -rf "$TMP_BUILD_DIR/hdf5-${HDF5_VER}"
tar -xzf "$ARCHIVE_DIR/hdf5-${HDF5_VER}.tar.gz" -C "$TMP_BUILD_DIR"
mkdir -p "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
</pre>

=== Configure ===

<pre>
cmake .. \
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DCMAKE_INSTALL_NAME_DIR="$PREFIX/lib" \
-DHDF5_ENABLE_PARALLEL=ON \
-DHDF5_BUILD_FORTRAN=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT=ON \
-DZLIB_INCLUDE_DIR="$HOMEBREW_PREFIX/opt/zlib/include" \
-DZLIB_LIBRARY="$HOMEBREW_PREFIX/opt/zlib/lib/libz.dylib" \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_TESTING=ON \
-DHDF5_TEST_TOOLS=OFF \
-DHDF5_TEST_SWMR=OFF \
-DHDF5_TEST_EXAMPLES=OFF \
-DMPIEXEC_MAX_NUMPROCS=4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends with <code>-- Build files have been written to: …/build</code>
</div>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends at <code>[100%]</code> with no <code>Error</code> lines.
</div>

=== Test (about 1.5 minutes) ===

Paste this block '''on its own''' and wait for it to finish. The MPI tests read from the terminal, so commands pasted ahead of time would be swallowed and never run.

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
ctest -j"$NUM_CORES" --output-on-failure 2>&1 | tee ctest.log | tail -5
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>100% tests passed out of 826</code>
</div>

=== Install ===

<pre>
cd "$TMP_BUILD_DIR/hdf5-${HDF5_VER}/build"
make install
otool -D "$PREFIX/lib/libhdf5_hl.dylib"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' the last line is <code>/opt/parallel-netcdf/lib/libhdf5_hl.320.dylib</code>. If it starts with <code>@rpath/</code> instead, the <code>CMAKE_INSTALL_NAME_DIR</code> setting was lost; redo this step.
</div>

== Step 3 – NetCDF-C (about 5–6 minutes) ==

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-c-${NETCDF_C_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
export LIBS="-lhdf5_hl -lhdf5 -lz"
./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --enable-netcdf-4 --enable-parallel4 --disable-dap --disable-parallel-tests
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' ends with a "NetCDF C Configuration Summary" that includes <code>HDF5 Support: yes</code> and <code>NC-4 Parallel Support: yes</code>.
</div>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make -j"$NUM_CORES"
</pre>

=== Test (about 5 minutes) ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log) FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 240 FAIL: 0</code>. A much smaller PASS count means testing stopped early at a failure; run <code>grep -E '^(FAIL|ERROR):' check.log</code> to see which one.
</div>

=== Install and verify ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-c-${NETCDF_C_VER}"
make install
unset LIBS
nc-config --has-parallel4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>yes</code>. If it prints <code>no</code>, NetCDF-C was built without parallel support: stop here.
</div>

== Step 4 – NetCDF-Fortran (about 2 minutes) ==

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-fortran-${NETCDF_F_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"
./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --disable-parallel-tests
</pre>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' many <code>Warning: Type mismatch …</code> / <code>Rank mismatch …</code> messages from gfortran. These are normal for NetCDF-Fortran; only <code>Error</code> lines matter.
</div>

=== Test ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log) FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 59 FAIL: 0</code>
</div>

=== Install and verify ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-fortran-${NETCDF_F_VER}"
make install
unset LIBS
nf-config --has-nc4
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>yes</code>
</div>

== Step 5 – NetCDF-C++4 (under 1 minute) ==

NetCDF-C++ needs two extra commands that remove files named <code>VERSION</code>; the reason is explained in [[#Troubleshooting|Troubleshooting]]. Don't skip them.

=== Extract and configure ===

<pre>
rm -rf "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
tar -xzf "$ARCHIVE_DIR/netcdf-cxx4-${NETCDF_CXX_VER}.tar.gz" -C "$TMP_BUILD_DIR"
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
find . -type f $ -name "version" -o -name "VERSION" $ -print -delete
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz"
CXXFLAGS="$CPPFLAGS" ./configure --prefix="$PREFIX" --host=aarch64-apple-darwin --disable-filter-testing
rm -f VERSION
</pre>

=== Build ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make -j"$NUM_CORES"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' no <code>error:</code> lines. If you see <code>./VERSION:1:1: error: expected unqualified-id</code>, the <code>rm -f VERSION</code> line was skipped: redo this step from "Extract and configure".
</div>

=== Test ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make check > check.log 2>&1
echo "PASS: $(grep -c '^PASS:' check.log) FAIL: $(grep -c -E '^(FAIL|ERROR):' check.log)"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>PASS: 16 FAIL: 0</code>
</div>

=== Install ===

<pre>
cd "$TMP_BUILD_DIR/netcdf-cxx4-${NETCDF_CXX_VER}"
make install
unset LIBS
ls "$PREFIX/lib/libnetcdf_c++4.dylib"
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:''' <code>/opt/parallel-netcdf/lib/libnetcdf_c++4.dylib</code>
</div>

== Step 6 – Final check ==

This confirms that the installed tools run without any special library path settings, and that no installed file depends on <code>@rpath</code>.

<pre>
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/h5dump" --version
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/nc-config" --version
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/nf-config" --has-nc4
env -u DYLD_LIBRARY_PATH "$PREFIX/bin/ncdump" 2>&1 | head -1
otool -L "$PREFIX"/lib/*.dylib "$PREFIX"/bin/* 2>/dev/null | grep -c '@rpath'
</pre>

<div style="border:1px solid #aaa; background:#f4f4f4; padding:0.5em;">
'''Expected:'''
<code>h5dump: Version 2.2.0</code>, <code>netCDF 4.10.1</code>, <code>yes</code>, a line starting with <code>/opt/parallel-netcdf/bin/ncdump [-c|-h]</code>, and finally <code>0</code>.
</div>

The scratch build directory is no longer needed. To free the disk space:

<pre>
rm -rf "$TMP_BUILD_DIR"
</pre>

== Shell profile ==

Add these lines to your shell profile (<code>~/.bash_profile</code> for bash, <code>~/.zshrc</code> for zsh) so programs can find the tools:

<pre>
export PATH="/opt/parallel-netcdf/bin:$PATH"
export OMPI_MCA_io=romio341
</pre>

* <code>PATH</code> is required.
* <code>OMPI_MCA_io</code> is '''possibly required, but untested with model runs''': it avoids the Open MPI hang described below. It was needed for the HDF5 parallel tests to pass.
* <code>DYLD_LIBRARY_PATH</code> is '''not''' needed: the libraries record their full install paths.

== Troubleshooting ==

; <code>curl: (22) The requested URL returned error: 404</code>
: That version isn't hosted at the URL. Unidata only keeps recent versions (for example, NetCDF-C++ 4.2.1 is gone). Check the latest versions with <code>check_versions.sh</code> in the <code>NetCDFlib</code> directory, or on the GitHub release pages.

; <code>configure: error: cannot run C compiled programs</code>
: The <code>--host=aarch64-apple-darwin</code> option is missing from the <code>./configure</code> line. It makes configure skip a test-run of a compiled program that fails on macOS with the MPI compiler wrappers; the libraries are still built natively.

; <code>dyld: Library not loaded: @rpath/libhdf5_hl.320.dylib</code>
: HDF5 was installed without <code>-DCMAKE_INSTALL_NAME_DIR="$PREFIX/lib"</code>. Redo Step 2, then Steps 3–5 so they link against the corrected libraries. Setting <code>DYLD_LIBRARY_PATH</code> is not a reliable fix: macOS removes it whenever a program is started via <code>/bin/sh</code> or <code>/bin/bash</code> (as <code>make</code>, job scripts and many <code>mpirun</code> setups do).

; A parallel test or model run hangs after <code>mca_fbtl_posix_ipwritev: error in aio_write(): Resource temporarily unavailable</code>
: <code>OMPI_MCA_io=romio341</code> is not set. Open MPI's default I/O layer (OMPIO) issues more asynchronous writes than macOS allows per process, then waits forever. ROMIO avoids this. Set the variable (Step 0 and your shell profile), or run <code>mpirun --mca io romio341 …</code>.

; NetCDF-C++: <code>./VERSION:1:1: error: expected unqualified-id</code> (or <code>expected external declaration</code>)
: The standard C++ headers include a header called <code><version></code>. On macOS the file system ignores case and the build searches the source directory for headers, so the compiler picks up the package's own <code>VERSION</code> file instead. NetCDF-C++'s <code>configure</code> creates that file, which is why Step 5 deletes it after configure. Never create <code>version</code>/<code>VERSION</code> files in that tree.

; NetCDF-C++: <code>Undefined symbols … _nc_set_log_level</code>
: <code>--disable-filter-testing</code> is missing from the configure line. Without it, an example program is built that needs a NetCDF-C function which only exists when NetCDF-C was built with logging enabled.

; <code>make check</code> shows far fewer passes than expected
: <code>make check</code> stops at the first directory with a failure, so the remaining tests never run. Look at the first <code>FAIL:</code> line in <code>check.log</code> and the matching <code>.log</code> file next to that test.

@@ Line 1: / Line 1: @@
-= Compiling XIOS on macOS (Apple Silicon) with Open MPI and Clang =
+= Compiling XIOS on macOS (Apple Silicon) with openmpi and clang =
-This page provides the three XIOS arch files needed to build XIOS on macOS with Homebrew gfortran and Homebrew Open MPI. In this setup the Open MPI C++ wrapper calls Apple '''clang''', not GCC. Because of that, two settings differ from the stock <code>arch-gfortran</code> files. They are explained in [[#Background: Open MPI uses Clang, not GCC|the background section]] below.
+This page makes the assumption that you have built and installed the parallel version of NetCDF similarly to that described on the page [[NetCDF-Parallel on an Apple Silicon Mac]].  Below, one finds the three XIOS arch files needed to build XIOS on macOS (using Homebrew gfortran and Homebrew Open MPI).  The customizations here with respect to the stock <code>arch-gfortran</code> files are driven by the Open MPI C++ wrapper calling Apple '''clang''', not GCC (e.g., [[#Background: Open MPI uses Clang, not GCC|the background section]]).
 == Tested environment ==
@@ Line 9: / Line 9: @@
 * Apple clang 21, which provides <code>/usr/bin/cpp</code> and is called by <code>mpiCC</code>
 * Open MPI 5.0.11 (Homebrew)
-* netCDF-C 4.10.1 and netCDF-Fortran 4.6.4, compiled from source against Open MPI and installed in <code>/opt/parallel-netcdf</code>
+* netCDF-C 4.10.1 and netCDF-Fortran 4.6.4, compiled from source against Open MPI(which is where the CLANG dependence originates) and installed in <code>/opt/parallel-netcdf</code>
 == Arch files ==
@@ Line 20: / Line 20: @@
 <pre>
-# Environment to compile XIOS on Mac OS X with NETCDF compiled with CLANG
+# Environment to compile XIOS on Mac OS X with NETCDF compiled with CLANG (through openmpi)
+#
 # Tested successfully with :
@@ Line 91: / Line 91: @@
 </pre>
-Use <code>--full</code> to rebuild from scratch. <code>make_xios</code> has no <code>--clean</code> option, and it ignores unknown options without any warning. So <code>./make_xios --clean</code> cleans nothing. FCM then keeps reusing old preprocessed files in <code>ppsrc/</code>, and a fix to the preprocessor settings does not take effect.
+Use <code>--full</code> to rebuild from scratch.
 A successful build produces:
 * <code>lib/libxios.a</code>
 == Background: Open MPI uses Clang, not GCC ==