Difference between revisions of "Some Ferret tips and pointers"

From Planets
Jump to: navigation, search
(→‎Ferret)
 
(19 intermediate revisions by 2 users not shown)
Line 4: Line 4:
 
This free software is from NOAA (National Oceanic and Atmospheric Administration) and the relevant webpage where you can learn how to install and use it is here:  https://ferret.pmel.noaa.gov/Ferret/
 
This free software is from NOAA (National Oceanic and Atmospheric Administration) and the relevant webpage where you can learn how to install and use it is here:  https://ferret.pmel.noaa.gov/Ferret/
  
−
[For Mac users] There is a bug in the Mac OSX version as of 2023-June with respect to plotting.  The bug is not subtle as the plotting does not work.  See section below [[#Installation_tips]]
+
[For Mac users] It is recommended to use/migrate-to PyFerret.  However, there is a bug in later versions that can render plotting problematic even in the Linux environment.  Fortunately, there is a work around, which we will address specifically for the Mac environment (a post to the Ferret listserv is referenced, and it deals specifically with Linux).  See [[Some_Ferret_tips_and_pointers#Installation_tips| section below]].
  
 
== For first timers ==
 
== For first timers ==
Line 11: Line 11:
 
ferret
 
ferret
 
</syntaxhighlight>
 
</syntaxhighlight>
−
(on some system, e.g. Ubuntu, the executable is called '''ferret_viz''' because another unrelated software also called '''ferret''' exists).
+
(on some system, e.g. Ubuntu, the executable is called '''ferret_c''' (and the related package '''ferret_viz''') because another unrelated software also called '''ferret''' exists).
  
 
Once under Ferret you should see the Ferret prompt:
 
Once under Ferret you should see the Ferret prompt:
Line 22: Line 22:
 
yes? use diagfi.nc
 
yes? use diagfi.nc
 
</pre>
 
</pre>
−
Once the file has been loaded, you can query for its contents with the command '''show data'''
+
Once the file has been loaded, you can query for its contents with the command '''show data'''. This will let you know about available fields (and their shapes: usually i along longitudes, j along latitudes, k along pseudo-altitudes and l along time). e.g
−
...
+
<pre>
 +
yes? show data
 +
    currently SET data sets:
 +
    1> ./diagfi.nc  (default)
 +
name    title                            I        J        K        L
 +
AIRE                                      1:65      1:49      ...      ...
 +
PHISINIT                                  1:65      1:49      ...      ...
 +
TSURF                                    1:65      1:49      ...      1:1464
 +
PS                                        1:65      1:49      ...      1:1464
 +
TEMP                                      1:65      1:49      1:73      1:1464
 +
U                                        1:65      1:49      1:73      1:1464
 +
V                                        1:65      1:49      1:73      1:1464
 +
</pre>
 +
In the example above the file contains '''AIRE''' and '''PHISINIT''', which are 2D (indexed along i and j indexes, i.e. longitude and latitude == surface fields), '''TSURF''', '''PS''', which are 3D (indexed along i, j and l indexes, i.e. longitude, latitude and time == surface fields evolving in time) and '''TEMP''', '''U''' and '''V''', which are 4D).
  
 
== Various tips and pointers ==
 
== Various tips and pointers ==
−
...
+
* Don't kill the Ferret graphic plot window (by clicking on the cross at the top right corner); Ferret won't recover, you'll have to exit and start again...
 +
* One can refer to a field coordinates either via indexes '''i''', '''j''', '''k''', '''l''' or via associated real world coordinates '''x''', '''y''', '''z''', '''t'''.
 +
 
 
==== How to make a 1D plot ====
 
==== How to make a 1D plot ====
−
...
+
You simply need to specify along which coordinate the variable is fixed and not mention the one along which it should be plotted, e.g.
 +
<pre>
 +
yes? plot/i=13/j=15 PS
 +
</pre>
 +
Or equivalently
 +
<pre>
 +
yes? plot PS[i=13,j=15]
 +
</pre>
 +
Or similarly using real world coordinates, e.g.
 +
<pre>
 +
yes? plot PS[x=12,y=35]
 +
</pre>
 +
Note that when using real world coordinate, if the sought coordinates are not exactly those of the underlying grid then Ferret will use the nearest neighbor (and not interpolate to the desired coordinates).
 +
 
 +
 
 +
One may limit the range of the coordinate along which the plot is made by explicitly providing the interval boundaries, e.g. to plot PS value at grid indexes i=13 and j=15 but for l ranging from 240 to 480:
 +
<pre>
 +
yes? plot PS[i=13,j=15,l=240:480]
 +
</pre>
 +
 
 
==== How to make a 2D map ====
 
==== How to make a 2D map ====
−
...
+
Same story as for a 1D plot, one needs to specify along which the coordinates the variable is fixed and not mention the remaining two along which the map should be made. e.g.
−
==== How to compute a zonal average ====
+
<pre>
−
...
+
yes? shade TSURF[l=120]
 +
</pre>
 +
Note that with "shade" the output maps is raw (cell values are displayed as homogeneous tiles); if you want Ferret to smooth between grid points then use "fill":
 +
<pre>
 +
yes? fill TSURF[l=120]
 +
</pre>
 +
 
 +
==== How to list data values ====
 +
Just use "list" instead of "plot" or "shade" etc. to get the values of the field
 +
 
 +
==== How to compute an average ====
 +
There are some operators which can be applied along axes, namely "@ave" for averaging (there are also "@min" for extracting minimum value, "@max" for extracting the maximum value and ""@sum" for computing the sum) which can be used on variables.
 +
For example to display the map of the time average over the whole time series of TSURF:
 +
<pre>
 +
yes? shade TSURF[l=@ave]
 +
</pre>
 +
Or to display the latitude-altitude map of the zonal value of TEMP at time index 12:
 +
<pre>
 +
yes? shade TEMP[x=@ave,l=12]
 +
</pre>
 +
Operators can be combined and applied along different direction; for instance to display the time averaged zonal value of TEMP:
 +
<pre>
 +
yes? shade TEMP[x=@ave,l=@ave]
 +
</pre>
 +
The operators can also be applied to a sub-set of the axis; for instance to display the temporal average evaluated between time steps 48 and 96 of the zonal value of TEMP:
 +
<pre>
 +
yes? shade TEMP[x=@ave,l=48:96@ave]
 +
</pre>
 +
 
 +
==== Saving the figure ====
 +
After the plot command use the "FRAME" command (and specify output file name, which has to be .gif), e.g.:
 +
<pre>
 +
yes? shade TEMP[x=@ave,l=@ave]
 +
yes? FRAME/FILE="mygreatplot.gif"
 +
</pre>
 +
 
 +
==== Creating intermediate variables ====
 +
You can create your own intermediate variable combining existing ones using the command "let", e.g. with
 +
<pre>
 +
yes? let MASS_MTOT=MTOT*AIRE
 +
</pre>
 +
where "MTOT" is the column amount of water vapor (in kg/m2) and "AIRE" is the mesh area (in m2) we create a new field "MASS_MTOT" which is the mass of water vapor (in kg) in each column.
 +
 
 +
You can then use your variable just like any other, including applying operations on it. For instance you can sum over longitudes and latitudes "MASS_MTOT" to plot the time evolution of the total planetary amount of water vapor (in kg) with:
 +
<pre>
 +
yes? plot MASS_MTOT[i=1:64@sum,j=@sum]
 +
</pre>
 +
where by the way we sum only over the first 64 longitude points as the 65th (longitude +180°) is redundant with the 1st (longitude -180°).
 +
 
 +
==== Writing instructions as jnl scripts ====
 +
You can write up some (ASCII) text file containing a series of Ferret commands, and save it as a "journal" script with the extension ".jnl", e.g. "myscript.jnl". To load it in your Ferret session simply use the command "go":
 +
<pre>
 +
yes? go myscript
 +
</pre>
  
 
== Installation tips ==
 
== Installation tips ==
 +
 +
=== Ubuntu ferret_c environment ===
 +
On the Ubuntu Linux where Ferret comes as "ferret_c", if the executable crashes when attempting to pot with a message of the likes of
 +
<pre>
 +
PPL+ error: Unable to read "default.spk" to set shade and fill colors.
 +
 +
          *** NOTE: Fonts directory is undefined. Check the Ferret environment.
 +
**ERROR: file undefined or not found: fondat.dat
 +
</pre>
 +
This is most likely because it is missing some necessary environment variable. The cure is easy: you should
 +
<syntaxhighlight lang="bash">
 +
source /usr/share/ferret-vis/bin/ferret_paths
 +
</syntaxhighlight>
 +
before running ferret_c (and possibly add that command in your .bashrc to not have to issue it every time you want to run Ferret).
 +
 +
===Mac OSX (PyFerret) and Graphics bug===
 +
 +
====Miniconda====
 +
conda is the preferred environment for the install.  miniconda (a subset of conda) is a lighter-weight option which is a good choice.  Installation instructions for both may be found [https://docs.conda.io/projects/conda/en/latest/user-guide/install/macos.html here].
 +
 +
====PyFerret====
 +
Once one has a conda installation of some type, the instructions are straight-forward.  The official installation documentation is [https://github.com/NOAA-PMEL/PyFerret/blob/master/README.md here].
 +
 +
====Commands used on a Macbook Pro with the M2 ARM chipset (summer 2023)====
 +
 +
* Install miniconda using script
 +
 +
<pre>
 +
bash Miniconda3-latest-MacOSX-arm64.sh      #can "yes" to activate, but we will deactivate
 +
 +
# do not automatically activate on startup
 +
conda config --set auto_activate_base false   
 +
 +
# initialize for tcsh (it will write code into one's .tcshrc file)
 +
/Users/mwolff/miniconda3/bin/conda init tcsh   
 +
</pre>
 +
 +
* Install pyferret (latest version)
 +
It is recommended that one use virtual environments, and alias the environment activation for simplicity.  The following section is to install pyferret in virtual environment called FERRET.
 +
 +
Note: the pyferret command from the official site did NOT work for me.  It complained about not finding the needed packages.  For reference, this command was
 +
<pre>
 +
# line below did not work.  it is here for reference
 +
conda create -n FERRET -c conda-forge pyferret ferret_datasets --yes
 +
</pre>
 +
 +
Instead, use the following instead:
 +
<pre>
 +
conda create -n FERRET -c conda-forge/osx-64 pyferret ferret_datasets --yes
 +
</pre>
 +
 +
This can be actived using an alias for brevity.  In (t)csh world, this would be
 +
<pre>
 +
alias FERRET "conda activate FERRET"
 +
</pre>
 +
 +
* Dealing with plotting bug.  As can be seen in the post on the Ferret group [https://www.pmel.noaa.gov/maillists/tmap/ferret_users/fu_2019/msg01059.html here], there can be problems for plotting with the latest version(s), even on Linux.  Below, we use a set of commands that works for Mac OSX.
 +
 +
Install a slightly older version of pyferret (i.e., one where the plotting works) using the command in the referenced post in the last paragrph):
 +
<pre>
 +
# we have had to change the channel from the example in the referenced ferret group post (to install for Mac OS
 +
conda create -n oldFERRET -c conda-forge/osx-64/label/cf202003 pyferret ferret_datasets --yes
 +
</pre>
 +
 +
Assuming an alias oldFERRET that activates this older version, a plotting test can be runs as
 +
<pre>
 +
[avory:~] mwolff  oldFERRET
 +
 +
(oldFERRET) [avory:~] mwolff pyferret
 +
        NOAA/PMEL TMAP
 +
        PyFerret v7.5 (optimized)
 +
        Darwin 17.7.0 - 12/16/19
 +
        6-Apr-23 10:50
 +
yes? use coads_climatology
 +
yes? fill sst[l=1]
 +
 +
</pre>
 +
 +
[[File:Ferret_testplot_coads.png]]
 +
 +
 +
[[Category:FAQ]]

Latest revision as of 09:16, 3 January 2024

Ferret

Ferret is a NetCDF vizualisation tool which can be used to postprocess and visualize PCM outputs.

This free software is from NOAA (National Oceanic and Atmospheric Administration) and the relevant webpage where you can learn how to install and use it is here: https://ferret.pmel.noaa.gov/Ferret/

[For Mac users] It is recommended to use/migrate-to PyFerret. However, there is a bug in later versions that can render plotting problematic even in the Linux environment. Fortunately, there is a work around, which we will address specifically for the Mac environment (a post to the Ferret listserv is referenced, and it deals specifically with Linux). See section below.

For first timers

Ferret is essentially a command line tool so the first thing you'll need to do is launch it from your terminal 

ferret

(on some system, e.g. Ubuntu, the executable is called ferret_c (and the related package ferret_viz) because another unrelated software also called ferret exists).

Once under Ferret you should see the Ferret prompt: 

yes?

To load a datafile, for example diagfi.nc issue command: 

yes? use diagfi.nc

Once the file has been loaded, you can query for its contents with the command show data. This will let you know about available fields (and their shapes: usually i along longitudes, j along latitudes, k along pseudo-altitudes and l along time). e.g 

yes? show data
     currently SET data sets:
    1> ./diagfi.nc  (default)
 name     title                             I         J         K         L
 AIRE                                      1:65      1:49      ...       ...
 PHISINIT                                  1:65      1:49      ...       ...
 TSURF                                     1:65      1:49      ...       1:1464
 PS                                        1:65      1:49      ...       1:1464
 TEMP                                      1:65      1:49      1:73      1:1464
 U                                         1:65      1:49      1:73      1:1464
 V                                         1:65      1:49      1:73      1:1464

In the example above the file contains AIRE and PHISINIT, which are 2D (indexed along i and j indexes, i.e. longitude and latitude == surface fields), TSURF, PS, which are 3D (indexed along i, j and l indexes, i.e. longitude, latitude and time == surface fields evolving in time) and TEMP, U and V, which are 4D).

Various tips and pointers

  • Don't kill the Ferret graphic plot window (by clicking on the cross at the top right corner); Ferret won't recover, you'll have to exit and start again...
  • One can refer to a field coordinates either via indexes i, j, k, l or via associated real world coordinates x, y, z, t.

How to make a 1D plot

You simply need to specify along which coordinate the variable is fixed and not mention the one along which it should be plotted, e.g. 

yes? plot/i=13/j=15 PS

Or equivalently 

yes? plot PS[i=13,j=15]

Or similarly using real world coordinates, e.g. 

yes? plot PS[x=12,y=35]

Note that when using real world coordinate, if the sought coordinates are not exactly those of the underlying grid then Ferret will use the nearest neighbor (and not interpolate to the desired coordinates).


One may limit the range of the coordinate along which the plot is made by explicitly providing the interval boundaries, e.g. to plot PS value at grid indexes i=13 and j=15 but for l ranging from 240 to 480: 

yes? plot PS[i=13,j=15,l=240:480]

How to make a 2D map

Same story as for a 1D plot, one needs to specify along which the coordinates the variable is fixed and not mention the remaining two along which the map should be made. e.g. 

yes? shade TSURF[l=120]

Note that with "shade" the output maps is raw (cell values are displayed as homogeneous tiles); if you want Ferret to smooth between grid points then use "fill": 

yes? fill TSURF[l=120]

How to list data values

Just use "list" instead of "plot" or "shade" etc. to get the values of the field

How to compute an average

There are some operators which can be applied along axes, namely "@ave" for averaging (there are also "@min" for extracting minimum value, "@max" for extracting the maximum value and ""@sum" for computing the sum) which can be used on variables. For example to display the map of the time average over the whole time series of TSURF: 

yes? shade TSURF[l=@ave]

Or to display the latitude-altitude map of the zonal value of TEMP at time index 12: 

yes? shade TEMP[x=@ave,l=12]

Operators can be combined and applied along different direction; for instance to display the time averaged zonal value of TEMP: 

yes? shade TEMP[x=@ave,l=@ave]

The operators can also be applied to a sub-set of the axis; for instance to display the temporal average evaluated between time steps 48 and 96 of the zonal value of TEMP: 

yes? shade TEMP[x=@ave,l=48:96@ave]

Saving the figure

After the plot command use the "FRAME" command (and specify output file name, which has to be .gif), e.g.: 

yes? shade TEMP[x=@ave,l=@ave]
yes? FRAME/FILE="mygreatplot.gif"

Creating intermediate variables

You can create your own intermediate variable combining existing ones using the command "let", e.g. with 

yes? let MASS_MTOT=MTOT*AIRE

where "MTOT" is the column amount of water vapor (in kg/m2) and "AIRE" is the mesh area (in m2) we create a new field "MASS_MTOT" which is the mass of water vapor (in kg) in each column.

You can then use your variable just like any other, including applying operations on it. For instance you can sum over longitudes and latitudes "MASS_MTOT" to plot the time evolution of the total planetary amount of water vapor (in kg) with: 

yes? plot MASS_MTOT[i=1:64@sum,j=@sum]

where by the way we sum only over the first 64 longitude points as the 65th (longitude +180°) is redundant with the 1st (longitude -180°).

Writing instructions as jnl scripts

You can write up some (ASCII) text file containing a series of Ferret commands, and save it as a "journal" script with the extension ".jnl", e.g. "myscript.jnl". To load it in your Ferret session simply use the command "go": 

yes? go myscript

Installation tips

Ubuntu ferret_c environment

On the Ubuntu Linux where Ferret comes as "ferret_c", if the executable crashes when attempting to pot with a message of the likes of 

 PPL+ error: Unable to read "default.spk" to set shade and fill colors.

           *** NOTE: Fonts directory is undefined. Check the Ferret environment.
 **ERROR: file undefined or not found: fondat.dat

This is most likely because it is missing some necessary environment variable. The cure is easy: you should 

source /usr/share/ferret-vis/bin/ferret_paths

before running ferret_c (and possibly add that command in your .bashrc to not have to issue it every time you want to run Ferret).

Mac OSX (PyFerret) and Graphics bug

Miniconda

conda is the preferred environment for the install. miniconda (a subset of conda) is a lighter-weight option which is a good choice. Installation instructions for both may be found here.

PyFerret

Once one has a conda installation of some type, the instructions are straight-forward. The official installation documentation is here.

Commands used on a Macbook Pro with the M2 ARM chipset (summer 2023)

  • Install miniconda using script
bash Miniconda3-latest-MacOSX-arm64.sh      #can "yes" to activate, but we will deactivate

 # do not automatically activate on startup
conda config --set auto_activate_base false    

# initialize for tcsh (it will write code into one's .tcshrc file)
/Users/mwolff/miniconda3/bin/conda init tcsh
  • Install pyferret (latest version)

It is recommended that one use virtual environments, and alias the environment activation for simplicity. The following section is to install pyferret in virtual environment called FERRET.

Note: the pyferret command from the official site did NOT work for me. It complained about not finding the needed packages. For reference, this command was 

# line below did not work.  it is here for reference
conda create -n FERRET -c conda-forge pyferret ferret_datasets --yes

Instead, use the following instead: 

conda create -n FERRET -c conda-forge/osx-64 pyferret ferret_datasets --yes

This can be actived using an alias for brevity. In (t)csh world, this would be 

alias FERRET "conda activate FERRET"
  • Dealing with plotting bug. As can be seen in the post on the Ferret group here, there can be problems for plotting with the latest version(s), even on Linux. Below, we use a set of commands that works for Mac OSX.

Install a slightly older version of pyferret (i.e., one where the plotting works) using the command in the referenced post in the last paragrph): 

# we have had to change the channel from the example in the referenced ferret group post (to install for Mac OS
conda create -n oldFERRET -c conda-forge/osx-64/label/cf202003 pyferret ferret_datasets --yes

Assuming an alias oldFERRET that activates this older version, a plotting test can be runs as 

[avory:~] mwolff  oldFERRET

(oldFERRET) [avory:~] mwolff pyferret
        NOAA/PMEL TMAP
        PyFerret v7.5 (optimized)
        Darwin 17.7.0 - 12/16/19 
        6-Apr-23 10:50
yes? use coads_climatology
yes? fill sst[l=1]

Ferret testplot coads.png

Retrieved from "http://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php?title=Some_Ferret_tips_and_pointers&oldid=1791"