See the cf-python home page for more documention on the CF data model, downloads and source code.
A field is composed as follows (refer to the data model description and accompanying UML diagram for more details):
All components of a field are optional.
All objects which inherit from the Variable class (namely Variable, Field, Coordinate, CoordinateBounds and CellMeasures) contain data arrays. The data storage, access and manipulation model is very similar for all of them and it will be noted when certain objects exhibit different behaviour. In this section, the word ‘variable’ refers to any object which inherits from the Variable class .
A variable’s data may be sliced by indexing its slice attribute. Indexing is similar to that of a numpy array. The differences to numpy array indexing are:
- An integer index i takes the i-th element but does not reduce the rank of the output array by one.
- More than one dimension’s slice may be a 1-d boolean array or 1-d sequence of integers, and these indices work independently along each dimension (similar to the way vector subscripts work in Fortran).
The result of a slice is a new variable instance and if the variable’s the new variable’s data will be a deep copy of the slice of the orignal array.
>>> isinstance(v, Variable)
True
>>> v.shape
(10, 20, 30)
>>> v.slice[0].shape
(1, 20, 30)
>>> v.slice[0, -1, 3].shape
(1, 1, 1)
>>> v.slice[0:5,...,slice(11,0,-2)].shape
(5, 20, 6)
>>> v.shape
(10, 20, 30)
>>> v.slice[:,:,numpy.arange(30) < 4].shape
(10, 20, 4)
>>> v.slice([1,2], [3,4], [5,6]).shape
(2, 2, 2)
For variables which contain other variable types, the contained variables are also sliced. So slicing a coordinate also slices its bounds, if there are any; and slicing a field also slices its space by applying each dimension’s slice to the appropriate space elements.
Size 1 dimensions resulting from a slice are always retained. However, a size 1 dimension may optionally be dropped from the field’s data and its dimension coordinate reduced to a scalar if no auxiliary coordinates or cell measures span that dimension. Therefore, fields and dimension coordinates may contain scalar data, but auxiliary coordinates and cell measures may not. A field’s squash method will carry out this reduction.
A field may be resized by specifying conditions on its coordinate’s values. These conditions are specified as arguments to a call to the field’s slice attribute (as opposed to indexing it). Note that only a field’s slice attribute may be called in this manner.
>>> s
<CF Field: air_temperature(7070, 64, 128)>
>>> print s
Data : air_temperature(time, latitude, longitude)
Cell methods : time: mean (interval: 1.0 month)
Dimensions : time(7070) -> 450-11-16 00:00:00 to 1049-12-16 12:00:00
: latitude(64) -> -87.8638 to 87.8638 degrees_north
: longitude(128) -> 0 to 357.1875 degrees_east
: height(1) -> 2 m
Auxiliary coords:
>>> s.slice[:,32, 64]
<CF Field: air_temperature(7070, 1, 1)
>>> s.slice(longitude=0.0, latitude=0.0)
<CF Field: air_temperature(7070, 1, 1)
Assignment to the data array is done with the same indexing of the slice attribute as for data access. Assignment can not change the shape of the data:
>>> v.shape
(10, 20, 30)
>>> v.slice[0, 0, 0] = 273.15
>>> v.varray[0,0,0]
273.15
>>> v.slice[0] = 273.15
>>> v.shape
(10, 20, 30)
>>> v.slice[1,2,:] = range(30)
>>> v.slice[...] = numpy.arange(6000).reshape(10, 20, 30)
The data may be stored as a numpy array or as a file pointer. A file pointer may be any kind of object which has attributes shape, size, ndim and dtype; and is indexable all in the same manner as a numpy array. A netCDF4.Variable instance is an example of a vaild file pointer object. Accessing the data makes no distinction between the two storage methods, but there are I/O, memory and speed performance issues to consider. If the data are a file pointer then accessing any part of the array will (at present) cause the entire array to be read from disk into memory and stored as a numpy array, replacing the original file pointer. Refer to read and write.
The numpy array or file pointer is held inside a Data instance and a variable’s data may be completely replaced by assigning a Data object to its _data attribute:
>>> type(ncvariable)
<type 'netCDF4.Variable'>
>>> v._data = cf.Data(ncvariable)
>>> v._data = cf.Data(numpy.arange(10))
Note that this assignment makes does not check on consistency with the variable’s metadata.
A deep copy of a variable may be created using its copy method or, equivalently, using the copy module.
>>> w = v.copy()
>>> w = copy.deepcopy(v)
This type of copy does not read any data from disk, i.e. it will not convert a file pointer to a numpy array. A third type of copy, however, will do such a conversion, namely slicing the variable with an ellipsis (or equivalent):
>>> v.type
<type 'netCDF4.Variable'>
>>> w = v.copy()
>>> w.type, v.type
(<type 'netCDF4.Variable'>, <type 'netCDF4.Variable'>)
>>> w = v.slice[...]
>>> w.type, v.type
(<type 'numpy.ndarray'>, <type 'numpy.ndarray'>)
The following operators, operations and assignments are overloaded in a variable to apply element-wise to the variable’s data numpy array.
Comparison operators:
==, !=, >, <, >=, <=
Binary arithmetic operations:
+, -, *, /, //, %, pow(), **, &, ^, |
Unary arithmetic operations:
-, +, abs(), ~
Augmented arithmetic assignments:
+=, -=, *=, /=, //=, %=, **=, &=, ^=, |=
Either side of an arithmetic operation may be a variable (in which case its data as a numpy array are used) or any object allowed by the equivalent numpy operation. Note that, as usual, if the left hand side object supports the operation with the right hand side then the returned object will be of former’s type:
>>> type(v)
<class 'cf.field.Field'>
>>> type(2.5 + v)
<class 'cf.field.Field'>
>>> type(v.varray + v)
<type 'numpy.ndarray'>
Apart from this exception, the unary and binary arithmetic operations return a new variable with modified data. The augmented arithmetic assignments change a variable’s data an in-place.
Numeric equalities for the == and != comparisons are determined to within a tolerance which may be adjusted.
Most objects defined in the cf package have an equals method which determines the congruence of two instances. The aspects of this equality vary between objects, but for all objects numeric equalities are tested to within a tolerance defined by parameters ‘rtol’ (relative tolerance) and ‘atol’ (absolute tolerance), where two numbers a and b (from the left and right sides respectively of the comparison) are considered equal if
Default tolerances may be found and set with the functions RTOL and ATOL. Numerically tolerant equality of two objects is also tested by the equals function.
A space contains any number of dimensions and space components, the latter comprising dimension coordinates, auxiliary coordinates, cell measures and transforms.
A field’s space is stored in its space attribute, the value of which is a Space object.
The dimensions of the space, and of its associated field, are given by the space’s dimension_sizes and dimensions attributes.
The dimension_sizes attribute is a dictionary whose keys are dimension identifiers and values are positive integer dimension sizes. A dimension identifier is the string ‘dim’ suffixed by an arbitrary, but unique, integer. For example:
>>> g
<CF Space: (30, 24, 1, 17)>
>>> g.dimension_sizes
{'dim0': 1, 'dim1': 17, 'dim2': 30, 'dim3': 24}
The dimensions attribute specifies which dimensions relate to each space component (coordinates and cell measures) and to the field which holds the space. For example:
>>> g.dimensions
{'data': ['dim1', 'dim2', 'dim3'],
'aux0': ['dim2', 'dim3'],
'aux1': ['dim2', 'dim3'],
'cm0' : ['dim2', 'dim3'],
'dim0': ['dim0'],
'dim1': ['dim1'],
'dim2': ['dim2'],
'dim3': ['dim3']}
A key of this dictionary identifies a space component or the field’s data as follows:
| Key prefix | Description |
|---|---|
| aux | Auxiliary coordinate |
| cm | Cell measures |
| data | The field containing the space |
| dim | Dimension coordinate |
Each key’s value is an ordered list which corresponds to the shape of that component’s data, with two exceptions:
- If the field’s data is a scalar then its value is an empty list
- If a dimension coordinate is a scalar it retains its dimension as a single element list.
An arbitrary non-negative integer after the prefix discerns between space components of the same type.
It is possible for a space dimension to have no space components.
The space object is a kind of dictionary which support the built-in dictionary’s methods. Its keys are the identifiers of the space components dimension coordinates, auxiliary coordinates and cell measures. Each key’s value stores the variable it identifies. For example, a space’s key and value pairs may be:
>>> g.keys()
['dim0', 'dim1', 'dim2', 'dim3', 'aux0', 'aux1']
>>> for key, value in g.iteritems():
... print key, ':', repr(value)
...
dim0: <CF Coordinate: sigma(20)>
dim1: <CF Coordinate: time(12)>
dim2: <CF Coordinate: latitude(111)>
dim3: <CF Coordinate: longitude(106)>
aux0: <CF Coordinate: grid_latitude(111, 106)>
aux1: <CF Coordinate: grid_longitude(111, 106)>
The space may have any number of transforms describing projection parameters (CF grid mappings) or potiential auxiliary coordinates (coordinates defined by CF formula_terms equations). These are stored in the space’s transform attribute, which is a dictionary of Transform objects. For example:
>>> g.transform
{'trans0': <CF Transform: atmosphere_sigma_coordinate>,
'trans1': <CF Transform: rotated_latitude_longitude>}
>>> cf.dump(g.transform['trans0'])
atmosphere_sigma_coordinate transform
-------------------------------------
Transform['ps'] = <CF Field: surface_air_pressure(106, 111)>
Transform['ptop'] = <CF Field: ptop>
Transform['sigma'] = 'dim0'
>>> cf.dump(g.transform['trans1'])
rotated_latitude_longitude transform
------------------------------------
Transform['trans0']['grid_mapping_name'] = 'rotated_latitude_longitude'
Transform['trans0']['grid_north_pole_latitude'] = 70.0
Transform['trans0']['grid_north_pole_longitude'] = 100.0
Transform['trans0']['north_pole_grid_longitude'] = -80.0
A transform may also be linked with any number of the space’s coordinates via their transform attributes:
>>> g['dim0']
<CF Coordinate: sigma(20)>
>>> g['dim0'].transform
'trans0'
>>> g['dim2'].transform, g['dim3'].transform
('trans1', trans1')
>>> g['aux0'].transform, g['aux1'].transform
('trans1', trans1')
Attributes may be set on a variable in the usual way, but these will not be recognized as CF attributes. Some attributes, such as existing methods, are reserved and may not be set:
>>> v.property = 999
>>> v.property
999
>>> v.match = 'test'
AttributeError
Refer to Variable for details on reserved attributes and methods.
Public variable attributes are those which you expect to read from or written to a CF variable on disk. They are set, retrieved and deleted with the methods setpub, getpub and delpub respectively, which allow public attributes to share names with reserved attribute names, instance attributes and private attributes:
>>> v.getpub('standard_name')
AttributeError: 'Variable' object doesn't have public attribute 'standard_name'
>>> v.setpub('standard_name', 'air_temperature')
>>> v.getpub('standard_name')
'air_temperature'
>>> v.delpub('standard_name')
>>> v.getpub('standard_name')
AttributeError: 'Variable' object doesn't have public attribute 'standard_name'
>>> v.getpub('standard_name', 'default_value')
'default_value'
For conviencience, the variable’s pub method provides a shorter way of setting and retrieving public attribues which is exactly equivalent to either the setpub or getpub methods:
>>> v.pub(standard_name='air_temperature')
>>> v.pub('standard_name')
'air_temperature'
>>> v.delpub('standard_name')
>>> v.pub('standard_name', 'default_value')
'default_value'
Note that for a coordinate variable, b and c aare not public attr but reserved ones, reflecting there different usegae comared with cf
Private attributes comprise attributes may set directly on the instance, as above, but may also be set, retrieved and deleted with the methods setpriv, getpriv and delpriv respectively, which allow private attributes to share names with reserved attribute, instance attributes public attributes:
>>> v.setpriv('file', '../file.nc')
>>> v.getpriv('file')
'../file.nc'
>>> v.delpriv('file')
>>> v.getpriv('file', 'default_value')
'default_value'
The Variable, Coordinate and Field objects have associated list objects which allow an ordered sequence of instances to be stored as a single, list-like object:
| Class | Associated list class |
|---|---|
| Variable | VariableList |
| Coordinate | CoordinateList |
| Field | FieldList |
These list-like objects behave in much the same way as built-in lists:
>>> type(sl)
<class 'cf.field.FieldList'>
>>> sl
[<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>]
>>> len(sl)
3
They are indexable in same way as built-in lists returning either list or non-list types depending on the nature of the index:
>>> type(sl[0])
<class 'cf.field.Field'>
>>> sl[0]
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>
>>> type(sl[2::-2])
<class 'cf.field.FieldList'>
>>> sl[2::-2]
[<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> type(sl[slice(1,2)])
<class 'cf.field.FieldList'>
>>> sl[slice(1,2)]
[<CF Field: precipitation_flux(2400, 96, 192)>]
The usual built-in list methods work as expected. For example:
>>> print repr(sl.reverse()
[<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> sl.append(sl[0])
>>> sl
[<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>]
>>> len(sl)
4
>>> sl.insert(1, sl[2])
>>> sl
[<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>]
>>> len(sl)
5
>>> sl.index(sl[2])
2
>>> sl[3] in sl
True
>>> sl.extend(sl[0:2])
>>> sl
[<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> len(sl)
7
>>> sl.pop()
<CF Field: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>
>>> len(sl)
6
>>> sl.remove(sl[0])
>>> len(sl)
5
Non-list instances are also indexable as if they were single element lists. Indexing them is essentially a null operation, but allows them to be used interchangeably with their list counterparts. This is particularly useful in iterative situations when it is not known if an instance is a variable or its list counterpart:
>>> type(x)
<class 'cf.field.Field'>
>>> type(x[0])
<class 'cf.field.Field'>
>>> type(y)
<class 'cf.field.FieldList'>
>>> len(x), len(y)
(1, 3)
>>> for i in x:
... print type(i), repr(i)
...
<class 'cf.field.Field'> <CF Field: air_density(2400, 96, 192)>
>>> for i in y:
... print type(i), repr(i)
...
<class 'cf.field.Field'> <CF Field: air_pressure(2400, 96, 192)>
<class 'cf.field.Field'> <CF Field: air_temperature(2400, 96, 192)>
<class 'cf.field.Field'> <CF Field: precipitation_flux(2400, 96, 192)>
Attributes and methods of a variable are all available on its list counterpart by broadcasting to each element of the list:
>>> sl.units
['Pa', 'K', 'kg m-2 s-1']
>>> sl[1].units
'K'
>>> sl.units[1]
'K'
>>> sl[1:2].units
['K']
>>> sl.new_attribute = 'test'
>>> sl.new_attribute
['test', 'test', 'test']
>>> s = sl[2]
>>> s.new_attribute
'test'
>>> sl.name()
['air_pressure',
'air_temperature',
'precipitation_flux']
>>> sl.__repr__()
[<CF Field: air_pressure(2400, 96, 192)>,
<CF Field: air_temperature(2400, 96, 192)>,
<CF Field: precipitation_flux(2400, 96, 192)>]
>>> sl.match(units='K')
[False, True, False]
Use the pub and priv (or getpub and getpriv) methods to return defualts values for missing attributes:
>>> sl._FillValue
AttributeError: Can't get 'Field' attribute '_FillValue'
>>> sl.pub('_FillValue')
AttributeError: Can't get 'Field' attribute '_FillValue'
>>> sl.pub('_FillValue', None)
[None, None, None, -1.0e+30]
Fields read from disk with the read function are, by default, aggregated in to as few fields as possible according to the CF aggregation rules. Fields in memory may also be aggregated with the aggregate function.
| Class | Description | Parent class |
|---|---|---|
| Variable | Abstract base class | object |
| CfList | Abstract base class | MutableSequence |
| CfDict | Abstract base class | MutableMapping |
| Field | Field | Variable |
| Coordinate | Dimension or auxiliary coordinate | Variable |
| CoordinateBounds | Coordinate bounds | Variable |
| CellMeasures | Cell measures | Variable |
| Space | Space | CfDict |
| Transform | Coordinate transforms | CfDict |
| CellMethods | Cell methods | CfList |
| Units | Comparison expression | object |
| Data | Comparison expression | object |
| VariableList | List of variables | CfList |
| CoordinateList | List of coordinates | VariableList |
| FieldList | List of fields | VariableList |
| Comparison | Comparison expression | object |
| Function | Description |
|---|---|
| aggregate | Aggregate fields into as few fields as possible. |
| dump | Print the string returned from an object’s dump method. |
| eq | Create a Comparison object. |
| equals | Determine whether two objects are congruent. |
| ATOL | Absolute tolerance for numerical equality. |
| RTOL | Relative tolerance for numerical equality. |
| ge | Create a Comparison object. |
| gt | Create a Comparison object. |
| inside | Create a Comparison object. |
| le | Create a Comparison object. |
| lt | Create a Comparison object. |
| ne | Create a Comparison object. |
| outside | Create a Comparison object. |
| read | Read fields from netCDF files. |
| write | Write fields to a netCDF file. |