See the cf-python home page for more documention on the CF data model, downloads and source code.
A space is composed as follows (refer to the data model description and accompanying UML diagram for more details):
All components of a space are optional.
All objects which inherit from the Variable class (namely Variable, Space, Coordinate, CoordinateBounds and CellMeasures) contain data arrays. The data storage, access and manipulation model is very similar for all of them and it will be noted when certain objects exhibit different behaviour. In this section, the word ‘variable’ refers to any object which inherits from the Variable class .
A variable’s data may be sliced by indexing its slice attribute. Indexing is similar to that of a numpy array. The differences to numpy array indexing are:
- An integer index i takes the i-th element but does not reduce the rank of the output array by one.
- More than one dimension’s slice may be a 1-d boolean array or 1-d sequence of integers, and these indices work independently along each dimension (similar to the way vector subscripts work in Fortran).
The result of a slice is a new variable instance and if the variable’s the new variable’s data will be a deep copy of the slice of the orignal array.
>>> isinstance(v, Variable)
True
>>> v.shape
(10, 20, 30)
>>> v.slice[0].shape
(1, 20, 30)
>>> v.slice[0, -1, 3].shape
(1, 1, 1)
>>> v.slice[0:5,...,slice(11,0,-2)].shape
(5, 20, 6)
>>> v.shape
(10, 20, 30)
>>> v.slice[:,:,numpy.arange(30) < 4].shape
(10, 20, 4)
>>> v.slice([1,2], [3,4], [5,6]).shape
(2, 2, 2)
For variables which contain other variable types, the contained variables are also sliced. So slicing a coordinate also slices its bounds, if there are any; and slicing a space also slices its grid by applying each dimension’s slice to the appropriate grid elements.
Size 1 dimensions resulting from a slice are always retained. However, a size 1 dimension may optionally be dropped from the space’s data and its dimension coordinate reduced to a scalar if no auxiliary coordinates or cell measures span that dimension. Therefore, spaces and dimension coordinates may contain scalar data, but auxiliary coordinates and cell measures may not. A space’s squash method will carry out this reduction.
A space may be resized by specifying conditions on its coordinate’s values. These conditions are specified as arguments to a call to the space’s slice attribute (as opposed to indexing it). Note that only a space’s slice attribute may be called in this manner.
>>> s
<CF Space: air_temperature(7070, 64, 128)>
>>> print s
Data : air_temperature(time, latitude, longitude)
Cell methods : time: mean (interval: 1.0 month)
Dimensions : time(7070) -> 450-11-16 00:00:00 to 1049-12-16 12:00:00
: latitude(64) -> -87.8638 to 87.8638 degrees_north
: longitude(128) -> 0 to 357.1875 degrees_east
: height(1) -> 2 m
Auxiliary coords:
>>> s.slice[:,32, 64]
<CF Space: air_temperature(7070, 1, 1)
>>> s.slice(longitude=0.0, latitude=0.0)
<CF Space: air_temperature(7070, 1, 1)
Assignment to the data array is done with the same indexing of the slice attribute as for data access. Assignment can not change the shape of the data:
>>> v.shape
(10, 20, 30)
>>> v.slice[0, 0, 0] = 273.15
>>> v.varray[0,0,0]
273.15
>>> v.slice[0] = 273.15
>>> v.shape
(10, 20, 30)
>>> v.slice[1,2,:] = range(30)
>>> v.slice[...] = numpy.arange(6000).reshape(10, 20, 30)
The data may be stored as a numpy array or as a file pointer. A file pointer may be any kind of object which has attributes shape, size, ndim and dtype; and is indexable all in the same manner as a numpy array. A netCDF4.Variable instance is an example of a vaild file pointer object. Accessing the data makes no distinction between the two storage methods, but there are I/O, memory and speed performance issues to consider. If the data are a file pointer then accessing any part of the array will (at present) cause the entire array to be read from disk into memory and stored as a numpy array, replacing the original file pointer. Refer to read and write.
The numpy array or file pointer is held inside a Data instance and a variable’s data may be completely replaced by assigning a Data object to its _data attribute:
>>> type(ncvariable)
<type 'netCDF4.Variable'>
>>> v._data = cf.Data(ncvariable)
>>> v._data = cf.Data(numpy.arange(10))
Note that this assignment makes does not check on consistency with the variable’s metadata.
A deep copy of a variable may be created using its copy method or, equivalently, using the copy module.
>>> w = v.copy()
>>> w = copy.deepcopy(v)
This type of copy does not read any data from disk, i.e. it will not convert a file pointer to a numpy array. A third type of copy, however, will do such a conversion, namely slicing the variable with an ellipsis (or equivalent):
>>> v.type
<type 'netCDF4.Variable'>
>>> w = v.copy()
>>> w.type, v.type
(<type 'netCDF4.Variable'>, <type 'netCDF4.Variable'>)
>>> w = v.slice[...]
>>> w.type, v.type
(<type 'numpy.ndarray'>, <type 'numpy.ndarray'>)
The following operators, operations and assignments are overloaded in a variable to apply element-wise to the variable’s data numpy array.
Comparison operators:
==, !=, >, <, >=, <=
Binary arithmetic operations:
+, -, *, /, //, %, pow(), **, &, ^, |
Unary arithmetic operations:
-, +, abs(), ~
Augmented arithmetic assignments:
+=, -=, *=, /=, //=, %=, **=, &=, ^=, |=
Either side of an arithmetic operation may be a variable (in which case its data as a numpy array are used) or any object allowed by the equivalent numpy operation. Note that, as usual, if the left hand side object supports the operation with the right hand side then the returned object will be of former’s type:
>>> type(v)
<class 'cf.space.Space'>
>>> type(2.5 + v)
<class 'cf.space.Space'>
>>> type(v.varray + v)
<type 'numpy.ndarray'>
Apart from this exception, the unary and binary arithmetic operations return a new variable with modified data. The augmented arithmetic assignments change a variable’s data an in-place.
Numeric equalities for the == and != comparisons are determined to within a tolerance which may be adjusted.
Most objects defined in the cf package have an equals method which determines the congruence of two instances. The aspects of this equality vary between objects, but for all objects numeric equalities are tested to within a tolerance defined by parameters ‘rtol’ (relative tolerance) and ‘atol’ (absolute tolerance), where two numbers a and b (from the left and right sides respectively of the comparison) are considered equal if
Default tolerances may be found and set with the functions RTOL and ATOL. Numerically tolerant equality of two objects is also tested by the equals function.
A grid contains any number of dimensions and grid components, the latter comprising dimension coordinates, auxiliary coordinates, cell measures and transforms.
A space’s grid is stored in its grid attribute, the value of which is a Grid object.
The dimensions of the grid, and of its associated space, are given by the grid’s dimension_sizes and dimensions attributes.
The dimension_sizes attribute is a dictionary whose keys are dimension identifiers and values are positive integer dimension sizes. A dimension identifier is the string ‘dim’ suffixed by an arbitrary, but unique, integer. For example:
>>> g
<CF Grid: (30, 24, 1, 17)>
>>> g.dimension_sizes
{'dim0': 1, 'dim1': 17, 'dim2': 30, 'dim3': 24}
The dimensions attribute specifies which dimensions relate to each grid component (coordinates and cell measures) and to the space which holds the grid. For example:
>>> g.dimensions
{'data': ['dim1', 'dim2', 'dim3'],
'aux0': ['dim2', 'dim3'],
'aux1': ['dim2', 'dim3'],
'cm0' : ['dim2', 'dim3'],
'dim0': ['dim0'],
'dim1': ['dim1'],
'dim2': ['dim2'],
'dim3': ['dim3']}
A key of this dictionary identifies a grid component or the space’s data as follows:
| Key prefix | Description |
|---|---|
| aux | Auxiliary coordinate |
| cm | Cell measures |
| data | The space containing the grid |
| dim | Dimension coordinate |
Each key’s value is an ordered list which corresponds to the shape of that component’s data, with two exceptions:
- If the space’s data is a scalar then its value is an empty list
- If a dimension coordinate is a scalar it retains its dimension as a single element list.
An arbitrary non-negative integer after the prefix discerns between grid components of the same type.
It is possible for a grid dimension to have no grid components.
The grid object is a kind of dictionary which support the built-in dictionary’s methods. Its keys are the identifiers of the grid components dimension coordinates, auxiliary coordinates and cell measures. Each key’s value stores the variable it identifies. For example, a grid’s key and value pairs may be:
>>> g.keys()
['dim0', 'dim1', 'dim2', 'dim3', 'aux0', 'aux1']
>>> for key, value in g.iteritems():
print key, ':', repr(value)
dim0: <CF Coordinate: sigma(20)>
dim1: <CF Coordinate: time(12)>
dim2: <CF Coordinate: latitude(111)>
dim3: <CF Coordinate: longitude(106)>
aux0: <CF Coordinate: grid_latitude(111, 106)>
aux1: <CF Coordinate: grid_longitude(111, 106)>
The grid may have any number of transforms describing projection parameters (CF grid mappings) or potiential auxiliary coordinates (coordinates defined by CF formula_terms equations). These are stored in the grid’s transform attribute, which is a dictionary of Transform objects. For example:
>>> g.transform
{'trans0': <CF Transform: atmosphere_sigma_coordinate>,
'trans1': <CF Transform: rotated_latitude_longitude>}
>>> cf.dump(g.transform['trans0'])
atmosphere_sigma_coordinate transform
-------------------------------------
Transform['ps'] = <CF Space: surface_air_pressure(106, 111)>
Transform['ptop'] = <CF Space: ptop>
Transform['sigma'] = 'dim0'
>>> cf.dump(g.transform['trans1'])
rotated_latitude_longitude transform
------------------------------------
Transform['trans0']['grid_mapping_name'] = 'rotated_latitude_longitude'
Transform['trans0']['grid_north_pole_latitude'] = 70.0
Transform['trans0']['grid_north_pole_longitude'] = 100.0
Transform['trans0']['north_pole_grid_longitude'] = -80.0
A transform may also be linked with any number of the grid’s coordinates via their transform attributes:
>>> g['dim0']
<CF Coordinate: sigma(20)>
>>> g['dim0'].transform
'trans0'
>>> g['dim2'].transform, g['dim3'].transform
('trans1', trans1')
>>> g['aux0'].transform, g['aux1'].transform
('trans1', trans1')
Attributes may be set on a variable in the usual way, but these will not be recognized as CF attributes. Some attributes, such as existing methods, are reserved and may not be set:
>>> v.property = 999
>>> v.property
999
>>> v.match = 'test'
AttributeError
Refer to Variable for details on reserved attributes and methods.
Public variable attributes are those which you expect to read from or written to a CF variable on disk. They are set, retrieved and deleted with the methods setpub, getpub and delpub respectively, which allow public attributes to share names with reserved attribute names, instance attributes and private attributes:
>>> v.getpub('standard_name')
AttributeError: 'Variable' object doesn't have public attribute 'standard_name'
>>> v.setpub('standard_name', 'air_temperature')
>>> v.getpub('standard_name')
'air_temperature'
>>> v.delpub('standard_name')
>>> v.getpub('standard_name')
AttributeError: 'Variable' object doesn't have public attribute 'standard_name'
>>> v.getpub('standard_name', 'default_value')
'default_value'
For conviencience, the variable’s pub method provides a shorter way of setting and retrieving public attribues which is exactly equivalent to either the setpub or getpub methods:
>>> v.pub(standard_name='air_temperature')
>>> v.pub('standard_name')
'air_temperature'
>>> v.delpub('standard_name')
>>> v.pub('standard_name', 'default_value')
'default_value'
Note that for a coordinate variable, b and c aare not public attr but reserved ones, reflecting there different usegae comared with cf
Private attributes comprise attributes may set directly on the instance, as above, but may also be set, retrieved and deleted with the methods setpriv, getpriv and delpriv respectively, which allow private attributes to share names with reserved attribute, instance attributes public attributes:
>>> v.setpriv('file', '../file.nc')
>>> v.getpriv('file')
'../file.nc'
>>> v.delpriv('file')
>>> v.getpriv('file', 'default_value')
'default_value'
The Variable, Coordinate and Space objects have associated list objects which allow an ordered sequence of instances to be stored as a single, list-like object:
| Class | Associated list class |
|---|---|
| Variable | VariableList |
| Coordinate | CoordinateList |
| Space | SpaceList |
These list-like objects behave in much the same way as built-in lists:
>>> type(sl)
<class 'cf.space.SpaceList'>
>>> sl
[<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>]
>>> len(sl)
3
They are indexable in same way as built-in lists returning either list or non-list types depending on the nature of the index:
>>> type(sl[0])
<class 'cf.space.Space'>
>>> sl[0]
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>
>>> type(sl[2::-2])
<class 'cf.space.SpaceList'>
>>> sl[2::-2]
[<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> type(sl[slice(1,2)])
<class 'cf.space.SpaceList'>
>>> sl[slice(1,2)]
[<CF Space: precipitation_flux(2400, 96, 192)>]
The usual built-in list methods work as expected. For example:
>>> print repr(sl.reverse()
[<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> sl.append(sl[0])
>>> sl
[<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>]
>>> len(sl)
4
>>> sl.insert(1, sl[2])
>>> sl
[<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>]
>>> len(sl)
5
>>> sl.index(sl[2])
2
>>> sl[3] in sl
True
>>> sl.extend(sl[0:2])
>>> sl
[<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>]
>>> len(sl)
7
>>> sl.pop()
<CF Space: toa_outgoing_longwave_flux_assuming_clear_sky(2400, 96, 192)>
>>> len(sl)
6
>>> sl.remove(sl[0])
>>> len(sl)
5
Non-list instances are also indexable as if they were single element lists. Indexing them is essentially a null operation, but allows them to be used interchangeably with their list counterparts. This is particularly useful in iterative situations when it is not known if an instance is a variable or its list counterpart:
>>> type(x)
<class 'cf.space.Space'>
>>> type(x[0])
<class 'cf.space.Space'>
>>> type(y)
<class 'cf.space.SpaceList'>
>>> len(x), len(y)
(1, 3)
>>> for i in x:
print type(i), repr(i)
<class 'cf.space.Space'> <CF Space: air_density(2400, 96, 192)>
>>> for i in y:
print type(i), repr(i)
<class 'cf.space.Space'> <CF Space: air_pressure(2400, 96, 192)>
<class 'cf.space.Space'> <CF Space: air_temperature(2400, 96, 192)>
<class 'cf.space.Space'> <CF Space: precipitation_flux(2400, 96, 192)>
Attributes and methods of a variable are all available on its list counterpart by broadcasting to each element of the list:
>>> sl.units
['Pa', 'K', 'kg m-2 s-1']
>>> sl[1].units
'K'
>>> sl.units[1]
'K'
>>> sl[1:2].units
['K']
>>> sl.new_attribute = 'test'
>>> sl.new_attribute
['test', 'test', 'test']
>>> s = sl[2]
>>> s.new_attribute
'test'
>>> sl.name()
['air_pressure',
'air_temperature',
'precipitation_flux']
>>> sl.__repr__()
[<CF Space: air_pressure(2400, 96, 192)>,
<CF Space: air_temperature(2400, 96, 192)>,
<CF Space: precipitation_flux(2400, 96, 192)>]
>>> sl.match(units='K')
[False, True, False]
Use the pub and priv (or getpub and getpriv) methods to return defualts values for missing attributes:
>>> sl._FillValue
AttributeError: Can't get 'Space' attribute '_FillValue'
>>> sl.pub('_FillValue')
AttributeError: Can't get 'Space' attribute '_FillValue'
>>> sl.pub('_FillValue', None)
[None, None, None, -1.0e+30]
| Class | Description | Parent class |
|---|---|---|
| Variable | Abstract base class | object |
| CfList | Abstract base class | MutableSequence |
| CfDict | Abstract base class | MutableMapping |
| Space | Space | Variable |
| Coordinate | Dimension or auxiliary coordinate | Variable |
| CoordinateBounds | Coordinate bounds | Variable |
| CellMeasures | Cell measures | Variable |
| Grid | Grid | CfDict |
| Transform | Coordinate transforms | CfDict |
| CellMethods | Cell methods | CfList |
| Units | Comparison expression | object |
| Data | Comparison expression | object |
| VariableList | List of variables | CfList |
| CoordinateList | List of coordinates | VariableList |
| SpaceList | List of spaces | VariableList |
| Comparison | Comparison expression | object |
| Function | Description |
|---|---|
| dump | Print the string returned from an object’s dump method. |
| eq | Create a Comparison object. |
| equals | Determine whether two objects are congruent. |
| ATOL | Absolute tolerance for numerical equality. |
| RTOL | Relative tolerance for numerical equality. |
| ge | Create a Comparison object. |
| gt | Create a Comparison object. |
| inside | Create a Comparison object. |
| le | Create a Comparison object. |
| lt | Create a Comparison object. |
| ne | Create a Comparison object. |
| outside | Create a Comparison object. |
| read | Read spaces from netCDF files. |
| write | Write spaces to a netCDF file. |