The cf package allows a data array and its associated metadata to be contained and manipulated as a single entity called a field, which is stored in a Field object.
Much of the basic field manipulation syntax can be seen in this simple read-modify-write example which:
- Reads a field from a file on disk and find out information about it.
- Modifies an attribute and the units of its data.
- Modifies the data values.
- Modifies a subset of the data values.
- Writes it out to another file on disk.
The example may be reproduced by downloading the sample netCDF file (file.nc) (taking care not to overwrite an existing file with that name).
1. Import the cf package.
>>> import cf
2. Read a field from disk and find a summary of its contents.
>>> f = cf.read('file.nc')[0]
>>> type(f)
<class 'cf.field.Field'>
>>> f
<CF Field: AIR_TEMP(4, 5)>
>>> print f
Data : AIR_TEMP(latitude, longitude)
Cell methods : time: mean
Dimensions : time(1) = [15] days since 2000-1-1
: latitude(4) = [-2.5, ..., 5] degrees_north
: longitude(5) = [0, ..., 15] degrees_east
: height(1) = [2] m
Auxiliary coords:
3. Find all of the field’s attributes and its data array as a masked numpy array.
>>> f.attributes
{'Conventions': 'CF-1.0',
'_FillValue': 1e+20,
'cell_methods': <CF CellMethods: time: mean>,
'experiment_id': 'stabilization experiment (SRES A1B)',
'long_name': 'Surface Air Temperature',
'standard_name': 'AIR_TEMP',
'title': 'SRES A1B',
'units': 'K'}
>>> f.array
masked_array(data =
[[ 274.15, 276.15, 275.15, 277.15, 278.15],
[ 274.15, 275.15, 276.15, 277.15, 276.15],
[ 277.15, 275.15, 278.15, 274.15, 278.15],
[ 276.15, 274.15, 275.15, 277.15, 274.15]],
mask = False,
fill_value = 1e+20)
4.. Modify the field’s standard name attribute and change the field’s data from units of Kelvin to Celsius.
Note
Changing the units automatically changes the data when it is next accessed.
>>> f.standard_name
'AIR_TEMP'
>>> f.standard_name = 'air_temperature'
>>> f.standard_name
'air_temperature'
>>> f.Units -= 273.15
>>> f.units
'K @ 273.15'
>>> f.array
masked_array(data =
[[ 1. 3. 2. 4. 5.]
[ 1. 2. 3. 4. 3.]
[ 4. 2. 5. 1. 5.]
[ 3. 1. 2. 4. 1.]],
mask = False,
fill_value = 1e+20)
>>> print f
Data : air_temperature(latitude, longitude)
Cell methods : time: mean
Dimensions : time(1) = [15] days since 2000-1-1
: latitude(4) = [-2.5, ..., 5] degrees_north
: longitude(5) = [0, ..., 15] degrees_east
: height(1) = [2] m
Auxiliary coords:
5. Check that the field has ‘temperature’ in its standard name and that at least one of its longitude coordinate values is greater than 0.0.
>>> f.match(attr = {'standard_name': '.*temperature.*'},
coord = {'longitude' : cf.gt(0.)})
True
6. Modify the data values.
>>> g = f + 100
>>> g.array
masked_array(data =
[[ 101. 103. 102. 104. 105.]
[ 101. 102. 103. 104. 103.]
[ 104. 102. 105. 101. 105.]
[ 103. 101. 102. 104. 101.]],
mask = False,
fill_value = 1e+20)
>>> g = f + f
>>> g.array
masked_array(data =
[[ 2. 6. 4. 8. 10.]
[ 2. 4. 6. 8. 6.]
[ 8. 4. 10. 2. 10.]
[ 6. 2. 4. 8. 2.]],
mask = False,
fill_value = 1e+20)
>>> g = f / cf.Data(2, 'seconds')
>>> g.array
masked_array(data =
[[0.5 1.5 1.0 2.0 2.5]
[0.5 1.0 1.5 2.0 1.5]
[2.0 1.0 2.5 0.5 2.5]
[1.5 0.5 1.0 2.0 0.5]],
mask = False,
fill_value = 1e+20)
>>> g.Units
<CF Units: s-1.K>
7. Access and modify a subset of the data values.
>>> g = f.subset[0::2, 2:4]
>>> g.array
masked_array(data =
[[ 2. 4.]
[ 5. 1.]],
mask = False,
fill_value = 1e+20)
>>> f.subset[0::2, ...] = -10
>>> f.array
masked_array(data =
[[-10. -10. -10. -10. -10.]
[ 1. 2. 3. 4. 3.]
[-10. -10. -10. -10. -10.]
[ 3. 1. 2. 4. 1.]],
mask = False,
fill_value = 1e+20)
8. Write the modified field to disk.
>>> cf.write(f, 'newfile.nc')
The read function will read CF-netCDF and PP format files from disk (or netCDF file from an OPeNDAP server) and return their contents as a field list, i.e. an ordered collection of fields stored in a FieldList object:
>>> f = cf.read('data.nc')
>>> type(f)
<class 'cf.field.FieldList'>
>>> f
[<CF Field: pressure(30, 24)>,
<CF Field: u_compnt_of_wind(19, 29, 24)>,
<CF Field: v_compnt_of_wind(19, 29, 24)>,
<CF Field: potential_temperature(19, 30, 24)>]
>>> type(f[-1])
<class 'cf.field.Field'>
>>> f[-1]
<CF Field: potential_temperature(19, 30, 24)>
The read function always returns a field list as opposed to a field. If a single field as a Field object is required, then the read function (or rather its returned field list) may be indexed:
>>> f = cf.read('file1.nc')[0]
>>> type(f)
<class 'cf.field.Field'>
See the section on variable lists for more details on fields and list of fields.
>>> f = cf.read('*.nc')
>>> f = cf.read('file[1-9a-c].nc')
>>> f = cf.read('dir*/*.pp')
>>> f = cf.read(['file1.nc', 'file2.nc'])
File name suffices (such as .nc and .pp) are not required to define a file’s format.
The write function will write fields to a CF-netCDF file on disk:
>>> type(g)
<class 'cf.field.Field'>
>>> cf.write(g, 'newfile.nc')
>>> type(f)
<class 'cf.field.FieldList'>
>>> cf.write(f, 'newfile.nc')
A sequence of fields and field lists may be written to the same file:
>>> cf.write([f, g], 'newfile.nc')
All of the input fields are written to the same output file, but if metadata (such as coordinates) are identical in two or more fields then that metadata is only written once to the output file.
Output file names are arbitrary (i.e. they do not require a particular suffix).
The field’s standard and non-standard CF attributes are returned by the attributes attribute:
>>> f.attributes
{'_FillValue': 1e+20,
'cell_methods': <CF CellMethods: time: mean>,
'standard_name': 'air_temperature',
'units': 'K'}
Individual attributes recognised by the CF conventions (such as standard_name) may be set, retrieved and deleted as standard python object attributes:
>>> f.standard_name = 'air_temperature'
>>> f.standard_name
'air_temperature'
>>> del f.standard_name
>>> setattr(f, 'standard_name', 'air_pressure')
>>> getattr(f, 'standard_name')
'air_pressure'
>>> delattr(f, 'standard_name')
Any attribute (recognised by the CF conventions or not) may be retrieved with the field’s getattr method, which accepts an optional default value argument in the same way as the python built-in getattr function:
>>> f.getattr('standard_name')
'air_temperature'
>>> f.getattr('non_standard_attribute', None)
3.5
>>> f.hasattr('another_non_standard_attribute')
False
>>> f.getattr('another_non_standard_attribute', None)
None
Any attribute (recognised by the CF conventions or not) may be set with the field’s setattr method:
>>> f.setattr('standard_name', 'air_temperature')
>>> f.setattr('non_standard_attribute', 3.5)
Any attribute (recognised by the CF conventions or not) may be deleted with the field’s delattr method.
>>> f.delattr('standard_name')
>>> f.delattr('non_standard_attribute')
Other attributes (called private attributes) which do not clash with reserved names (such as ‘long_name’, ‘match’, etc.) may be set, retrieved and deleted as usual, but they will not appear in the attributes returned by the attributes dictionary:
>>> f.ncvar = 'tas'
>>> f.ncvar
>>> 'tas'
>>> del f.ncvar
>>> f.file
'file.nc'
Fields may be selected with the match and extract. These methods take conditions on field attributes and coordinates as inputs:
>>> f
[<CF Field: eastward_wind(110, 106)>,
<CF Field: air_temperature(12, 73, 96)>]
>>> f.match(attr={'standard_name': '.*temperature'})
[False, True]
>>> g = f.extract(attr={'standard_name': '.*temperature'}, coord={'longitude': 0})
>>> g
[<CF Field: air_temperature(12, 73, 96)>]
The fields data array may be retrieved as an independent numpy array with the field’s array attribute:
>>> f.array
masked_array(data =
[[ 2. 4.]
[ 5. 1.]],
mask = False,
fill_value = 1e+20)
The first and last elements of the data array may be retrieved as with the field’s first_datum and last_datum attributes:
>>> f.first_datum
2.0
>>> f.last_datum
1.0
A coordinate of the field’s coordinate system is returned by the field’s coord method. A coordinate has the same attribute and data access principles as a field:
>>> c = f.coord('time')
>>> c
<CF Coordinate: time(12)>
>>> c.attributes
{'_FillValue': None,
'axis': 'T',
'bounds': <CF CoordinateBounds: time_bnds(12, 2)>,
'calendar': 'noleap',
'long_name': 'time',
'standard_name': 'time',
'units': 'days since 0000-1-1'}
>>> c.Units
<CF Units: days since 0000-1-1 calendar=noleap>
>>> c.array
masked_array(data = [ 0 30 60 90 120 150 180 210 240 270 300 330],
mask = False,
fill_value = 999999)