cf.Data

class cf.Data(data=None, units=None, fill_value=None, hardmask=True, chunk=True, loadd=None, loads=None, dt=False)

Bases: object

An N-dimensional data array with units and masked values.

• Contains an N-dimensional, indexable and broadcastable array with many similarities to a numpy array.
• Contains the units of the array elements.
• Supports masked arrays, regardless of whether or not it was initialised with a masked array.
• Stores and operates on data arrays which are larger then the available memory.

Indexing

A data array is indexable in a similar way to numpy array:

>>> d.shape
(12, 19, 73, 96)
>>> d[...].shape
(12, 19, 73, 96)
>>> d[slice(0, 9), 10:0:-2, :, :].shape
(9, 5, 73, 96)

There are three extensions to the numpy indexing functionality:

• Size 1 dimensions are never removed bi indexing.

  An integer index i takes the i-th element but does not reduce the rank of the output array by one:

  >>> d.shape
  (12, 19, 73, 96)
  >>> d[0, ...].shape
  (1, 19, 73, 96)
  >>> d[:, 3, slice(10, 0, -2), 95].shape
  (12, 1, 5, 1)
  

  Size 1 dimensions may be removed with the squeeze method.

• The indices for each axis work independently.

  When more than one dimension’s slice is a 1-d boolean sequence or 1-d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:

  >>> d.shape
  (12, 19, 73, 96)
  >>> d[0, :, [0, 1], [0, 13, 27]].shape
  (1, 19, 2, 3)
  

• Boolean indices may be any object which exposes the numpy array interface.

  >>> d.shape
  (12, 19, 73, 96)
  >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
  

Cyclic axes

Miscellaneous

A Data object is picklable.

A Data object is hashable, but note that, since it is mutable, its hash value is only valid whilst the data array is not changed in place.

Initialization

Parameters:

data: array-like, optional

The data for the array.

units : str or Units, optional

The units of the data. By default the array elements are dimensionless.

fill_value: optional

The fill value of the data. By default, or if None, the numpy fill value appropriate to the array’s data type will be used.

hardmask: bool, optional

If False then the mask is soft. By default the mask is hard.

chunk: bool, optional

If False then the data array will be stored in a single partition. By default the data array will be partitioned if it is larger than the chunk size, as returned by the cf.CHUNKSIZE function.

dt: bool, optional

If True then strings (such as '1990-12-1 12:00') given by the data argument are interpreted as date-times. By default they are not interpreted as date-times.

loadd: dict, optional

Initialise the data array from a dictionary serialization of a cf.Data object. All other arguments are ignored.

loads: str, optional

Initialise the data array from a JSON string serialization of a Data object. All other arguments are ignored.

Examples:

>>> d = cf.Data(5)
>>> d = cf.Data([1,2,3], units='K')
>>> import numpy   
>>> d = cf.Data(numpy.arange(10).reshape(2,5), units=cf.Units('m/s'), fill_value=-999)
>>> d = cf.Data(tuple('fly'))

Data attributes

array	A numpy array copy the data array.
binary_mask	A binary (0 and 1) mask of the data array.
data	The data array object as an object identity.
day	The day of each data array element.
dtarray	An independent numpy array of date-time objects.
dtype	The numpy data type of the data array.
fill_value	The data array missing data value.
hardmask	Whether the mask is hard (True) or soft (False).
hour	The hour of each data array element.
ismasked	True if the data array has any masked values.
isscalar	True if the data array is a 0-d scalar array.
mask	The boolean missing data mask of the data array.
minute	The minute of each data array element.
month	The month of each data array element.
nbytes	Total number of bytes consumed by the elements of the array.
ndim	Number of dimensions in the data array.
second	The second of each data array element.
shape	Tuple of the data array’s dimension sizes.
size	Number of elements in the data array.
Units	The cf.Units object containing the units of the data array.
varray	A numpy array view the data array.
year	The year of each data array element.

Data methods

all	Test whether all data array elements evaluate to True.
allclose	Returns True if two broadcastable arrays have equal values, False otherwise.
max	Collapse axes with their maximum.
min	Collapse axes with their minimum.
any	Test whether any data array elements evaluate to True.
chunk	Partition the data array.
ceil	Return the ceiling of the data array.
clip	Clip (limit) the values in the data array in place.
close	Close all files referenced by the data array.
copy	Return a deep copy.
cos	Take the trigonometric cosine of the data array in place.
datum	Return an element of the data array as a standard Python scalar.
dump	Return a string containing a full description of the instance.
dumpd	Return a serialization of the data array.
equals	True if two data arrays are logically equal, False otherwise.
cf.Data.equivalent
expand_dims	Expand the shape of the data array in place.
files	Return the names of files containing parts of the data array.
flat	Return a flat iterator over elements of the data array.
flip	Reverse the direction of axes of the data array.
floor	Return the floor of the data array.
func	Apply an element-wise array operation to the data array.
HDF_chunks
isclose	Return a boolean data array showing where two broadcastable arrays have equal values within a tolerance.
loadd	Reset the data array in place from a data array serialization.
mask_invalid	Mask the array where invalid values occur (NaN or inf).
mean	Collapse axes with their weighted mean.
mid_range	Collapse axes with the unweighted average of their maximum and minimum values.
ndindex	Return an iterator over the N-dimensional indices of the data array.
outerproduct	Compute the outer product with another data array.
override_calendar	Override the calendar of the data array elements.
override_units	Override the data array units.
partition_boundaries	Return the partition boundaries for each partition matrix dimension.
range	Collapse axes with the absolute difference between their maximum and minimum values.
rint	Round elements of the data array to the nearest integer.
roll	A lot like numpy.roll
save_to_disk
sample_size	Parameters:
sd	Collapse axes by calculating their standard deviation.
sin	Take the trigonometric sine of the data array in place.
squeeze	Remove size 1 axes from the data array.
sum	Collapse axes with their sum.
sum_of_weights	Missing data array elements are omitted from the calculation.
sum_of_weights2	Missing data array elements are omitted from the calculation.
swapaxes	Interchange two axes of an array.
tan	Take the trigonometric tangent of the data array element-wise.
to_disk	Store the data array on disk.
to_memory	Store each partition’s data in memory in place if the master array is smaller than the chunk size.
transpose	Permute the axes of the data array.
trunc	Return the truncated values of the data array.
unique	The unique elements of the array.
var	Collapse axes with their weighted variance.
where	Set data array elements depending on a condition.

Data static methods

mask_fpe	Masking of floating-point errors in the results of arithmetic operations.
seterr	Set how floating-point errors in the results of arithmetic operations are handled.

Data arithmetic and comparison operations

Arithmetic, bitwise and comparison operations are defined as element-wise data array operations which yield a new cf.Data object or, for augmented assignments, modify the data array in-place.

Comparison operators

__lt__	The rich comparison operator <
__le__	The rich comparison operator <=
__eq__	The rich comparison operator ==
__ne__	The rich comparison operator !=
__gt__	The rich comparison operator >
__ge__	The rich comparison operator >=

Truth value of an array

__nonzero__

Truth value testing and the built-in operation bool

Binary arithmetic operators

__add__	The binary arithmetic operation +
__sub__	The binary arithmetic operation -
__mul__	The binary arithmetic operation *
__div__	The binary arithmetic operation /
__truediv__	The binary arithmetic operation / (true division)
__floordiv__	The binary arithmetic operation //
__pow__	The binary arithmetic operations ** and pow
__mod__	The binary arithmetic operation %

Binary arithmetic operators with reflected (swapped) operands

__radd__	The binary arithmetic operation + with reflected operands
__rsub__	The binary arithmetic operation - with reflected operands
__rmul__	The binary arithmetic operation * with reflected operands
__rdiv__	The binary arithmetic operation / with reflected operands
__rtruediv__	The binary arithmetic operation / (true division) with reflected
__rfloordiv__	The binary arithmetic operation // with reflected operands
__rpow__	The binary arithmetic operations ** and pow with reflected
__rmod__	The binary arithmetic operation % with reflected operands

Augmented arithmetic assignments

__iadd__	The augmented arithmetic assignment +=
__isub__	The augmented arithmetic assignment -=
__imul__	The augmented arithmetic assignment *=
__idiv__	The augmented arithmetic assignment /=
__itruediv__	The augmented arithmetic assignment /= (true division)
__ifloordiv__	The augmented arithmetic assignment //=
__ipow__	The augmented arithmetic assignment **=
__imod__	The binary arithmetic operation %=

Unary arithmetic operators

__neg__	The unary arithmetic operation -
__pos__	The unary arithmetic operation +
__abs__	The unary arithmetic operation abs

Binary bitwise operators

__and__	The binary bitwise operation &
__or__	The binary bitwise operation |
__xor__	The binary bitwise operation ^
__lshift__	The binary bitwise operation <<
__rshift__	The binary bitwise operation >>

Binary bitwise operators with reflected (swapped) operands

__rand__	The binary bitwise operation & with reflected operands
__ror__	The binary bitwise operation | with reflected operands
__rxor__	The binary bitwise operation ^ with reflected operands
__rlshift__	The binary bitwise operation << with reflected operands
__rrshift__	The binary bitwise operation >> with reflected operands

Augmented bitwise assignments

__iand__	The augmented bitwise assignment &=
__ior__	The augmented bitwise assignment |=
__ixor__	The augmented bitwise assignment ^=
__ilshift__	The augmented bitwise assignment <<=
__irshift__	The augmented bitwise assignment >>=

Unary bitwise operators

__invert__

The unary bitwise operation ~

Data special methods

Standard library functions

__deepcopy__	Used if copy.deepcopy is called.
__hash__	The built-in function hash

Container customization

__len__	The built-in function len
__getitem__	Implement indexing
__iter__	Efficient iteration.
__setitem__	Implement indexed assignment
__contains__	Membership test operator in

String representations

__repr__	The built-in function repr
__str__	The built-in function str