cf.Data

class cf.Data(data=None, units=None, fill_value=None, hardmask=True, chunk=True, loadd=None, dt=False)[source]

Bases: object

An N-dimensional data array with units and masked values.

  • Contains an N-dimensional, indexable and broadcastable array with many similarities to a numpy array.
  • Contains the units of the array elements.
  • Supports masked arrays, regardless of whether or not it was initialised with a masked array.
  • Stores and operates on data arrays which are larger then the available memory.

Indexing

A data array is indexable in a similar way to numpy array:

>>> d.shape
(12, 19, 73, 96)
>>> d[...].shape
(12, 19, 73, 96)
>>> d[slice(0, 9), 10:0:-2, :, :].shape
(9, 5, 73, 96)

There are three extensions to the numpy indexing functionality:

  • Size 1 dimensions are never removed bi indexing.

    An integer index i takes the i-th element but does not reduce the rank of the output array by one:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, ...].shape
    (1, 19, 73, 96)
    >>> d[:, 3, slice(10, 0, -2), 95].shape
    (12, 1, 5, 1)
    

    Size 1 dimensions may be removed with the squeeze method.

  • The indices for each axis work independently.

    When more than one dimension’s slice is a 1-d boolean sequence or 1-d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, :, [0, 1], [0, 13, 27]].shape
    (1, 19, 2, 3)
    
  • Boolean indices may be any object which exposes the numpy array interface.

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
    

Cyclic axes

Miscellaneous

A Data object is picklable.

A Data object is hashable, but note that, since it is mutable, its hash value is only valid whilst the data array is not changed in place.

Initialization

Parameters:
data : array-like, optional

The data for the array.

units : str or Units, optional

The units of the data. By default the array elements are dimensionless.

fill_value : optional

The fill value of the data. By default, or if None, the numpy fill value appropriate to the array’s data type will be used.

hardmask : bool, optional

If False then the mask is soft. By default the mask is hard.

chunk : bool, optional

If False then the data array will be stored in a single partition. By default the data array will be partitioned if it is larger than the chunk size, as returned by the cf.CHUNKSIZE function.

dt : bool, optional

If True then strings (such as '1990-12-1 12:00') given by the data argument are interpreted as date-times. By default they are not interpreted as date-times.

loadd: dict, optional

Initialise the data array from a dictionary serialization of a cf.Data object. All other arguments are ignored.

Examples:
>>> d = cf.Data(5)
>>> d = cf.Data([1,2,3], units='K')
>>> import numpy   
>>> d = cf.Data(numpy.arange(10).reshape(2,5), units=cf.Units('m/s'), fill_value=-999)
>>> d = cf.Data(tuple('fly'))

Data attributes

array A numpy array copy the data array.
binary_mask A binary (0 and 1) mask of the data array.
data The data array object as an object identity.
day The day of each data array element.
dtarray An independent numpy array of date-time objects.
dtype The numpy data type of the data array.
fill_value The data array missing data value.
hardmask Whether the mask is hard (True) or soft (False).
hour The hour of each data array element.
ismasked True if the data array has any masked values.
isscalar True if the data array is a 0-d scalar array.
mask The boolean missing data mask of the data array.
minute The minute of each data array element.
month The month of each data array element.
nbytes Total number of bytes consumed by the elements of the array.
ndim Number of dimensions in the data array.
second The second of each data array element.
shape Tuple of the data array’s dimension sizes.
size Number of elements in the data array.
Units The cf.Units object containing the units of the data array.
varray A numpy array view the data array.
year The year of each data array element.

Data methods

all Test whether all data array elements evaluate to True.
allclose Returns True if two broadcastable arrays have equal values, False otherwise.
max Collapse axes with their maximum.
min Collapse axes with their minimum.
any Test whether any data array elements evaluate to True.
chunk Partition the data array.
ceil Return the ceiling of the data array.
clip Clip (limit) the values in the data array in place.
close Close all files referenced by the data array.
copy Return a deep copy.
cos Take the trigonometric cosine of the data array in place.
datum Return an element of the data array as a standard Python scalar.
dump Return a string containing a full description of the instance.
dumpd Return a dictionary serialization of the data array.
equals True if two data arrays are logically equal, False otherwise.
cf.Data.equivalent
expand_dims Expand the shape of the data array in place.
files Return the names of files containing parts of the data array.
flat Return a flat iterator over elements of the data array.
flip Reverse the direction of axes of the data array.
floor Return the floor of the data array.
func Apply an element-wise array operation to the data array.
HDF_chunks
isclose Return a boolean data array showing where two broadcastable arrays have equal values within a tolerance.
loadd Reset the data array in place from a dictionary serialization.
mask_invalid Mask the array where invalid values occur (NaN or inf).
mean Collapse axes with their weighted mean.
mid_range Collapse axes with the unweighted average of their maximum and minimum values.
ndindex Return an iterator over the N-dimensional indices of the data array.
outerproduct Compute the outer product with another data array.
override_calendar Override the calendar of the data array elements.
override_units Override the data array units.
partition_boundaries Return the partition boundaries for each partition matrix dimension.
range Collapse axes with the absolute difference between their maximum and minimum values.
rint Round elements of the data array to the nearest integer.
roll A lot like numpy.roll
save_to_disk
sample_size
Parameters:
sd Collapse axes by calculating their standard deviation.
sin Take the trigonometric sine of the data array in place.
squeeze Remove size 1 axes from the data array.
sum Collapse axes with their sum.
sum_of_weights Missing data array elements are omitted from the calculation.
sum_of_weights2 Missing data array elements are omitted from the calculation.
swapaxes Interchange two axes of an array.
tan Take the trigonometric tangent of the data array element-wise.
to_disk Store the data array on disk.
to_memory Store each partition’s data in memory in place if the master array is smaller than the chunk size.
transpose Permute the axes of the data array.
trunc Return the truncated values of the data array.
unique The unique elements of the array.
var Collapse axes with their weighted variance.
where Set data array elements depending on a condition.

Data static methods

mask_fpe Masking of floating-point errors in the results of arithmetic operations.
seterr Set how floating-point errors in the results of arithmetic operations are handled.

Data arithmetic and comparison operations

Arithmetic, bitwise and comparison operations are defined as element-wise data array operations which yield a new cf.Data object or, for augmented assignments, modify the data array in-place.

Comparison operators

__lt__ The rich comparison operator <
__le__ The rich comparison operator <=
__eq__ The rich comparison operator ==
__ne__ The rich comparison operator !=
__gt__ The rich comparison operator >
__ge__ The rich comparison operator >=

Truth value of an array

__nonzero__ Truth value testing and the built-in operation bool

Binary arithmetic operators

__add__ The binary arithmetic operation +
__sub__ The binary arithmetic operation -
__mul__ The binary arithmetic operation *
__div__ The binary arithmetic operation /
__truediv__ The binary arithmetic operation / (true division)
__floordiv__ The binary arithmetic operation //
__pow__ The binary arithmetic operations ** and pow
__mod__ The binary arithmetic operation %

Binary arithmetic operators with reflected (swapped) operands

__radd__ The binary arithmetic operation + with reflected operands
__rsub__ The binary arithmetic operation - with reflected operands
__rmul__ The binary arithmetic operation * with reflected operands
__rdiv__ The binary arithmetic operation / with reflected operands
__rtruediv__ The binary arithmetic operation / (true division) with reflected
__rfloordiv__ The binary arithmetic operation // with reflected operands
__rpow__ The binary arithmetic operations ** and pow with reflected
__rmod__ The binary arithmetic operation % with reflected operands

Augmented arithmetic assignments

__iadd__ The augmented arithmetic assignment +=
__isub__ The augmented arithmetic assignment -=
__imul__ The augmented arithmetic assignment *=
__idiv__ The augmented arithmetic assignment /=
__itruediv__ The augmented arithmetic assignment /= (true division)
__ifloordiv__ The augmented arithmetic assignment //=
__ipow__ The augmented arithmetic assignment **=
__imod__ The binary arithmetic operation %=

Unary arithmetic operators

__neg__ The unary arithmetic operation -
__pos__ The unary arithmetic operation +
__abs__ The unary arithmetic operation abs

Binary bitwise operators

__and__ The binary bitwise operation &
__or__ The binary bitwise operation |
__xor__ The binary bitwise operation ^
__lshift__ The binary bitwise operation <<
__rshift__ The binary bitwise operation >>

Binary bitwise operators with reflected (swapped) operands

__rand__ The binary bitwise operation & with reflected operands
__ror__ The binary bitwise operation | with reflected operands
__rxor__ The binary bitwise operation ^ with reflected operands
__rlshift__ The binary bitwise operation << with reflected operands
__rrshift__ The binary bitwise operation >> with reflected operands

Augmented bitwise assignments

__iand__ The augmented bitwise assignment &=
__ior__ The augmented bitwise assignment |=
__ixor__ The augmented bitwise assignment ^=
__ilshift__ The augmented bitwise assignment <<=
__irshift__ The augmented bitwise assignment >>=

Unary bitwise operators

__invert__ The unary bitwise operation ~

Data special methods

Standard library functions

__deepcopy__ Used if copy.deepcopy is called.
__hash__ The built-in function hash

Container customization

__len__ The built-in function len
__getitem__ Implement indexing
__iter__ Efficient iteration.
__setitem__ Implement indexed assignment
__contains__ Membership test operator in

String representations

__repr__ The built-in function repr
__str__ The built-in function str