cf.Data

class cf.Data(data=None, units=None, fill_value=None, hardmask=True, chunk=True, loadd=None, dt=False)[source]

Bases: object

An N-dimensional data array with units and masked values.

  • Contains an N-dimensional, indexable and broadcastable array with many similarities to a numpy array.
  • Contains the units of the array elements.
  • Supports masked arrays, regardless of whether or not it was initialised with a masked array.
  • Stores and operates on data arrays which are larger then the available memory.

Indexing

A data array is indexable in a similar way to numpy array:

>>> d.shape
(12, 19, 73, 96)
>>> d[...].shape
(12, 19, 73, 96)
>>> d[slice(0, 9), 10:0:-2, :, :].shape
(9, 5, 73, 96)

There are three extensions to the numpy indexing functionality:

  • Size 1 dimensions are never removed bi indexing.

    An integer index i takes the i-th element but does not reduce the rank of the output array by one:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, ...].shape
    (1, 19, 73, 96)
    >>> d[:, 3, slice(10, 0, -2), 95].shape
    (12, 1, 5, 1)
    

    Size 1 dimensions may be removed with the squeeze method.

  • The indices for each axis work independently.

    When more than one dimension’s slice is a 1-d boolean sequence or 1-d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, :, [0, 1], [0, 13, 27]].shape
    (1, 19, 2, 3)
    
  • Boolean indices may be any object which exposes the numpy array interface.

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
    

Cyclic axes

Miscellaneous

A Data object is picklable.

A Data object is hashable, but note that, since it is mutable, its hash value is only valid whilst the data array is not changed in place.

Initialization

Parameters :
data : array-like, optional

The data for the array.

units : str or Units, optional

The units of the data. By default the array elements are dimensionless.

fill_value : optional

The fill value of the data. By default, or if None, the numpy fill value appropriate to the array’s data type will be used.

hardmask : bool, optional

If False then the mask is soft. By default the mask is hard.

chunk : bool, optional

If False then the data array will be stored in a single partition. By default the data array will be partitioned if it is larger than the chunk size, as returned by the cf.CHUNKSIZE function.

dt : bool, optional

If True then strings (such as '1990-12-1 12:00') given by the data argument are interpreted as date-times. By default they are not interpreted as date-times.

loadd : dict, optional

Initialise the data array from a dictionary serialization of a cf.Data object. All other arguments are ignored.

Examples

>>> d = cf.Data(5)
>>> d = cf.Data([1,2,3], units='K')
>>> import numpy   
>>> d = cf.Data(numpy.arange(10).reshape(2,5), units=cf.Units('m/s'), fill_value=-999)
>>> d = cf.Data(tuple('fly'))

Data attributes

array A numpy array copy the data array.
data The data array object as an object identity.
day The day of each data array element.
dtarray An independent numpy array of date-time objects.
dtvarray A numpy array view the data array converted to date-time objects.
dtype The numpy data type of the data array.
fill_value The data array missing data value.
hardmask Whether the mask is hard (True) or soft (False).
hour The hour of each data array element.
ismasked True if the data array has any masked values.
isscalar True if the data array is a 0-d scalar array.
mask The boolean missing data mask of the data array.
minute The minute of each data array element.
month The month of each data array element.
nbytes Total bytes consumed by the elements of the array.
ndim Number of dimensions in the data array.
second The second of each data array element.
shape Tuple of the data array’s dimension sizes.
size Number of elements in the data array.
unique The unique elements of the array.
Units The cf.Units object containing the units of the data array.
varray A numpy array view the data array.
year The year of each data array element.

Data methods

all Test whether all data array elements evaluate to True.
allclose Returns True if two broadcastable arrays have equal values, False
cf.Data.amax
cf.Data.amin
any Test whether any data array elements evaluate to True.
cf.Data.average
asdatetime Change the internal representation of data array elements from numeric reference times to datatime-like objects.
asreftime Change the internal representation of data array elements from datatime-like objects to numeric reference times.
binary_mask Return a binary missing data mask of the data array.
chunk Partition the data array
clip Clip (limit) the values in the data array in place.
close Close all files referenced by the data array.
copy Return a deep copy.
cos Take the trigonometric cosine of the data array in place.
datum Return an element of the data array as a standard Python scalar.
dump Return a string containing a full description of the instance.
dumpd Return a dictionary serialization of the data array.
equals True if two data arrays are logically equal, False otherwise.
cf.Data.equivalent
expand_dims Expand the shape of the data array in place.
flat Return a flat iterator over elements of the data array.
flip Flip (reverse the direction of) axes of the data array in place.
func Apply an element-wise array operation to the data array in place.
isclose Return a boolean data array showing where two broadcastable arrays have equal values.
loadd Reset the data array in place from a dictionary serialization.
mask_invalid Mask the array where invalid values occur (NaNs or infs).
mid_range Collapse axes with their mid range.
ndindex Return an iterator over the N-dimensional indices of the data array.
outerproduct Compute the outer product with another data array.
override_calendar Override the data array units.
override_units Override the data array units.
partition_boundaries Return the partition boundaries for each partition matrix dimension.
save_to_disk
sd Collapse axes by calculating their standard deviation.
setdata Set data array elements depending on a condition.
sin Take the trigonometric sine of the data array in place.
squeeze Remove size 1 axes from the data array in place.
sum Collapse axes with their sum.
swapaxes Interchange two axes of an array.
tan Take the trigonometric tangent of the data array element-wise.
to_disk Store the data array on disk in place.
to_memory Store each partition’s data in memory in place if the master array is smaller than the chunk size.
transpose Permute the axes of the data array.
var Collapse axes with their weighted variance.

Data static methods

mask_fpe Masking of floating-point errors in the results of arithmetic operations.
seterr Set how floating-point errors in the results of arithmetic operations are handled.

Data arithmetic and comparison operations

Arithmetic, bitwise and comparison operations are defined as element-wise data array operations which yield a new cf.Data object or, for augmented assignments, modify the data array in-place.

Comparison operators

__lt__ The rich comparison operator <
__le__ The rich comparison operator <=
__eq__ The rich comparison operator ==
__ne__ The rich comparison operator !=
__gt__ The rich comparison operator >
__ge__ The rich comparison operator >=

Truth value of an array

__nonzero__ Truth value testing and the built-in operation bool

Binary arithmetic operators

__add__ The binary arithmetic operation +
__sub__ The binary arithmetic operation -
__mul__ The binary arithmetic operation *
__div__ The binary arithmetic operation /
__truediv__ The binary arithmetic operation / (true division)
__floordiv__ The binary arithmetic operation //
__pow__ The binary arithmetic operations ** and pow
__mod__ The binary arithmetic operation %

Binary arithmetic operators with reflected (swapped) operands

__radd__ The binary arithmetic operation + with reflected operands
__rsub__ The binary arithmetic operation - with reflected operands
__rmul__ The binary arithmetic operation * with reflected operands
__rdiv__ The binary arithmetic operation / with reflected operands
__rtruediv__ The binary arithmetic operation / (true division) with reflected
__rfloordiv__ The binary arithmetic operation // with reflected operands
__rpow__ The binary arithmetic operations ** and pow with reflected
__rmod__ The binary arithmetic operation % with reflected operands

Augmented arithmetic assignments

__iadd__ The augmented arithmetic assignment +=
__isub__ The augmented arithmetic assignment -=
__imul__ The augmented arithmetic assignment *=
__idiv__ The augmented arithmetic assignment /=
__itruediv__ The augmented arithmetic assignment /= (true division)
__ifloordiv__ The augmented arithmetic assignment //=
__ipow__ The augmented arithmetic assignment **=
__imod__ The binary arithmetic operation %=

Unary arithmetic operators

__neg__ The unary arithmetic operation -
__pos__ The unary arithmetic operation +
__abs__ The unary arithmetic operation abs

Binary bitwise operators

__and__ The binary bitwise operation &
__or__ The binary bitwise operation |
__xor__ The binary bitwise operation ^
__lshift__ The binary bitwise operation <<
__rshift__ The binary bitwise operation >>

Binary bitwise operators with reflected (swapped) operands

__rand__ The binary bitwise operation & with reflected operands
__ror__ The binary bitwise operation | with reflected operands
__rxor__ The binary bitwise operation ^ with reflected operands
__rlshift__ The binary bitwise operation << with reflected operands
__rrshift__ The binary bitwise operation >> with reflected operands

Augmented bitwise assignments

__iand__ The augmented bitwise assignment &=
__ior__ The augmented bitwise assignment |=
__ixor__ The augmented bitwise assignment ^=
__ilshift__ The augmented bitwise assignment <<=
__irshift__ The augmented bitwise assignment >>=

Unary bitwise operators

__invert__ The unary bitwise operation ~

Data special methods

Standard library functions

__deepcopy__ Used if copy.deepcopy is called.
__hash__ The built-in function hash

Container customization

__len__ The built-in function len
__getitem__ Implement indexing
__iter__ Efficient iteration.
__setitem__ Implement indexed assignment
__contains__ Membership test operator in

String representations

__repr__ The built-in function repr
__str__ The built-in function str

Table Of Contents

Previous topic

cf.CoordinateBounds.transpose

Next topic

cf.Data.array

This Page