cf.Data

class cf.Data(data=None, units=None, fill_value=None, hardmask=True, chunk=True, loadd=None, dt=False)[source]

Bases: object

An N-dimensional data array with units and masked values.

  • Contains an N-dimensional, indexable and broadcastable array with many similarities to a numpy array.
  • Contains the units of the array elements.
  • Supports masked arrays, regardless of whether or not it was initialized with a masked array.
  • Uses Large Amounts of Massive Arrays (LAMA) functionality to store and operate on aggregated (master) data arrays and arrays which are larger then the available memory.

Indexing

A data array is indexable in a similar way to numpy array

>>> d.shape
(12, 19, 73, 96)
>>> d[...].shape
(12, 19, 73, 96)
>>> d[slice(0, 12), 10:0:-2, :, :].shape
(12, 5, 73, 96)

There are two important extensions to the numpy indexing functionality:

  • Size 1 dimensions are never removed.

    An integer index i takes the i-th element but does not reduce the rank of the output array by one:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, ...].shape
    (1, 19, 73, 96)
    >>> d[:, 3, slice(10, 0, -2), 95].shape
    (12, 1, 5, 1)
    
  • When advanced indexing is used on more than one dimension, the advanced indices work independently.

    When more than one dimension’s slice is a 1-d boolean sequence or 1-d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:

    >>> d.shape
    (12, 19, 73, 96)
    >>> d[0, :, [0,1], [0,1,2]].shape
    (1, 19, 2, 3)
    

Miscellaneous

Data objects are picklable.

Data objects are hashable, but note that, since Data objects are mutable, their hash values may change if created at different times.

Initialization

Parameters :
data : array-like, optional

The data for the array.

units : str or Units, optional

The units of the data. By default the array elements are dimensionless.

fill_value : optional

The fill value of the data. By default, or if None, the numpy fill value appropriate to the array’s data type will be used.

hardmask : bool, optional

If False then the mask is soft. By default the mask is hard.

chunk : bool, optional

If True (the default) then the data array will be partitioned if it is larger than the chunk size. If False then the data array will be stored in a single partition.

Examples

>>> d = cf.Data(5)
>>> d = cf.Data([1,2,3], units='K')
>>> import numpy   
>>> d = cf.Data(numpy.arange(10).reshape(2,5), units=cf.Units('m/s'), fill_value=-999)
>>> d = cf.Data(tuple('fly'))

Data attributes

array A numpy array copy the data array.
data The data array object as an object identity.
dtype The numpy data type of the data array.
hardmask Whether the mask is hard (True) or soft (False).
ismasked True if the data array has any masked values.
isscalar True if the data array is a 0-d scalar array.
mask The boolean missing data mask of the data array.
ndim Number of dimensions in the data array.
shape Tuple of the data array’s dimension sizes.
size Number of elements in the data array.
Units The Units object containing the units of the data array.
unsafe_array A numpy array of the data array.
varray A numpy array view the data array.

Partition matrix attributes

pmdimensions
pmndim Number of dimensions in the partition matrix.
pmshape Tuple of the partition matrix’s dimension sizes.
pmsize Number of partitions in the partition matrix.

Data methods

add_partitions Add partition boundaries.
all Test whether all data array elements evaluate to True.
all_axis_names Return a set of all the dimension names in use by the data array.
any Test whether any data array elements evaluate to True.
asdatetime Change the internal representation of data array elements from numeric reference times to datatime-like objects.
asreftime Change the internal representation of data array elements from datatime-like objects to numeric reference times.
binary_mask Return a binary missing data mask of the data array.
change_axis_names Change the axis names.
chunk Partition the data array
clip Clip (limit) the values in the data array in place.
close Close all files referenced by the data array.
conform_args Return a dictionary of arguments for the cf.Partition.dataarray
copy Return a deep copy.
cos Take the trigonometric cosine of the data array in place.
datum Return an element of the data array as a standard Python scalar.
dump Return a string containing a full description of the instance.
dumpd Serialize the data array as a dictionary following loosely the CFA
equals True if two data arrays are logically equal, False otherwise.
equivalent True if and only if two data arrays are logically equivalent.
expand_dims Expand the shape of the data array in place.
expand_partition_dims Expand the shape of the partition matrix in place.
flat Return a flat iterator over elements of the data array.
flip Flip (reverse the direction) axes of the data array in place.
func Apply an element-wise array operation to the data array in place.
loadd Deserialize the dictionary, updating the master data array in place.
ndindex Return an iterator over the N-dimensional indices of the data array.
new_axis_identifier Return an axis name not being used by the data array.
override_calendar Override the data array units in place.
override_directions Override the data array directions in place.
override_units Override the data array units in place.
partition_boundaries Return the partition boundaries for each partition matrix dimension.
save_to_disk Return True if the master array is large enough to be saved to disk.
setdata Set data array elements depending on a condition.
sin Take the trigonometric sine of the data array in place.
squeeze Remove size 1 axes from the data array in place.
tan Take the trigonometric tangent of the data array in place.
to_disk Store the data array on disk in place.
to_memory Store each partition’s data in memory in place if the master array is smaller than the chunk size.
transpose Permute the dimensions of the data array in place.

Data arithmetic and comparison operations

Arithmetic, bitwise and comparison operations are defined as element-wise data array operations which yield a new cf.Data object or, for augmented assignments, modify the data array in-place.

Comparison operators

__lt__ The rich comparison operator <
__le__ The rich comparison operator <=
__eq__ The rich comparison operator ==
__ne__ The rich comparison operator !=
__gt__ The rich comparison operator >
__ge__ The rich comparison operator >=

Truth value of an array

__nonzero__ Truth value testing and the built-in operation bool

Binary arithmetic operators

__add__ The binary arithmetic operation +
__sub__ The binary arithmetic operation -
__mul__ The binary arithmetic operation *
__div__ The binary arithmetic operation /
__truediv__ The binary arithmetic operation / (true division)
__floordiv__ The binary arithmetic operation //
__pow__ The binary arithmetic operations ** and pow

Binary arithmetic operators with reflected (swapped) operands

__radd__ The binary arithmetic operation + with reflected operands
__rsub__ The binary arithmetic operation - with reflected operands
__rmul__ The binary arithmetic operation * with reflected operands
__rdiv__ The binary arithmetic operation / with reflected operands
__rtruediv__ The binary arithmetic operation / (true division) with reflected
__rfloordiv__ The binary arithmetic operation // with reflected operands
__rpow__ The binary arithmetic operations ** and pow with reflected

Augmented arithmetic assignments

__iadd__ The augmented arithmetic assignment +=
__isub__ The augmented arithmetic assignment -=
__imul__ The augmented arithmetic assignment *=
__idiv__ The augmented arithmetic assignment /=
__itruediv__ The augmented arithmetic assignment /= (true division)
__ifloordiv__ The augmented arithmetic assignment //=
__ipow__ The augmented arithmetic assignment **=

Unary arithmetic operators

__neg__ The unary arithmetic operation -
__pos__ The unary arithmetic operation +
__abs__ The unary arithmetic operation abs

Binary bitwise operators

__and__ The binary bitwise operation &
__or__ The binary bitwise operation |
__xor__ The binary bitwise operation ^
__lshift__ The binary bitwise operation <<
__rshift__ The binary bitwise operation >>

Binary bitwise operators with reflected (swapped) operands

__rand__ The binary bitwise operation & with reflected operands
__ror__ The binary bitwise operation | with reflected operands
__rxor__ The binary bitwise operation ^ with reflected operands
__rlshift__ The binary bitwise operation << with reflected operands
__rrshift__ The binary bitwise operation >> with reflected operands

Augmented bitwise assignments

__iand__ The augmented bitwise assignment &=
__ior__ The augmented bitwise assignment |=
__ixor__ The augmented bitwise assignment ^=
__ilshift__ The augmented bitwise assignment <<=
__irshift__ The augmented bitwise assignment >>=

Unary bitwise operators

__invert__ The unary bitwise operation ~

Data special methods

Standard library functions

__deepcopy__ Used if copy.deepcopy is called
__hash__ The built-in function hash

Container customization

__len__ The built-in function len
__getitem__ Evaluation of self[indices]
__iter__ Efficient iteration
__setitem__ Assignment to self[indices]
__contains__ Membership test operators

String representations

__repr__ The built-in function repr
__str__ The built-in function str

Table Of Contents

Previous topic

cf.CoordinateBounds.transpose

Next topic

cf.Data.array

This Page