cf.Data¶

class
cf.
Data
(data=None, units=None, fill_value=None, hardmask=True, chunk=True, loadd=None, loads=None, dt=False)[source]¶ Bases:
object
An Ndimensional data array with units and masked values.
 Contains an Ndimensional, indexable and broadcastable array with
many similarities to a
numpy
array.  Contains the units of the array elements.
 Supports masked arrays, regardless of whether or not it was initialised with a masked array.
 Stores and operates on data arrays which are larger then the available memory.
Indexing
A data array is indexable in a similar way to numpy array:
>>> d.shape (12, 19, 73, 96) >>> d[...].shape (12, 19, 73, 96) >>> d[slice(0, 9), 10:0:2, :, :].shape (9, 5, 73, 96)
There are three extensions to the numpy indexing functionality:
Size 1 dimensions are never removed bi indexing.
An integer index i takes the ith element but does not reduce the rank of the output array by one:
>>> d.shape (12, 19, 73, 96) >>> d[0, ...].shape (1, 19, 73, 96) >>> d[:, 3, slice(10, 0, 2), 95].shape (12, 1, 5, 1)
Size 1 dimensions may be removed with the
squeeze
method.The indices for each axis work independently.
When more than one dimension’s slice is a 1d boolean sequence or 1d sequence of integers, then these indices work independently along each dimension (similar to the way vector subscripts work in Fortran), rather than by their elements:
>>> d.shape (12, 19, 73, 96) >>> d[0, :, [0, 1], [0, 13, 27]].shape (1, 19, 2, 3)
Boolean indices may be any object which exposes the numpy array interface.
>>> d.shape (12, 19, 73, 96) >>> d[..., d[0, 0, 0]>d[0, 0, 0].min()]
Cyclic axes
Miscellaneous
A
Data
object is picklable.A
Data
object is hashable, but note that, since it is mutable, its hash value is only valid whilst the data array is not changed in place.Initialization
Parameters:  data: arraylike, optional
The data for the array.
 units : str or Units, optional
The units of the data. By default the array elements are dimensionless.
 fill_value: optional
The fill value of the data. By default, or if None, the numpy fill value appropriate to the array’s data type will be used.
 hardmask:
bool
, optional If False then the mask is soft. By default the mask is hard.
 chunk:
bool
, optional If False then the data array will be stored in a single partition. By default the data array will be partitioned if it is larger than the chunk size, as returned by the
cf.CHUNKSIZE
function. dt:
bool
, optional If True then strings (such as
'1990121 12:00'
) given by the data argument are interpreted as datetimes. By default they are not interpreted as datetimes. loadd:
dict
, optional Initialise the data array from a dictionary serialization of a
cf.Data
object. All other arguments are ignored. loads:
str
, optional Initialise the data array from a JSON string serialization of a
Data
object. All other arguments are ignored.
Examples: >>> d = cf.Data(5) >>> d = cf.Data([1,2,3], units='K') >>> import numpy >>> d = cf.Data(numpy.arange(10).reshape(2,5), units=cf.Units('m/s'), fill_value=999) >>> d = cf.Data(tuple('fly'))
 Contains an Ndimensional, indexable and broadcastable array with
many similarities to a
Data attributes¶
array 
A numpy array copy the data array. 
binary_mask 
A binary (0 and 1) mask of the data array. 
data 
The data array object as an object identity. 
day 
The day of each data array element. 
dtarray 
An independent numpy array of datetime objects. 
dtype 
The numpy data type of the data array. 
fill_value 
The data array missing data value. 
hardmask 
Whether the mask is hard (True) or soft (False). 
hour 
The hour of each data array element. 
ismasked 
True if the data array has any masked values. 
isscalar 
True if the data array is a 0d scalar array. 
mask 
The boolean missing data mask of the data array. 
minute 
The minute of each data array element. 
month 
The month of each data array element. 
nbytes 
Total number of bytes consumed by the elements of the array. 
ndim 
Number of dimensions in the data array. 
second 
The second of each data array element. 
shape 
Tuple of the data array’s dimension sizes. 
size 
Number of elements in the data array. 
Units 
The cf.Units object containing the units of the data array. 
varray 
A numpy array view the data array. 
year 
The year of each data array element. 
Data methods¶
all 
Test whether all data array elements evaluate to True.  
allclose 
Returns True if two broadcastable arrays have equal values, False otherwise.  
max 
Collapse axes with their maximum.  
min 
Collapse axes with their minimum.  
any 
Test whether any data array elements evaluate to True.  
chunk 
Partition the data array.  
ceil 
Return the ceiling of the data array.  
clip 
Clip (limit) the values in the data array in place.  
close 
Close all files referenced by the data array.  
copy 
Return a deep copy.  
cos 
Take the trigonometric cosine of the data array in place.  
datum 
Return an element of the data array as a standard Python scalar.  
dump 
Return a string containing a full description of the instance.  
dumpd 
Return a serialization of the data array.  
equals 
True if two data arrays are logically equal, False otherwise.  
cf.Data.equivalent 

expand_dims 
Expand the shape of the data array in place.  
files 
Return the names of files containing parts of the data array.  
flat 
Return a flat iterator over elements of the data array.  
flip 
Reverse the direction of axes of the data array.  
floor 
Return the floor of the data array.  
func 
Apply an elementwise array operation to the data array.  
HDF_chunks 

isclose 
Return a boolean data array showing where two broadcastable arrays have equal values within a tolerance.  
loadd 
Reset the data array in place from a data array serialization.  
mask_invalid 
Mask the array where invalid values occur (NaN or inf).  
mean 
Collapse axes with their weighted mean.  
mid_range 
Collapse axes with the unweighted average of their maximum and minimum values.  
ndindex 
Return an iterator over the Ndimensional indices of the data array.  
outerproduct 
Compute the outer product with another data array.  
override_calendar 
Override the calendar of the data array elements.  
override_units 
Override the data array units.  
partition_boundaries 
Return the partition boundaries for each partition matrix dimension.  
range 
Collapse axes with the absolute difference between their maximum and minimum values.  
rint 
Round elements of the data array to the nearest integer.  
roll 
A lot like numpy.roll 

save_to_disk 

sample_size 


sd 
Collapse axes by calculating their standard deviation.  
sin 
Take the trigonometric sine of the data array in place.  
squeeze 
Remove size 1 axes from the data array.  
sum 
Collapse axes with their sum.  
sum_of_weights 
Missing data array elements are omitted from the calculation.  
sum_of_weights2 
Missing data array elements are omitted from the calculation.  
swapaxes 
Interchange two axes of an array.  
tan 
Take the trigonometric tangent of the data array elementwise.  
to_disk 
Store the data array on disk.  
to_memory 
Store each partition’s data in memory in place if the master array is smaller than the chunk size.  
transpose 
Permute the axes of the data array.  
trunc 
Return the truncated values of the data array.  
unique 
The unique elements of the array.  
var 
Collapse axes with their weighted variance.  
where 
Set data array elements depending on a condition. 
Data static methods¶
mask_fpe 
Masking of floatingpoint errors in the results of arithmetic operations. 
seterr 
Set how floatingpoint errors in the results of arithmetic operations are handled. 
Data arithmetic and comparison operations¶
Arithmetic, bitwise and comparison operations are defined as
elementwise data array operations which yield a new cf.Data
object
or, for augmented assignments, modify the data array inplace.
Comparison operators
__lt__ 
The rich comparison operator < 
__le__ 
The rich comparison operator <= 
__eq__ 
The rich comparison operator == 
__ne__ 
The rich comparison operator != 
__gt__ 
The rich comparison operator > 
__ge__ 
The rich comparison operator >= 
Truth value of an array
__nonzero__ 
Truth value testing and the builtin operation bool 
Binary arithmetic operators
__add__ 
The binary arithmetic operation + 
__sub__ 
The binary arithmetic operation  
__mul__ 
The binary arithmetic operation * 
__div__ 
The binary arithmetic operation / 
__truediv__ 
The binary arithmetic operation / (true division) 
__floordiv__ 
The binary arithmetic operation // 
__pow__ 
The binary arithmetic operations ** and pow 
__mod__ 
The binary arithmetic operation % 
Binary arithmetic operators with reflected (swapped) operands
__radd__ 
The binary arithmetic operation + with reflected operands 
__rsub__ 
The binary arithmetic operation  with reflected operands 
__rmul__ 
The binary arithmetic operation * with reflected operands 
__rdiv__ 
The binary arithmetic operation / with reflected operands 
__rtruediv__ 
The binary arithmetic operation / (true division) with reflected operands 
__rfloordiv__ 
The binary arithmetic operation // with reflected operands 
__rpow__ 
The binary arithmetic operations ** and pow with reflected operands 
__rmod__ 
The binary arithmetic operation % with reflected operands 
Augmented arithmetic assignments
__iadd__ 
The augmented arithmetic assignment += 
__isub__ 
The augmented arithmetic assignment = 
__imul__ 
The augmented arithmetic assignment *= 
__idiv__ 
The augmented arithmetic assignment /= 
__itruediv__ 
The augmented arithmetic assignment /= (true division) 
__ifloordiv__ 
The augmented arithmetic assignment //= 
__ipow__ 
The augmented arithmetic assignment **= 
__imod__ 
The binary arithmetic operation %= 
Unary arithmetic operators
__neg__ 
The unary arithmetic operation  
__pos__ 
The unary arithmetic operation + 
__abs__ 
The unary arithmetic operation abs 
Binary bitwise operators
__and__ 
The binary bitwise operation & 
__or__ 
The binary bitwise operation  
__xor__ 
The binary bitwise operation ^ 
__lshift__ 
The binary bitwise operation << 
__rshift__ 
The binary bitwise operation >> 
Binary bitwise operators with reflected (swapped) operands
__rand__ 
The binary bitwise operation & with reflected operands 
__ror__ 
The binary bitwise operation  with reflected operands 
__rxor__ 
The binary bitwise operation ^ with reflected operands 
__rlshift__ 
The binary bitwise operation << with reflected operands 
__rrshift__ 
The binary bitwise operation >> with reflected operands 
Augmented bitwise assignments
__iand__ 
The augmented bitwise assignment &= 
__ior__ 
The augmented bitwise assignment = 
__ixor__ 
The augmented bitwise assignment ^= 
__ilshift__ 
The augmented bitwise assignment <<= 
__irshift__ 
The augmented bitwise assignment >>= 
Unary bitwise operators
__invert__ 
The unary bitwise operation ~ 
Data special methods¶
Standard library functions
__deepcopy__ 
Used if copy.deepcopy is called. 
__hash__ 
The builtin function hash 
Container customization
__len__ 
The builtin function len 
__getitem__ 
Implement indexing 
__iter__ 
Efficient iteration. 
__setitem__ 
Implement indexed assignment 
__contains__ 
Membership test operator in 
String representations
__repr__ 
The builtin function repr 
__str__ 
The builtin function str 