speasy.products.variable module

class speasy.products.variable.SpeasyVariable(axes: List[VariableAxis], values: DataContainer, columns: List[str] | None = None)[source]

Bases: SpeasyProduct

SpeasyVariable object. Base class for storing variable data.

Attributes:

time: numpy.ndarray: time vector (x-axis data)
values: numpy.ndarray: data
meta: Optional[dict]: metadata
columns: Optional[List[str]]: column names, might be empty for spectrograms or 3D+ data
axes: List[np.ndarray]: Collection composed of time axis plus eventual additional axes according to values’ shape
axes_labels: List[str]: Axes names
unit: str: Values physical unit
name: str: SpeasyVariable name
nbytes: int: memory usage in bytes
fill_value: Any: fill value if found in meta-data
valid_range: Tuple[Any, Any]: valid range if found in meta-data

Methods

view:	Returns a view of the current variable within the desired `index_range`
to_dataframe:	Converts the variable to a pandas.DataFrame object
from_dataframe:	Builds a SpeasyVariable from a pandas.DataFrame object
to_astropy_table:	Converts the variable to an astropy.table.Table
unit_applied:	Returns a copy where values are astropy.units.Quantity
filter_columns:	Returns a copy only containing selected columns
replace_fillval_by_nan:	Returns a SpeasyVaraible with NaN instead of fill value if fill value is set in meta data
plot:	Plot the data with matplotlib by default
to_dictionary:	Converts a SpeasyVariable to a Python dictionary, mostly used for serialization purposes
copy:	Returns a copy

astype(dtype) → SpeasyVariable[source]

Returns a SpeasyVariable with values converted to given dtype

Parameters:

dtypestr or np.dtype or type: desired dtype

Returns:

SpeasyVariable: SpeasyVariable with values converted to given dtype

property axes: List[VariableTimeAxis]

SpeasyVariable axes, axis 0 is always a VariableTimeAxis, there should be the same number of axes than values dimensions

Returns:

List[VariableTimeAxis or VariableAxis]: list of variable axes

property axes_labels: List[str]

Axes names respecting axes order

Returns:

List[str]: list of axes names

clamp_with_nan(inplace=False, valid_min=None, valid_max=None, convert_to_float=False) → SpeasyVariable[source]

Replaces values outside valid range by NaN, valid range is taken from metadata fields “VALIDMIN” and “VALIDMAX”. Automatically converts variable to float if convert_to_float is True and needed.

Parameters:

inplacebool, optional: Modifies source variable when true else modifies and returns a copy, by default False
valid_minFloat, optional: Optional minimum valid value, takes metadata field “VALIDMIN” if not provided, by default None
valid_maxFloat, optional: Optional maximum valid value, takes metadata field “VALIDMAX” if not provided, by default None
convert_to_floatbool, optional: Automatically converts variable to float if true and needed, by default False.

Returns:

SpeasyVariable: source variable or copy with values clamped by NaN

See also

replace_fillval_by_nan: replaces fill values by NaN
sanitized: removes fill and invalid values

property columns: List[str]

SpeasyVariable columns names when it makes sense

Returns:

List[str]: list of columns names

copy(name=None) → SpeasyVariable[source]

Makes a deep copy the variable

Parameters:

name: str, optional: new variable name, by default None, keeps the same name

Returns:

SpeasyVariable: deep copy the variable

property dtype

static empty_like(other: SpeasyVariable) → SpeasyVariable[source]

Create a SpeasyVariable with the same properties than given variable but unset values

Parameters:

otherSpeasyVariable: variable used as reference for shape and meta-data

Returns:

SpeasyVariable: a SpeasyVariable similar to given one

property fill_value: Any | None

SpeasyVariable fill value if found in meta-data

Returns:

Any: fill value if found in meta-data

filter_columns(columns: List[str]) → SpeasyVariable[source]

Builds a SpeasyVariable with only selected columns

Parameters:

columnsList[str]: list of column names to keep

Returns:

SpeasyVariable: a SpeasyVariable with only selected columns

static from_dataframe(df: DataFrame, meta: Dict[str, Any] | None = None, name: str = 'Unknown') → SpeasyVariable[source]

Load from pandas.DataFrame object.

Parameters:

df: pandas.DataFrame: Input DataFrame to convert
meta: Optional[Dict[str, Any]], optional: Optional metadata dictionary, by default None
name: str, optional: Variable name, by default “Unknown”

Returns:

SpeasyVariable:: Variable created from DataFrame

See also

to_dataframe: exports a SpeasyVariable to a pandas DataFrame
to_astropy_table: exports a SpeasyVariable to an astropy.Table object

static from_dictionary(dictionary: Dict[str, object]) → SpeasyVariable[source]

Builds a SpeasyVariable from a well formed dictionary

Returns:

SpeasyVariable or None

See also

to_dictionary: exports SpeasyVariable to dictionary

property meta: Dict

SpeasyVariable meta-data

Returns:

Dict: SpeasyVariable meta-data

property name: str

SpeasyVariable name

Returns:

str: SpeasyVariable name

property nbytes: int

SpeasyVariable’s values and axes memory usage

Returns:

int: number of bytes used to store values and axes

property ndim

static ones_like(other: SpeasyVariable) → SpeasyVariable[source]

Create a SpeasyVariable with the same properties than given variable but filled with ones

Parameters:

otherSpeasyVariable: variable used as reference for shape and meta-data

Returns:

SpeasyVariable: a SpeasyVariable similar to given one filled with ones

property plot: Plot the variable, tries to do its best to detect variable type and to populate plot labels

replace_fillval_by_nan(inplace=False, convert_to_float=False) → SpeasyVariable[source]

Replaces fill values by NaN, non float values are automatically converted to float if convert_to_float is True. Fill value is taken from metadata field “FILLVAL”

Parameters:

inplacebool, optional: Modifies source variable when true else modifies and returns a copy, by default False
convert_to_floatbool, optional: Automatically converts variable to float if true and needed, by default False.

Returns:

SpeasyVariable: source variable or copy with fill values replaced by NaN

See also

clamp_with_nan: replaces values outside valid range by NaN
sanitized: removes fill and invalid values

static reserve_like(other: SpeasyVariable, length: int = 0) → SpeasyVariable[source]

Create a SpeasyVariable of given length and with the same properties than given variable but unset values

Parameters:

otherSpeasyVariable: variable used as reference for shape and meta-data
lengthint, optional: output variable length, by default 0

Returns:

SpeasyVariable: a SpeasyVariable similar to given one of given length

sanitized(drop_fill_values=True, drop_out_of_range_values=True, drop_nan_and_inf=True, valid_min=None, valid_max=None) → SpeasyVariable[source]

Returns a copy of the variable with fill values and invalid values removed

Parameters:

drop_fill_valuesbool, optional: Remove fill values, by default True
drop_out_of_range_valuesbool, optional: Remove values outside valid range, by default True
drop_nan_and_infbool, optional: Remove NaN and Infinite values, by default True
valid_minFloat, optional: Minimum valid value, takes metadata field “VALIDMIN” if not provided, by default None
valid_maxFloat, optional: Maximum valid value, takes metadata field “VALIDMAX” if not provided, by default None

Returns:

SpeasyVariable: source variable or copy with fill and invalid values removed

See also

replace_fillval_by_nan: replaces fill values by NaN
clamp_with_nan: replaces values outside valid range by NaN

property shape

property time: array

Time axis values, equivalent to var.axes[0].values

Returns:

np.array: time axis values as numpy array of datetime64[ns]

to_astropy_table() → Table[source]

Convert the variable to an astropy.Table object.

Parameters:

datetime_index: bool: boolean indicating that the index is datetime

Returns:

astropy.Table:: Variable converted to astropy.Table

See also

from_dataframe: builds a SpeasyVariable from a pandas DataFrame
to_dataframe: exports a SpeasyVariable to a pandas DataFrame

to_dataframe() → DataFrame[source]

Convert the variable to a pandas.DataFrame object.

Returns:

pandas.DataFrame:: Variable converted to Pandas DataFrame

See also

from_dataframe: builds a SpeasyVariable from a pandas DataFrame
to_astropy_table: exports a SpeasyVariable to an astropy.Table object

to_dictionary(array_to_list=False) → Dict[str, object][source]

Converts SpeasyVariable to dictionary

Parameters:

array_to_listbool, optional: Converts numpy arrays to Python Lists when true, by default False

Returns:

Dict[str, object]

See also

from_dictionary: builds variable from dictionary

property unit: str

SpeasyVariable unit if found in meta-data

Returns:

str: unit if found in meta-data

unit_applied(unit: str = None, copy=True) → SpeasyVariable[source]

Returns a SpeasyVariable with given or automatically found unit applied to values

Parameters:

unitstr or None, optional: Use given unit or gets one from variable metadata, by default None
copybool, optional: Preserves source variable and returns a modified copy if true, by default True

Returns:

SpeasyVariable: SpeasyVariable identic to source one with values converted to astropy.units.Quantity according to given or found unit

See also

unit: returns variable unit if found in meta-data

Notes

This interface assume that there is only one unit for the whole variable since all stored in the same array

property valid_range: Tuple[Any, Any] | None

SpeasyVariable valid range if found in meta-data

Returns:

Tuple[Any, Any]: valid range if found in meta-data

property values: array

SpeasyVariable values

Returns:

np.array: SpeasyVariable values

view(index_range: slice | ndarray) → SpeasyVariable[source]

Return view of the current variable within the desired index_range.

Parameters:

index_range: slice: index range

Returns:

speasy.common.variable.SpeasyVariable: view of the variable on the given range

static zeros_like(other: SpeasyVariable) → SpeasyVariable[source]

Create a SpeasyVariable with the same properties than given variable but filled with zeros

Parameters:

otherSpeasyVariable: variable used as reference for shape and meta-data

Returns:

SpeasyVariable: a SpeasyVariable similar to given one filled with zeros

speasy.products.variable.from_dataframe(df: DataFrame) → SpeasyVariable[source]

Convert a dataframe to SpeasyVariable.

See also

SpeasyVariable.from_dataframe

speasy.products.variable.from_dictionary(dictionary: Dict[str, object]) → SpeasyVariable[source]

speasy.products.variable.merge(variables: Iterable[SpeasyVariable]) → SpeasyVariable | None[source]

Merge a list of SpeasyVariable objects.

Parameters:

variables: Iterable[SpeasyVariable]: Variables to merge together. Each one is released as soon as it is copied into the result, so passing an iterator that hands over the only reference keeps the memory peak near the result size.

Returns:

SpeasyVariable:: Resulting variable from merge operation

speasy.products.variable.same_time_axis(variables: List[SpeasyVariable]) → bool[source]

Check if all variables have the same time axis values and length If only one variable is provided, it returns True.

Parameters:

variablesList[SpeasyVariable]: list of variables to check

Returns:

bool: True if all variables have the same time axis values and length

speasy.products.variable.to_dataframe(var: SpeasyVariable) → DataFrame[source]

Convert a SpeasyVariable to pandas.DataFrame.

See also

SpeasyVariable.to_dataframe

speasy.products.variable.to_dictionary(var: SpeasyVariable, array_to_list=False) → Dict[str, object][source]