Virtual products¶
A virtual product is not data downloaded from a web service, it is a local function dynamically computing a product.
You register your function with an inventory path, and then use it as any other
Speasy product: request it with speasy.get_data(), or call it by its name.
The registration can be done either directly, or with the decorator
speasy.virtual_products.register_virtual_product().
For arithmetic on existing products, such as 10 * B / N, you don’t need to
write a function: see Virtual expressions.
Writing the function¶
The user function to register takes a time range as argument (start_time,
stop_time) and returns a SpeasyVariable, or None when there is no
data.
It receives the time range exactly as it was given to speasy.get_data():
>>> import pandas as pd
>>> import speasy as spz
>>> def hourly_ramp(start_time, stop_time):
... index = pd.date_range(start_time, stop_time, freq=pd.Timedelta(hours=1), inclusive="left")
... return spz.SpeasyVariable.from_dataframe(pd.DataFrame({"ramp": range(len(index))}, index=index))
Registering with a decorator¶
The registering path must start with virtual/, followed by the inventory
path of your choice (e.g. virtual/demo/ramp).
After the decorator, the function name refers to the product, and the function is registered in the inventory tree:
>>> from speasy.virtual_products import register_virtual_product
>>> @register_virtual_product("virtual/demo/ramp")
... def ramp(start_time, stop_time):
... return hourly_ramp(start_time, stop_time)
>>> spz.inventories.tree.virtual.demo.ramp is ramp
True
Registering directly without a decorator¶
Pass the function as second argument; the product is returned:
>>> slope = register_virtual_product("virtual/demo/slope", hourly_ramp)
>>> spz.inventories.tree.virtual.demo.slope is slope
True
Describing the product¶
Both forms accept meta= and labels=.
meta= — a dictionary that does two things. Its keys become attributes
of the product in the inventory tree, and they are also injected into the
SpeasyVariable.meta returned by the callback, with keys uppercased to
follow the ISTP convention (e.g. "units" becomes "UNITS"). Values
from the inventory always take precedence over any conflicting metadata
returned by the callback:
>>> level = register_virtual_product("virtual/demo/level", hourly_ramp, meta={"units": "nT"})
>>> spz.inventories.tree.virtual.demo.level.units
'nT'
>>> spz.get_data("virtual/demo/level", "2016-10-10", "2016-10-11").unit
'nT'
labels= — a list of strings that overrides the column names of the
SpeasyVariable returned by the callback:
>>> labeled = register_virtual_product("virtual/demo/labeled", hourly_ramp, labels=["value"])
>>> spz.get_data("virtual/demo/labeled", "2016-10-10", "2016-10-11").columns
['value']
Using a virtual product¶
By path, by the decorated name, through the inventory tree, or by a direct call:
>>> spz.get_data("virtual/demo/ramp", "2016-10-10", "2016-10-11").shape
(24, 1)
>>> spz.get_data(ramp, "2016-10-10", "2016-10-11").shape
(24, 1)
>>> spz.get_data(spz.inventories.tree.virtual.demo.ramp, "2016-10-10", "2016-10-11").shape
(24, 1)
>>> ramp("2016-10-10", "2016-10-11").shape
(24, 1)
Distributing virtual products as a plugin¶
A package can ship virtual products that are registered as soon as Speasy is imported.
Declare a zero-argument function as a speasy.virtual_products entry point in the
package’s pyproject.toml:
[project.entry-points."speasy.virtual_products"]
my_products = "my_pkg:register"
The function only has to import the module holding the decorated functions:
# my_pkg/__init__.py
def register():
from . import products # noqa: F401
# my_pkg/products.py
from speasy.virtual_products import register_virtual_product
@register_virtual_product("virtual/my_pkg/beta")
def beta(start_time, stop_time):
...
Once the package is installed, spz.inventories.tree.virtual.my_pkg.beta exists after
import speasy.
Plugins are loaded while
import speasyis running: importregister_virtual_productfromspeasy.virtual_products, not from the top-levelspeasypackage. The function bodies only run when the product is requested, so they can use all of Speasy.A plugin that fails to load is logged as a warning and skipped;
import speasystill completes.A plugin can be turned off with the
core.disabled_pluginsconfig entry, by its name (my_products) or by its group-qualified name (speasy.virtual_products.my_products).
Loading virtual products from a directory¶
Without packaging anything, every .py file in the user virtual products directory
(~/.local/share/speasy/virtual_products/ on Linux; speasy.virtual_products.registry.user_vp_dir()
gives it on your system) is executed by import speasy:
# ~/.local/share/speasy/virtual_products/my_products.py
from speasy.virtual_products import register_virtual_product
@register_virtual_product("virtual/mine/beta")
def beta(start_time, stop_time):
...
More directories can be scanned with the core.user_vp_extra_dirs config entry, a
comma-separated list (see Configuration).
Files are loaded after the plugins: a file registering the same path as an installed plugin replaces it, with a warning.
A file that fails to load is logged as a warning, with a traceback pointing to the file, and skipped; the other files and
import speasyare unaffected.
Good to know¶
Registering again at the same path replaces the previous product and emits a warning, which is convenient when iterating in a notebook.
Requesting a path that was never registered raises
UnknownVirtualProduct, aValueError. Its message suggests the closest registered paths, if any.If the function returns anything other than a
SpeasyVariableorNone, calling the product orspeasy.get_data()raisesVirtualProductTypeError, aTypeErrornaming the function.A virtual product may request other virtual products, but never itself, not even for another time range and not even through other products. Such a request raises
VirtualProductCycleError, aRecursionError, whose message shows the chain of products. To reuse a product’s computation, call its function directly.This check follows the requests made in one thread. If your function starts its own threads and requests virtual products from them, a product requesting itself through them is not detected.