HAPI servers

HAPI (Heliophysics Application Programmer’s Interface) is a standard web API for time series data, implemented by many data centers. Speasy gives access to every server of the official HAPI server list through a single hapi provider: each server is a branch of spz.inventories.tree.hapi, and products are named hapi/<server>/<dataset id>/<parameter name>.

Basics: browsing and getting data

Browse spz.inventories.tree.hapi with your python terminal completion, then pass the product you found to speasy.get_data():

import speasy as spz
# the Dst index from the World Data Center for Geomagnetism, Kyoto
dst = spz.get_data(spz.inventories.tree.hapi.wdc_kyoto_hapi.hour_dst.dstValue, "2024-05-10", "2024-05-13")
# the same product, by its path
dst = spz.get_data("hapi/wdc_kyoto_hapi/hour_dst/dstValue", "2024-05-10", "2024-05-13")

Paths use the raw HAPI dataset ids and parameter names, which may contain / or spaces, for example "hapi/intermagnet/aae/definitive/PT1M/native/Field_Vector". In the tree, each / of a dataset id is a level, and names are made valid Python identifiers.

Each dataset only covers a finite time range, parameter_range() and dataset_range() tell you what it is:

spz.hapi.dataset_range("wdc_kyoto_hapi/hour_dst")
spz.hapi.parameter_range("wdc_kyoto_hapi/hour_dst/dstValue")

All the HAPI metadata of a dataset or parameter (cadence, units, description, resourceURL, maxRequestDuration, …) is available as attributes of its inventory node.

What gets downloaded, and when

Most HAPI servers list thousands of datasets, so nothing is downloaded until you need it:

  • starting the hapi provider contacts no server;

  • the first time you open a server (spz.inventories.tree.hapi.lisird, its completion, or a get_data on one of its products), Speasy downloads its catalog;

  • the first time you open a dataset, Speasy downloads its parameter list (/info).

An unreachable server only makes its own branch empty, the next access tries again.

Note

Code that walks the inventory tree through __dict__ sees servers and datasets that haven’t been opened yet as empty. dir(), attribute access, iteration and in load them.

Data requests longer than the dataset’s maxRequestDuration (one week when the server gives none) are split, and results are cached on disk like the other providers.

Choosing servers

CDAWeb, AMDA, CSA and SSCWeb also run HAPI servers, but Speasy serves them natively. Speasy never sends them HAPI requests, whatever the configuration: it would only add load to them, and binary requests crash AMDA’s. Use their native providers instead (spz.cda, spz.amda, spz.csa, spz.ssc).

Other servers are disabled or added in the [HAPI] section of the configuration file, see Configuration:

[HAPI]
# don't list KNMI
disabled_servers = knmi
# a server missing from the official list, available as spz.inventories.tree.hapi.mylab
extra_servers = {"mylab": "https://mylab.example.org/hapi"}

To turn HAPI off entirely, add hapi to disabled_providers (see Disabling data providers).

The server list comes from a snapshot of the official list shipped with Speasy and refreshed with each release.