Cache Setting

cache_setting parameter to the RetrievalSettings class. Some subclasses and mappings covering common use cases for retrieving data are provided.

Usage

Currently, these are the options that can be passed to the cache_setting parameter in RetrievalSettings:

• An instance of a subclass of CacheSetting
• A string identifier in the _cache_setting_mappings dictionary:

  _cache_setting_mappings: Dict[str, CacheSetting] = {
      "sql": SQLCacheSetting(),
  }
  

• A dictionary mapping tokamak type strings to the desired CacheSetting for that tokamak. E.g. {'cmod': 'sql'}. Currently supported tokamak type strings are: "cmod", "d3d"

Built-in Implementations

This module provides classes for managing and retrieving cached data from various sources, including SQL databases and Pandas DataFrames.

disruption_py.settings.cache_setting.CSVCacheSetting

Bases: CacheSetting

Cache setting for retrieving data from a CSV file.

PARAMETER	DESCRIPTION
cache_file	The path to the CSV file to use as the cached data. TYPE: str

Source code in disruption_py/settings/cache_setting.py

class CSVCacheSetting(CacheSetting):
    """
    Cache setting for retrieving data from a CSV file.

    Parameters
    ----------
    cache_file : str
        The path to the CSV file to use as the cached data.
    """

    def __init__(self, cache_file: str):
        self.cache_file = cache_file

    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        return pd.read_csv(self.cache_file)

disruption_py.settings.cache_setting.CacheSettingDict

Bases: CacheSetting

A class that resolves and manages cache settings for Tokamak instances.

This class takes a dictionary of cache settings and resolves them for easy access.

ATTRIBUTE	DESCRIPTION
resolved_cache_setting_dict	A mapping of Tokamak instances to their resolved cache settings. TYPE: Dict[Tokamak, CacheSettingType]

PARAMETER	DESCRIPTION
cache_setting_dict	A dictionary of initial cache settings for each Tokamak. TYPE: Dict[Tokamak, CacheSettingType]

METHOD	DESCRIPTION
_get_cache_data	Retrieves cache data for the specified Tokamak.

Source code in disruption_py/settings/cache_setting.py

class CacheSettingDict(CacheSetting):
    """
    A class that resolves and manages cache settings for Tokamak instances.

    This class takes a dictionary of cache settings and resolves them for easy access.

    Attributes
    ----------
    resolved_cache_setting_dict : Dict[Tokamak, CacheSettingType]
        A mapping of Tokamak instances to their resolved cache settings.

    Parameters
    ----------
    cache_setting_dict : Dict[Tokamak, CacheSettingType]
        A dictionary of initial cache settings for each Tokamak.

    Methods
    -------
    _get_cache_data(params: CacheSettingParams) -> pd.DataFrame
        Retrieves cache data for the specified Tokamak.
    """

    def __init__(self, cache_setting_dict: Dict[Tokamak, CacheSettingType]):
        resolved_cache_setting_dict = {
            map_string_to_enum(tokamak, Tokamak): resolve_cache_setting(
                individual_setting
            )
            for tokamak, individual_setting in cache_setting_dict.items()
        }
        self.resolved_cache_setting_dict = resolved_cache_setting_dict

    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        chosen_setting = self.resolved_cache_setting_dict.get(params.tokamak, None)
        if chosen_setting is not None:
            return chosen_setting.get_cache_data(params)
        params.logger.warning(
            "No cache setting for tokamak {tokamak}", tokamak=params.tokamak
        )
        return None

disruption_py.settings.cache_setting.DFCacheSetting

Bases: CacheSetting

Cache setting for retrieving data from a Pandas DataFrame.

PARAMETER	DESCRIPTION
cache_data	The DataFrame to use as the cached data. TYPE: DataFrame

Source code in disruption_py/settings/cache_setting.py

class DFCacheSetting(CacheSetting):
    """
    Cache setting for retrieving data from a Pandas DataFrame.

    Parameters
    ----------
    cache_data : pd.DataFrame
        The DataFrame to use as the cached data.
    """

    def __init__(self, cache_data: pd.DataFrame):
        self.cache_data = cache_data

    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        return self.cache_data

disruption_py.settings.cache_setting.HDF5CacheSetting

Bases: CacheSetting

Cache setting for retrieving data from an HDF5 file.

PARAMETER	DESCRIPTION
cache_file	The path to the HDF5 file to use as the cached data. TYPE: str

Source code in disruption_py/settings/cache_setting.py

class HDF5CacheSetting(CacheSetting):
    """
    Cache setting for retrieving data from an HDF5 file.

    Parameters
    ----------
    cache_file : str
        The path to the HDF5 file to use as the cached data.
    """

    def __init__(self, cache_file: str):
        self.cache_file = cache_file

    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        # Some shots may not have been cached, so the key will not exist, and
        # it throws an error. If there is no cached data, return None.
        try:
            return pd.read_hdf(self.cache_file, key=f"df_{params.shot_id}")
        except KeyError:
            return None

disruption_py.settings.cache_setting.SQLCacheSetting

Bases: CacheSetting

Cache setting for retrieving data from SQL database.

Source code in disruption_py/settings/cache_setting.py

class SQLCacheSetting(CacheSetting):
    """Cache setting for retrieving data from SQL database."""

    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        params.logger.info("retrieving sql data")
        return params.database.get_shots_data(shotlist=[params.shot_id])

disruption_py.settings.cache_setting.resolve_cache_setting

resolve_cache_setting(
    cache_setting: CacheSettingType,
) -> CacheSetting

Resolve the cache setting to a CacheSetting instance.

PARAMETER	DESCRIPTION
cache_setting	The cache setting to resolve. This can be an instance of CacheSetting, a string representing a cache setting type, a Pandas DataFrame, or a dictionary of cache settings. TYPE: CacheSettingType

RETURNS	DESCRIPTION
CacheSetting	An instance of CacheSetting corresponding to the provided cache setting.

Source code in disruption_py/settings/cache_setting.py

def resolve_cache_setting(
    cache_setting: CacheSettingType,
) -> CacheSetting:
    """
    Resolve the cache setting to a CacheSetting instance.

    Parameters
    ----------
    cache_setting : CacheSettingType
        The cache setting to resolve. This can be an instance of CacheSetting,
        a string representing a cache setting type, a Pandas DataFrame, or a
        dictionary of cache settings.

    Returns
    -------
    CacheSetting
        An instance of CacheSetting corresponding to the provided cache setting.
    """
    if cache_setting is None:
        return None

    if isinstance(cache_setting, CacheSetting):
        return cache_setting

    if isinstance(cache_setting, str):
        cache_setting_object = _cache_setting_mappings.get(cache_setting, None)
        if cache_setting_object is not None:
            return cache_setting_object

        # assume that it is a file path
        for (
            suffix,
            cache_setting_type,
        ) in _file_suffix_to_cache_setting.items():
            if cache_setting.endswith(suffix):
                return cache_setting_type(cache_setting)

    if isinstance(cache_setting, pd.DataFrame):
        return DFCacheSetting(cache_setting)

    if isinstance(cache_setting, dict):
        return CacheSettingDict(cache_setting)

    raise ValueError("Invalid cache setting")

Custom Implementations

Custom implementations of cache setting must inherit from the CacheSetting abstract class, implementing the abstract methods.

This module provides classes for managing and retrieving cached data from various sources, including SQL databases and Pandas DataFrames.

disruption_py.settings.cache_setting.CacheSetting

Bases: ABC

CacheSetting abstract class that should be inherited by all cache setting classes.

Subclasses must implement the _get_cache_data method to define how cached data is retrieved.

Source code in disruption_py/settings/cache_setting.py

class CacheSetting(ABC):
    """
    CacheSetting abstract class that should be inherited by all cache setting classes.

    Subclasses must implement the `_get_cache_data` method to define how cached data is retrieved.
    """

    def get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        """
        Return cached data, using tokamak-specific overrides if available.
        """
        if hasattr(self, "tokamak_overrides"):
            if params.tokamak in self.tokamak_overrides:
                return self.tokamak_overrides[params.tokamak](params)
        return self._get_cache_data(params)

    @abstractmethod
    def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
        """
        Abstract method implemented by subclasses to get cached data for a
        given set of params as a Pandas dataframe.

        Parameters
        ----------
        params : CacheSettingParams
            Params that can be used to determine and retrieve cached data.

        Returns
        -------
        pd.DataFrame
            Pandas dataframe containing cached data.
        """

_get_cache_data abstractmethod

_get_cache_data(params: CacheSettingParams) -> pd.DataFrame

Abstract method implemented by subclasses to get cached data for a given set of params as a Pandas dataframe.

PARAMETER	DESCRIPTION
params	Params that can be used to determine and retrieve cached data. TYPE: CacheSettingParams

RETURNS	DESCRIPTION
DataFrame	Pandas dataframe containing cached data.

Source code in disruption_py/settings/cache_setting.py

@abstractmethod
def _get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
    """
    Abstract method implemented by subclasses to get cached data for a
    given set of params as a Pandas dataframe.

    Parameters
    ----------
    params : CacheSettingParams
        Params that can be used to determine and retrieve cached data.

    Returns
    -------
    pd.DataFrame
        Pandas dataframe containing cached data.
    """

get_cache_data

get_cache_data(params: CacheSettingParams) -> pd.DataFrame

Return cached data, using tokamak-specific overrides if available.

Source code in disruption_py/settings/cache_setting.py

def get_cache_data(self, params: CacheSettingParams) -> pd.DataFrame:
    """
    Return cached data, using tokamak-specific overrides if available.
    """
    if hasattr(self, "tokamak_overrides"):
        if params.tokamak in self.tokamak_overrides:
            return self.tokamak_overrides[params.tokamak](params)
    return self._get_cache_data(params)

disruption_py.settings.cache_setting.CacheSettingParams dataclass

Params passed by disruption_py to _get_cache_data() method.

ATTRIBUTE	DESCRIPTION
shot_id	Shot Id for which to get cached data. Defaults to logbook. TYPE: int
database	Database object to use for getting cached data. A different database connection is used by each thread/process. TYPE: ShotDatabase
tokamak	The tokamak being run. TYPE: Tokamak

Source code in disruption_py/settings/cache_setting.py

@dataclass
class CacheSettingParams:
    """
    Params passed by disruption_py to _get_cache_data() method.

    Attributes
    ----------
    shot_id : int
        Shot Id for which to get cached data. Defaults to logbook.
    database : ShotDatabase
        Database object to use for getting cached data.
        A different database connection is used by each thread/process.
    tokamak : Tokamak
        The tokamak being run.
    """

    shot_id: int
    database: ShotDatabase
    tokamak: Tokamak

    def __post_init__(self):
        self.logger = logger.patch(
            lambda record: record.update(
                message=shot_log_msg(self.shot_id, record["message"])
            )
        )