Configuration

The configuration file is a .yaml file that makes up the majority of the settings. It is specified by the user, and if a field is not set, it falls back to a set of pre-defined defaults. It is split into sections to easily organize different parameters.

Specifying the Configuration

  • Pipeline Mode

To run diagnostics via the command line, pass the path to a yaml file to the diagnostic command. This will run the entire set of diagnostics according to the configuration file.

diagnose --config path/to/your/config.yaml

  • Standalone Mode

The configuration file is not strictly required for running in standalone mode, but it can be specified to quickly access variables to avoid re-writing initialization parameters or ensure repeatability.

from deepdiagnostics.utils.configuration import Config


Config("path/to/your/config.yaml")

Configuration Description

common
Parameters:

  • out_dir – Folder where the results of program are saved. The path need not exist, it will be created if it does not.

  • temp_config – Path to a yaml to store a temporary config. Used only if some arguments are specified outside the config (eg, if using both the –config and –model_path arguments)

  • sim_location – Path to store settings for simulations. When using the register_simulator method, this is where the registered simulations are catalogued.

  • random_seed – Integer random seed to use.

common:
    out_dir: "./DeepDiagnosticsResources/results/"
    temp_config: "./DeepDiagnosticsResources/temp/temp_config.yml"
    sim_location: "DeepDiagnosticsResources/simulators"
    random_seed: 42
model
Parameters:

  • model_path – Path to stored model. Required.

  • model_engine – Loading method to use. Choose from methods listed in Models.

model:
    model_path: {No Default}
    model_engine: "SBIModel"
data
Parameters:

  • data_path – Path to stored data. Required.

  • data_engine – Loading method to use. Choose from methods listed in Data.

  • simulator – String name of the simulator. Must be pre-registered .

  • prior – Prior distribution used in training. Used if “prior” is not included in the passed data.

  • prior_kwargs – kwargs to use with the initialization of the prior

  • simulator_kwargs – kwargs to use with the initialization of the simulation

  • simulator_dimensions – If the output of the simulation is 1D (non-image) or 2D (images.)

data:
    data_path: {No Default}
    data_engine: "H5Data"
    prior: "normal"
    prior_kwargs: {No Default}
    simulator_kwargs: {No Default}
    simulator_dimensions: 1
plots_common
Parameters:

  • axis_spines – Show axis ticks

  • tight_layout – Minimize the space between axes and labels

  • default_colorway – String colorway to use. Choose from matplotlib’s named colorways.

  • plot_style – Style sheet. Choose form matplotlib’s style sheets.

  • parameter_labels – Name of each theta parameter to use for titling and labels. Corresponding with the dim=1 axis of theta given by data.

  • parameter_colors – Colors to use for each theta parameters when representing the parameters on the same plot.

  • line_style_cycle – Line styles that can be used (besides for solid lines, which are always used.)

  • figure_size – Default size for square figures. Will be adapted (slightly expanded) for multi-plot figures.

plots_common:
    axis_spines: False
    tight_layout: True
    default_colorway: "viridis"
    plot_style: "fast"
    parameter_labels: ["$m$", "$b$"]
    parameter_colors: ["#9C92A3", "#0F5257"]
    line_style_cycle: ["-", "-."]
    figure_size: [6, 6]
metrics_common

These parameters are used for every metric calculated, and for plots that require new inference to be run.

Parameters:

  • use_progress_bar – Show a progress bar when iteratively performing inference.

  • samples_per_inference – Number of samples used in a single iteration of inference.

  • percentiles – List of integer percentiles, for defining coverage regions.

  • number_simulations – Number of different simulations to run. Often, this means that the number of inferences performed for a metric is samples_per_inference*number_simulations

metrics_common:
    use_progress_bar: False
    samples_per_inference: 1000
    percentiles: [75, 85, 95]
    number_simulations: 50
plots

A dictionary of different plots to generate and their arguments. Can be any of the implemented plots listed in plots If the plots are specified with an empty dictionary, defaults from the class are used. Defaults: [“CDFRanks”, “Ranks”, “CoverageFraction”, “TARP”, “LC2ST”, “PPC”]

plots:
    TARP: {}
metrics

A dictionary of different metrics to generate and their arguments. Can be any of the implemented plots listed in metrics If the metrics are specified with an empty dictionary, defaults from the class are used. Defaults: [“AllSBC”, “CoverageFraction”, “LC2ST”]

metrics:
    LC2ST: {}