explainable_rl.performance package
Submodules
explainable_rl.performance.performance_evaluator module
- class PerformanceEvaluator(results_path='src/performance/evaluations', num_sample_range=[100, 1000, 10000, 100000], num_ep_range=[10, 100, 1000, 10000], num_bin_range=[2, 5, 10, 20, 30, 40, 50], verbose=False)[source]
Bases:
objectTime and Memory Evaluator.
- __init__(results_path='src/performance/evaluations', num_sample_range=[100, 1000, 10000, 100000], num_ep_range=[10, 100, 1000, 10000], num_bin_range=[2, 5, 10, 20, 30, 40, 50], verbose=False)[source]
Initialise a PerformanceEvaluator.
- Parameters
results_path (str) – Path to store performance results.
num_sample_range (list[int]) – Range of dataset samples to evaluate performance over.
num_ep_range (list[int]) – Range of episodes to evaluate performance over.
num_bin_range (list[int]) – Range of bins to evaluate performance over.
verbose (bool) – Whether to print statements of flow progression.
- _get_hyperparam_dict_ds_data(num_episodes, num_bins, num_samples)[source]
Load the hyperparameter dictionaries for the Datasparq data.
- Parameters
num_episodes (int) – Number of episodes in the training loop.
num_bins (int) – Number of bins used for digitisation (equal number of bins for all dimensions).
num_samples (int) – Number of datapoint samples used for training.
- Returns
Hyperparameter dictionary specific to the Datasparq dataset.
- Return type
dict
- _get_times_and_memory_from_parameter_range(parameter_name, x)[source]
Get time and memory complexities for varying values of an inputted parameter name.
- Parameters
parameter_name (str) – Name of the parameter against which time and memory complexities are recorded.
x (list[int]) – Range of values to vary the parameter.
- Returns
Time and memory recordings for the different parameter values in x.
- Return type
list[float], list[float]
- _load_data(path, delimiter=',')[source]
Load data from file.
- Parameters
delimiter (str) – Which separates columns.
- _plot_performance_graph(x_label, x, times, memory)[source]
Plot and save a performance graph.
- Parameters
x_label (str) – Label of the horizontal axis (e.g. “episodes”, “samples”, or “bins”).
x (list[int]) – Range of the x parameter across which time and memory have been recorded.
times (list[int]) – Results of time complexity recording for different values of x.
memory (list[int]) – Results of memory complexity recording for different values of x.
- _run_training_loop(num_episodes, num_bins, num_samples)[source]
Run an example main.py.
Eventually, may load the hyperparameter data from a user-facing file.
- Parameters
num_episodes (int) – Number of episodes in the training loop.
num_bins (int) – Number of bins used for digitisation (equal number of bins for all dimensions).
num_samples (int) – Number of datapoint samples used for training.
- get_benchmark_performance()[source]
Get performance (space and time) for constant benchmark settings.
Note, the reason the timeit module is not used is because the piece of code being profiled is lengthy (whole training flow); timeit is designed to test small snippets of code by running them many times.
- get_performance_graphs()[source]
Plot performance (time and space) against chosen varying parameters.
Parameters: - Number of samples - Number of episodes - Number of bins (held constant for all dimensions)
- get_time_breakdown_per_function()[source]
Get a per-function breakdown of time complexity.
See: https://www.machinelearningplus.com/python/cprofile-how-to-profile-your-python-code/. And: https://stackoverflow.com/questions/51536411/saving-cprofile-results-to-readable-external-file.