# Metric¶

Inheritance Diagram

class ashpy.metrics.metric.Metric(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/latest/docs/source/log'))[source]

Bases: abc.ABC

Metric is the abstract class that every ash Metric must implement.

AshPy Metrics wrap and extend Keras Metrics.

Methods

 __init__(name, metric[, …]) Initialize the Metric object. json_read(filename) Read a JSON file. json_write(filename, what_to_write) Write inside the specified JSON file the mean and stddev. log(step) Log the metric. model_selection(checkpoint, global_step) Perform model selection. reset_states() Reset the state of the metric. result() Get the result of the metric. update_state(context) Update the internal state of the metric, using the information from the context object.

Attributes

 best_folder Retrieve the folder used to save the best model when doing model selection. best_model_sel_file Retrieve the path to JSON file containing the measured performance of the best model. logdir Retrieve the log directory. metric Retrieve the tf.keras.metrics.Metric object. model_selection_operator Retrieve the operator used for model selection. name Retrieve the metric name. sanitized_name all / are _.
__init__(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/latest/docs/source/log'))[source]

Initialize the Metric object.

Parameters: name (str) – Name of the metric. metric (tf.keras.metrics.Metric) – The Keras metric to use. model_selection_operator (typing.Callable) – The operation that will be used when model_selection is triggered to compare the metrics, used by the update_state. Any typing.Callable behaving like an operator is accepted. Note Model selection is done ONLY if an model_selection_operator is specified here. logdir (str) – Path to the log dir, defaults to a log folder in the current directory. None
best_folder

Retrieve the folder used to save the best model when doing model selection.

Return type: Path
best_model_sel_file

Retrieve the path to JSON file containing the measured performance of the best model.

Return type: Path
static json_read(filename)[source]

Parameters: filename (str) – The path to the JSON file to read. Dict[str, Any] typing.Dict – Dictionary containing the content of the JSON file.
static json_write(filename, what_to_write)[source]

Write inside the specified JSON file the mean and stddev.

Parameters: filename (str) – Path to the JSON file to write. what_to_write (dict) – A dictionary containing what to write. None
log(step)[source]

Log the metric.

Parameters: step (int) – global step of training None
logdir

Retrieve the log directory.

Return type: Path
metric

Retrieve the tf.keras.metrics.Metric object.

Return type: Metric
model_selection(checkpoint, global_step)[source]

Perform model selection.

Parameters: checkpoint (tf.train.Checkpoint) – Checkpoint object that contains the model status. global_step (tf.Variable) – current training step
model_selection_operator

Retrieve the operator used for model selection.

Return type: Optional[Callable]
name

Retrieve the metric name.

Return type: str
reset_states()[source]

Reset the state of the metric.

Return type: None
result()[source]

Get the result of the metric.

Returns: numpy.ndarray – The current value of the metric.
sanitized_name

all / are _.

This is done since adding a prefix to a metric name with a / allows for TensorBoard automatic grouping. When we are not working with TB we want to replace all / with _.

Type: Retrieve the sanitized name str
update_state(context)[source]

Update the internal state of the metric, using the information from the context object.

Parameters: context (ashpy.contexts.Context) – An AshPy Context holding all the information the Metric needs. None