metric¶

Metric is the abstract class that every ash metric must implement.

Classes

Metric Metric is the abstract class that every ash Metric must implement.

class ashpy.metrics.metric.Metric(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/stable/docs/source/log'))[source]¶

Bases: abc.ABC

Metric is the abstract class that every ash Metric must implement.

AshPy Metrics wrap and extend Keras Metrics.

__init__(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/stable/docs/source/log'))[source]¶

Initialize the Metric object.

Parameters:

name (str) – Name of the metric.
metric (tf.keras.metrics.Metric) – The Keras metric to use.
model_selection_operator (typing.Callable) –
The operation that will be used when model_selection is triggered to compare the metrics, used by the update_state. Any typing.Callable behaving like an operator is accepted.

Note

Model selection is done ONLY if an model_selection_operator is specified here.
logdir (str) – Path to the log dir, defaults to a log folder in the current directory.

Return type:

None

best_folder¶

Retrieve the folder used to save the best model when doing model selection.

Return type:	`Path`

best_model_sel_file¶

Retrieve the path to JSON file containing the measured performance of the best model.

Return type:	`Path`

static json_read(filename)[source]¶

Read a JSON file.

Parameters:	filename (str) – The path to the JSON file to read.
Return type:	`Dict`[`str`, `Any`]
Returns:	`typing.Dict` – Dictionary containing the content of the JSON file.

static json_write(filename, what_to_write)[source]¶

Write inside the specified JSON file the mean and stddev.

Parameters:	filename (str) – Path to the JSON file to write. what_to_write (dict) – A dictionary containing what to write.
Return type:	`None`

log(step)[source]¶

Log the metric.

Parameters:	step (`int`) – global step of training
Return type:	`None`

logdir¶

Retrieve the log directory.

Return type:	`Path`

metric¶

Retrieve the tf.keras.metrics.Metric object.

Return type:	`Metric`

model_selection(checkpoint, global_step)[source]¶

Perform model selection.

Parameters:	checkpoint (`tf.train.Checkpoint`) – Checkpoint object that contains the model status. global_step (`tf.Variable`) – current training step
Return type:	`Optional`[`Path`]

model_selection_operator¶

Retrieve the operator used for model selection.

Return type:	`Optional`[`Callable`]

name¶

Retrieve the metric name.

Return type:	`str`

reset_states()[source]¶

Reset the state of the metric.

Return type:	`None`

result()[source]¶

Get the result of the metric.

Returns:	`numpy.ndarray` – The current value of the metric.

sanitized_name¶

all / are _.

This is done since adding a prefix to a metric name with a / allows for TensorBoard automatic grouping. When we are not working with TB we want to replace all / with _.

Type:	Retrieve the sanitized name
Return type:	`str`

update_state(context)[source]¶

Update the internal state of the metric, using the information from the context object.

Parameters:	context (`ashpy.contexts.Context`) – An AshPy Context holding all the information the Metric needs.
Return type:	None