Metric

Inheritance Diagram

---

class ashpy.metrics.metric.Metric(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/master/docs/source/log'))

Bases: abc.ABC

Metric is the abstract class that every AshPy Metric must implement.

AshPy Metric wrap and extend tf.keras.metrics.Metric.

Methods

__init__(name, metric[, …])	Initialize the Metric object.
json_read(filename)	Read a JSON file.
json_write(filename, what_to_write)	Write inside the specified JSON file the mean and stddev.
log(step)	Log the metric.
model_selection(checkpoint, global_step)	Perform model selection.
reset_states()	Reset the state of the metric.
result()	Get the result of the metric.
update_state(context)	Update the internal state of the metric, using the information from the context object.

Attributes

best_folder	Retrieve the folder used to save the best model when doing model selection.
best_model_sel_file	Retrieve the path to JSON file containing the measured performance of the best model.
logdir	Retrieve the log directory.
metric	Retrieve the tf.keras.metrics.Metric object.
model_selection_operator	Retrieve the operator used for model selection.
name	Retrieve the metric name.
sanitized_name	all / are _.

__init__(name, metric, model_selection_operator=None, logdir=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/master/docs/source/log'))

Initialize the Metric object.

Parameters:

• name (str) – Name of the metric.
• metric (tf.keras.metrics.Metric) – The Keras metric to use.
• model_selection_operator (typing.Callable) –

  The operation that will be used when model_selection is triggered to compare the metrics, used by the update_state. Any typing.Callable behaving like an operator is accepted.

  Note

  Model selection is done ONLY if an model_selection_operator is specified here.

• logdir (str) – Path to the log dir, defaults to a log folder in the current directory.

Return type:

None

best_folder

Retrieve the folder used to save the best model when doing model selection.

Return type:Path

best_model_sel_file

Retrieve the path to JSON file containing the measured performance of the best model.

Return type:Path

static json_read(filename)

Read a JSON file.

Parameters:filename (str) – The path to the JSON file to read. Return type:Dict[str, Any] Returns:typing.Dict – Dictionary containing the content of the JSON file.

static json_write(filename, what_to_write)

Write inside the specified JSON file the mean and stddev.

Parameters:

• filename (str) – Path to the JSON file to write.
• what_to_write (dict) – A dictionary containing what to write.

Return type:

None

log(step)

Log the metric.

Parameters:step (int) – global step of training Return type:None

logdir

Retrieve the log directory.

Return type:Path

metric

Retrieve the tf.keras.metrics.Metric object.

Return type:Metric

model_selection(checkpoint, global_step)

Perform model selection.

Parameters:

• checkpoint (tf.train.Checkpoint) – Checkpoint object that contains the model status.
• global_step (tf.Variable) – current training step

Return type:

Optional[Path]

model_selection_operator

Retrieve the operator used for model selection.

Return type:Optional[Callable]

name

Retrieve the metric name.

Return type:str

reset_states()

Reset the state of the metric.

Return type:None

result()

Get the result of the metric.

Returns:numpy.ndarray – The current value of the metric.

sanitized_name

all / are _.

This is done since adding a prefix to a metric name with a / allows for TensorBoard automatic grouping. When we are not working with TB we want to replace all / with _.

Type:Retrieve the sanitized name Return type:str

update_state(context)

Update the internal state of the metric, using the information from the context object.

Parameters:context (ashpy.contexts.Context) – An AshPy Context holding all the information the Metric needs. Return type:None