Metric

Inheritance Diagram

Inheritance diagram of ashpy.metrics.metric.Metric

class ashpy.metrics.metric.Metric(name, metric, model_selection_operator=None, logdir='/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/master/docs/source/log')[source]

Bases: abc.ABC

Metric is the abstract class that every ash Metric must implement.

AshPy Metrics wrap and extend Keras Metrics.

Methods

__init__(name, metric[, …])

Initialize the Metric object.

json_read(filename)

Read a JSON file.

json_write(filename, what_to_write)

Write inside the specified JSON file the mean and stddev.

log(step)

Log the metric

model_selection(checkpoint, global_step)

Perform model selection.

reset_states()

Reset the state of the metric.

result()

Get the result of the metric.

update_state(context)

Update the internal state of the metric, using the information from the context object.

Attributes

best_folder

Retrieve the folder used to save the best model when doing model selection.

best_model_sel_file

Retrieve the path to JSON file containing the measured performance of the best model.

logdir

Retrieve the log directory.

metric

Retrieve the tf.keras.metrics.Metric object.

model_selection_operator

Retrieve the operator used for model selection.

name

Retrieve the metric name.

__init__(name, metric, model_selection_operator=None, logdir='/home/docs/checkouts/readthedocs.org/user_builds/ashpy/checkouts/master/docs/source/log')[source]

Initialize the Metric object.

Parameters
  • name (str) – The name of the metric.

  • metric (tf.keras.metrics.Metric) – The Keras metric to use.

  • model_selection_operator (typing.Callable) –

    The operation that will be used when model_selection is triggered to compare the metrics, used by the update_state. Any typing.Callable behaving like an operator is accepted.

    Note

    Model selection is done ONLY if an model_selection_operator is specified here.

  • logdir (str) – Path to the log dir, defaults to a log folder in the current directory.

Return type

None

property best_folder

Retrieve the folder used to save the best model when doing model selection.

Return type

str

property best_model_sel_file

Retrieve the path to JSON file containing the measured performance of the best model.

Return type

str

static json_read(filename)[source]

Read a JSON file.

Parameters

filename (str) – The path to the JSON file to read.

Return type

Dict[str, Any]

Returns

typing.Dict – Dictionary containing the content of the JSON file.

static json_write(filename, what_to_write)[source]

Write inside the specified JSON file the mean and stddev.

Parameters
  • filename (str) – Path to the JSON file to write.

  • what_to_write (dict) – A dictionary containing what to write.

Return type

None

log(step)[source]

Log the metric

Parameters

step (int) – global step of training

Return type

None

property logdir

Retrieve the log directory.

Return type

str

property metric

Retrieve the tf.keras.metrics.Metric object.

Return type

Metric

model_selection(checkpoint, global_step)[source]

Perform model selection.

Parameters
Return type

None

property model_selection_operator

Retrieve the operator used for model selection.

Return type

Optional[Callable]

property name

Retrieve the metric name.

Return type

str

reset_states()[source]

Reset the state of the metric.

Return type

None

result()[source]

Get the result of the metric.

Returns

numpy.ndarray – The current value of the metric.

abstract update_state(context)[source]

Update the internal state of the metric, using the information from the context object.

Parameters

context (ashpy.contexts.BaseContext) – An AshPy Context holding all the information the Metric needs.

Return type

None