base

Module: missions.operations.base

Interface for the numerous operations and processes

An operation takes one summary of data as input and produces a second one as output with a set of processes.

Each process can be considered as an atomic task that can be executed independently from all other processes of the operation.

Standard Specification File Parameters

type

Specifies which operation is used. Type and corresponding module name are the same in lower case (with underscore). The respective operation name is written in camel case with the ending Operation.

(obligatory)

input_path

Path name relative to the storage in your configuration file.

The input has to fulfill certain rules, specified in the fitting: pySPACE.resources.dataset_defs subpackage.

Depending on the operation, the input datasets are combined, or each parameter combination is applied separately to the data.

(mostly obligatory)

storage_format

Some datasets give the opportunity to choose between different formats for storing the result. The choice can be specified here. For details look at the dataset_defs documentation. If the format is not specified, the default of the dataset is used.

(optional, default: default of dataset)

hide_parameters

Normally each parameter is added in the format {__PARAMETER__: value} added to the __RESULT_DIRECTORY__. Every parameter specified in the list hide_parameters is not added to the folder name. Be careful not to have the same name for different results.

(optional, default: [])

runs

Number of repetitions for each running process. The run number is given to the process, to use it for choosing random components, especially when using cross-validation.

(optional, default: 1)

backend

Overwrite the backend defined in the command line or, if not specified, the default (serial) backend

(optional, default: command line or default backend)

others

For the operation specific parameters have a look at operation documentation.

Inheritance diagram for pySPACE.missions.operations.base:

Class Summary

Operation(processes, operation_spec, ...)	Take one summary of data as input and produces a second one as output with a set of processes.,
Process()	A process is an atomic task that can be executed on an arbitrary backend modality by executing the __call__ method.

Function Summary

create_operation_from_file(operation_filename)	Creates the operation for the file operation_filename
create_operation(operation_spec[, ...])	Creates the operation for the given operation spec

Classes

Operation

class pySPACE.missions.operations.base.Operation(processes, operation_spec, result_directory)

Bases: object

Take one summary of data as input and produces a second one as output with a set of processes., that can be executed on an arbitrary backend modality

Class Components Summary

__repr__()	Return a representation of this class
_get_parameter_space(operation_spec)	Determines all parameter combinations that should be tested
_log(message[, level])	Logs the given message with the given logging level
consolidate(results)	Consolidates the results obtained by the single processes of this operation into a consistent structure of collections.
create(operation_spec[, base_result_dir])	A factory method that calls the responsible method for creating an operation of the type specified in the operation specification dictionary (operation_spec).
get_output_directory()	Returns the directory where the output is stored
get_unique_result_dir(base_result_dir)	Creates a new result directory with current time stamp

__init__(processes, operation_spec, result_directory)

consolidate(results)

Consolidates the results obtained by the single processes of this operation into a consistent structure of collections.

_log(message, level=20)

Logs the given message with the given logging level

classmethod create(operation_spec, base_result_dir=None)

A factory method that calls the responsible method for creating an operation of the type specified in the operation specification dictionary (operation_spec).

classmethod get_unique_result_dir(base_result_dir)

Creates a new result directory with current time stamp

classmethod _get_parameter_space(operation_spec)

Determines all parameter combinations that should be tested

get_output_directory()

Returns the directory where the output is stored

__repr__()

Return a representation of this class

__weakref__

list of weak references to the object (if defined)

Process

class pySPACE.missions.operations.base.Process

Bases: object

A process is an atomic task that can be executed on an arbitrary backend modality by executing the __call__ method.

Class Components Summary

__repr__()	Return a representation of this class
_log(message[, level])	Log the given message into the logger of this class
post_benchmarking()	Execute some code which is not specific for the respective operation
pre_benchmarking()	Execute some code which is not specific for the respective operation but needs to be executed before benchmarking remotely (e.g.
prepare(configuration[, handler_class, ...])	This method is called by the respective backend in order to prepare

__init__()

prepare(configuration, handler_class=None, handler_args=None, backend_com=None)

This method is called by the respective backend in order to prepare the process for remote execution

__repr__()

Return a representation of this class

pre_benchmarking()

Execute some code which is not specific for the respective operation but needs to be executed before benchmarking remotely (e.g. set up logging, setting environment variables etc.)

post_benchmarking()

Execute some code which is not specific for the respective operation but needs to be executed after benchmarking remotely to clean up

_log(message, level=20)

Log the given message into the logger of this class

__weakref__

list of weak references to the object (if defined)

Functions

create_operation_from_file()

pySPACE.missions.operations.base.create_operation_from_file(operation_filename, base_result_dir=None)

Creates the operation for the file operation_filename

Creates the operation for a given operation filename. If operation_filename is an absolute path, it is expected to be the path to the YAML specification file of an operation. Alternatively, if operation_filename is not an absolute path, the name is used to look up a YAML specification file for the operation in the spec dir.

If base_result_dir is not None, the results of the operation are written into the specified directory.

create_operation()

pySPACE.missions.operations.base.create_operation(operation_spec, base_result_dir=None)

Creates the operation for the given operation spec

Simple wrapper to the create function of the operation, getting the operation result dir from the configuration file.