base

Module: resources.data_types.base

Type superclass providing common variables that survive the data processing

The BaseData is almost invisible for the user. No matter which data type is used, all of them inherit from this class. This inheritance is basically performed by the function inherit_meta_from where a new object can inherit the meta data from another object. This is necessary between the nodes and is automatically performed by the nodes.

But what is the meta data? Four elements are inherent in the BaseData type: key, tag, specs and history. These elements are Python properties, so it is recommended to know about the set and get methods of these properties.

Note

Just by loading an element within the framework, none of these elements is set. Instead, they are first set within the processing node chain using generate_meta.

Central elements are the unique identifier key and an identifying tag. Both complement each other and are automatically set by the method generate_meta which is in called within the node.

The key must be a UUID and the tag must be a STRING giving unique information about the underlying data. Every data type can have an automatic tag generation method (called _generate_tag). In particular, the key is automatically created using a uuid when the element is processed for the first time and then belongs to this data element. The tag is very similar, but its generation can be implemented by the subclass, because it contains semantic information (like e.g. time stamps) which is not known by the BaseData class.

The other two properties are more flexible: the specs and the history are

able to backup information the user is needing. specs is a dictionary and the history is a list. Both are empty by default.

The history is a placeholder (empty list) for storing results of previous nodes within a node chain. It can be filled by setting the keep_in_history parameter to True within a node chain. Then the data object is copied into the history as it is after this processing step. Additionally, the specs receive an entry labeled ‘node_specs’ containing a dictionary of additional information from the saving node. For n times storing the results within one node chain, the history property has the length n.

Note

(I) When a data element is stored in the history, it loses its own history. (II) Using history=myhistory appends something to the history; the only way of deleting information here is to empty it (history=[]).

The specs property is an empty dictionary which can be filled with whatever

the user or developer needs in more than one node. The syntax is exactly the

same as with a normal python dictionary.

Note

All BaseData type properties survive data type conversion within a NodeChain (e.g. from TimeSeries to FeatureVector)! The central function used for this is inherit_meta_from().

Warning

When slicing the data, all meta data is copied. So please try to avoid slicing and better use ‘x=data.view(numpy.ndarray)’ to replace data by x for further array processing. This does not copy memory, but creates a new clean reference to the array without meta data.

Author:Hendrik Woehrle, Sirko Straube, Mario Krell, David Feess Created:2010/08/09 Major Revision:2012/01/20

Inheritance diagram for pySPACE.resources.data_types.base:

BaseData

class pySPACE.resources.data_types.base.BaseData

Bases: numpy.ndarray

Basic Data object

Superclass for every data type - provides common variables that survive the data processing.

For further details refer to the module documentation.

Class Components Summary

__array_finalize__(obj)
__del_history__()	delete history
__del_key__()	delete key
__del_specs__()	delete specs
__del_tag__()	delete tag
__generate_key__()	uuid for key
__get_history__()	return history
__get_key__()	return key
__get_specs__()	return specs
__get_tag__()	return tag
__new__(subtype, input_array)	Constructor for BaseData object
__reduce__()
__set_history__(elem)	set method for history
__set_key__(new_key)	set method for key
__set_specs__(key[, value])	set method for specs
__set_tag__(new_tag)	set method for tag
__setstate__(state)
add_to_history(obj[, node_specs])	Add the passed object to history.
generate_meta()	generate basic meta data (key and tag)
get_data()	A simple method that returns the data as a numpy array
has_history()	Return whether history is present.
has_meta()	Return whether basic meta data is present (key and tag)
history	Property history of BaseData type.
inherit_meta_from(obj)	Inherit history, key, tag and specs from the passed object
key	Property key of BaseData type.
specs	Property specs of BaseData type.
tag	Property tag of BaseData type.

static __new__(subtype, input_array)

Constructor for BaseData object

If the input_array has meta data, it will survive the construction process and be present in the new object.

Refer to http://docs.scipy.org/doc/numpy/user/basics.subclassing.html for infos about subclassing ndarray

__array_finalize__(obj)

__reduce__()

__setstate__(state)

__generate_key__()

uuid for key

__set_key__(new_key)

set method for key

Generally the key should be generated one time and then stay the same. When creating it, a uuid has to be used! Then, the user can still change it: Either he can set it to None or use another uuid.

__get_key__()

return key

__del_key__()

delete key

key

Property key of BaseData type.

__set_tag__(new_tag)

set method for tag

Similar behavior to __set_key__. The tag is ALWAYS a string. So whatever is given as new_tag is casted into string (exception: None).

__get_tag__()

return tag

__del_tag__()

delete tag

tag

Property tag of BaseData type.

__set_specs__(key, value=None)

set method for specs

The specs property is a dictionary, so every attempt to add something here is interpreted as an attempt to fill this dictionary with dict.__setitem__(self.specs, key, value). The key is always casted into a string. If value is ‘None’ the operation is not performed.

__get_specs__()

return specs

__del_specs__()

delete specs

specs

Property specs of BaseData type. This property is a dictionary.

__set_history__(elem)

set method for history

The history is a list of elements. Therefore, the history can only be set by the user in two ways: - it can be deleted by setting history=[] or - it can be extended with another element

__get_history__()

return history

__del_history__()

delete history

history

Property history of BaseData type. This property is a list.

has_meta()

Return whether basic meta data is present (key and tag)

has_history()

Return whether history is present.

get_data()

A simple method that returns the data as a numpy array

generate_meta()

generate basic meta data (key and tag)

inherit_meta_from(obj)

Inherit history, key, tag and specs from the passed object

add_to_history(obj, node_specs=None)

Add the passed object to history.

further specs can be given here which are passed into specs property of the object (default: None)