external_generator_source

Module: missions.nodes.source.external_generator_source

Use external data as input

Inheritance diagram for pySPACE.missions.nodes.source.external_generator_source:

ExternalGeneratorSourceNode

class pySPACE.missions.nodes.source.external_generator_source.ExternalGeneratorSourceNode(**kwargs)

Bases: pySPACE.missions.nodes.base_node.BaseNode

Yield the data provided by an external generator

This node enables to easy input data from an external data generator within an internal data node chain.

The easiest usage is together with meta nodes, where the data is given explicitly to the node with the set_generator function.

This node only uses one data generator which is taken as testing data. For training usage an additional splitter is needed as usual.

Note

Below you can see a more complex example on how to make use of this node.

# Training is done in separate threads, we send the time series
# windows to these threads via two queues
self.queueS2 = Queue.Queue()
self.queueS3 = Queue.Queue()

# The two classification threads access the two queues via two 
# generators
def s2_generator():
    # Yield all windows until a None item is found in the queue
    while True:
        window = self.queueS2.get(block = True, timeout = None)
        if window == None: break
        yield window

def s3_generator():
    # Yield all windows until a None item is found in the queue
    while True:
        window = self.queueS3.get(block = True, timeout = None)
        if window == None: break
        yield window

# Create the actual data chains for S1 vs S2 discrimination
# and S1 vs S3 discrimination
self.S1S2 = NodeChainFactory.flow_from_yaml(Flow_Class = NodeChain,
                                  flow_spec_file = dataflow_spec_S2)
self.S1S2[0].set_generator(s2_generator())

self.S1S3 = NodeChainFactory.flow_from_yaml(Flow_Class = NodeChain,
                                  flow_spec_file = dataflow_spec_S3)
self.S1S3[0].set_generator(s3_generator())

Parameters

Exemplary Call

- 
    node: External_Generator_Source_Node

Author:

Jan Hendrik Metzen (jhm@informatik.uni-bremen.de)

Created:

2009/09/17

POSSIBLE NODE NAMES:  

• External_Generator_Source_Node
• ExternalGeneratorSourceNode
• ExternalGeneratorSource

POSSIBLE INPUT TYPES:  

• TimeSeries
• FeatureVector
• PredictionVector

Class Components Summary

__getstate__()	Return a pickable state for this object
get_metadata(key)	This source node does not contain collection meta data.
input_types
process()	Returns a generator that yields all data contained in the generator
register_input_node(node)	Register the given node as input
request_data_for_testing()	Returns time windows usable for testing of subsequent nodes
request_data_for_training(use_test_data)	Returns time windows usable for training of subsequent nodes
set_generator(generator)	Sets the generator from which this node reads the data
train_sweep(use_test_data)	Performs the actual training of the node.
use_next_split()	Use the next split (if any)

input_types = ['TimeSeries', 'FeatureVector', 'PredictionVector']

__init__(**kwargs)

set_generator(generator)

Sets the generator from which this node reads the data

register_input_node(node)

Register the given node as input

use_next_split()

Use the next split (if any)

train_sweep(use_test_data)

Performs the actual training of the node.

Note

Source nodes cannot be trained

request_data_for_training(use_test_data)

Returns time windows usable for training of subsequent nodes

request_data_for_testing()

Returns time windows usable for testing of subsequent nodes

process()

Returns a generator that yields all data contained in the generator

__getstate__()

Return a pickable state for this object

get_metadata(key)

This source node does not contain collection meta data.