windower

Module: missions.support.windower

Performs windowing of incoming stream and produces instances of fixed length for preprocessing and classification.

The SlidingWindower class performs windowing for the online setting where no markers are available.

The MarkerWindower class extracts windows according to definitions like the presence or non-presence of markers. Additionally, exclude conditions can be defined that exclude certain markers in proximity to extracted events.

The WindowFactory loads a windowing specification from a yaml file. The window definitions are then stored in a dictionary which is then used by one of the Windowers (MarkerWindower, SlidingWindower etc.) to cut the incoming data stream.

The Windower definition is always application specific and can contain many keys/values. In order to construct your own windower definition see the short explanation in MarkerWindower.

Time is always measured in ms. If there are mistakes in time, this should be because of unknown block size or frequency.

Additionally include conditions can be added to ensure the presence of certain markers in a specific range. So ‘or’ conditions between the conditions are reached by repeating the definitions of the marker with different exclude or include definitions and ‘and’ conditions are simply reached by concatenation. Negation is now possible by switching to the other kind of condition.

Author:Timo Duchrow Created:2008/08/29 modified:Mario Michael Krell (include and exclude defs)

Inheritance diagram for pySPACE.missions.support.windower:

Class Summary

Windower(data_client)	Windower base class
SlidingWindower(data_client[, windowsizems, ...])	An iterable class that produces sliding windows for online classification.
MarkerWindower(data_client[, windowdefs, ...])	returns (<numpy.ndarray> window, <str> class)
MarkerWindowerException(arg)
LabeledWindowDef(windef_name, classname, ...)	Labeled window definition that is to be extracted from EEG stream.
ExcludeDef(markername, preexcludems, ...)	Definition of exclude constraints for window extraction.
IncludeDef(markername, preincludems, ...)	Definition of include constraints for window extraction.
RingBuffer(size_max)	Generic ring buffer class
RingBufferFull(n)	Generic ring buffer when full
WindowFactory	Factory class to create window definition objects with static methods

Classes

Windower

class pySPACE.missions.support.windower.Windower(data_client)

Bases: object

Windower base class

Class Components Summary

_load_window_spec([windower_spec, ...])	Load the window definitions to extract the labeled samples
_mstosamples(ms)	Convert from milliseconds to number of samples based on the parameters of data_client.
_samplestoms(samples)	Convert from number of samples to milliseconds based on the parameters of data_client.

__init__(data_client)

_mstosamples(ms)

Convert from milliseconds to number of samples based on the parameters of data_client.

_samplestoms(samples)

Convert from number of samples to milliseconds based on the parameters of data_client.

classmethod _load_window_spec(windower_spec='', local_window_conf=False)

Load the window definitions to extract the labeled samples

Parameters

windower_spec:

file name of the windower specification

(optional, default:`default_windower_spec`)

local_window_conf:  

Windower file is looked up in the local directory if set True.

Otherwise it is looked up in the subdirectory windower in node_chains in the spec_dir, which is the better way of using it.

(optional, default: False)

__weakref__

list of weak references to the object (if defined)

SlidingWindower

class pySPACE.missions.support.windower.SlidingWindower(data_client, windowsizems=1000, stridems=100, underfull=False)

Bases: pySPACE.missions.support.windower.Windower

An iterable class that produces sliding windows for online classification.

Class Components Summary

__iter__()
_addblock(ndsamples, ndmarkers)	Add incoming data block to ring buffers
next()	Retrieve the next window according to windowsize and stride.

__init__(data_client, windowsizems=1000, stridems=100, underfull=False)

__iter__()

next()

Retrieve the next window according to windowsize and stride.

_addblock(ndsamples, ndmarkers)

Add incoming data block to ring buffers

MarkerWindower

class pySPACE.missions.support.windower.MarkerWindower(data_client, windowdefs=None, debug=False, nullmarker_stride_ms=1000, no_overlap=False, data_consistency_check=False)

Bases: pySPACE.missions.support.windower.Windower

returns (<numpy.ndarray> window, <str> class)

MarkerWindower maintains a ring buffer for incoming sample blocks. The buffer is divided into three segments:

Example:

t0                  t1                  t3                  t4     <---
+---------+---------+---------+---------+---------+---------+---------+
| block 1 | block 2 | block 3 | block 4 | block 5 | block 6 | block 7 |
+---------+---------+---------+---------+---------+---------+---------+

|<      prebuflen = 3        >|         |<      postbuflen = 3       >|

[////////////////////////////][---------][/////////////////////////////]
            history           ``current''          lookahead
                                                             __________
                                                                 scan

MarkerWindower scans windows for markers as they come in (block 7 in example). When the block passes into the current section all windows are extracted that meet the constraints. To accomplish this prebuflen and postbuflen have been calculated so that the buffer enables extraction of sufficient window lengths for all window definitions as well as lookaheads.

A windowdef looks like:

startmarker : "S  8"
endmarker : "S  9"
skip_ranges :
         - {start : 0, end: 300000}
window_defs :
     s16:
         classname : LRP
         markername : "S 16"
         startoffsetms : -1280
         endoffsetms : 0
         excludedefs : []
         includedefs : [immediate response]
     null:
         classname : NoLRP
         markername : "null"
         startoffsetms : -1280
         endoffsetms : 0
         excludedefs : [all]
exclude_defs:
      all:
        markernames : ["S  1", "S  2", "S  8", "S 16", "S 32"]
        preexcludems : 2000
        postexcludems : 2000
include_defs:
     immediate_response:
         markernames : ["S 32"]
         preincludems: -200
         postincludems: 1200

Parameters

startmarker:

name of the marker where at the earliest cutting begins

endmarker:

name of the marker where at the latest cutting ends

skip_ranges:

Not completely implemented! The ‘end’ component results in the parameter skipfirstms which tells the windower, which time points to skip at the beginning.

window_def:

includes names of definitions of window cuts

classname:

name of the label given to the window, when cut

markername:

name of the marker being in the ‘current block’. MUST BE A STRING!

Note

The null marker is a synthetic marker, which is internally added to the stream every nullmarker_stride_ms milliseconds. Currently, this parameter has to be set separately and is 1000ms by default.

startoffsetms:

start of the window relative to the marker in the ‘current block’

endoffsetms:

end of the window relative to the marker in the ‘current block’

jitter:

Not implemented! Was intended to add an artificial jittering during the segmentation.

exclude_defs:

excludes each marker in markernames defined by the interval ‘[-preexcludems, postexludems]’ relative to the window marker lying at zero

preexcludems:

time before the window marker, where the exclude markers are forbidden. This time can be chosen negative,

postexcludems:

time after the window marker, where the exclude markers are forbidden. This time can be chosen negative.

include_defs:

everything is the same to exclude defs, except, that one of the specified markers has to lie in the interval.

Time is always measured in ms. If there are mistakes in time, this should be because of unknown block size or frequency.

Class Parameters

data_client:

Client, delivering the data

windowdefs:

List of window definitions generated by WindowFactory.create_window_defs()

(optional, default: None)

debug:

Enable debug print outs to command line

(optional, default: False)

nullmarker_stride_ms:  

Set artificial markers with this constant distance into the stream with the Name “null”. If this parameter is set to None, no artificial markers are generated.

(optional, default: 1000)

no_overlap:

Ignore the last sample in each window (important for streaming data)

(optional, default: False)

data_consistency_check:  

Currently it is only checked, that the standard deviation is not 0

(optional, default: False)

Class Components Summary

__iter__()
_addblock(ndsamples, ndmarkers)	Add incoming block to ring buffer.
_check_exclude_defs_ok(markeroffset, excludedefs)	Check whether the exclude definitions match
_check_include_defs_ok(markeroffset, includedefs)	Check whether all the include definitions match
_decmarkeroffsets()	Decrement all offsets for markers in buffer as new blocks come in.
_extract_markers_cur_window(...)	Filter out all markers that lie in the current window to store this information.
_extract_windows_cur_block()	Add windows for markers in current block to self.cur_extract_windows.
_extractwindow(cur_sample_block_offset, ...)	Extracts a sample window from the ring buffer and consolidates it into a single numpy array object.
_insertnullmarkers([debug])	Insert epsilon markers according to nullmarker stride.
_max_scan_ranges()	Scan window and constraint definitions to determine maximum extent of buffer for marker and for samples.
_readnextblock()	Read next block from EEG stream client.
_scanmarkers(ndmarkers[, debug])	Scan incoming block for markers.
next([debug])	Return next labeled window when used in iterator context.

__init__(data_client, windowdefs=None, debug=False, nullmarker_stride_ms=1000, no_overlap=False, data_consistency_check=False)

_max_scan_ranges()

Scan window and constraint definitions to determine maximum extent of buffer for marker and for samples. Return (max_postscan_samples, max_prescan_samples, max_prescan_markers)

_decmarkeroffsets()

Decrement all offsets for markers in buffer as new blocks come in.

_addblock(ndsamples, ndmarkers)

Add incoming block to ring buffer.

_insertnullmarkers(debug=False)

Insert epsilon markers according to nullmarker stride.

_scanmarkers(ndmarkers, debug=False)

Scan incoming block for markers.

self.buffermarkers contains offsets of markers w.r.t. to current block @ position 0

__iter__()

next(debug=False)

Return next labeled window when used in iterator context.

_readnextblock()

Read next block from EEG stream client.

_extract_windows_cur_block()

Add windows for markers in current block to self.cur_extract_windows.

_check_exclude_defs_ok(markeroffset, excludedefs)

Check whether the exclude definitions match

Note

Changes in this section need to be checked also in the following -check_include_defs_ok class, because they are very similar.

_check_include_defs_ok(markeroffset, includedefs)

Check whether all the include definitions match

_extractwindow(cur_sample_block_offset, start_offset, end_offset, debug=False)

Extracts a sample window from the ring buffer and consolidates it into a single numpy array object.

_extract_markers_cur_window(buf_extract_start, buf_extract_end)

Filter out all markers that lie in the current window to store this information. The markers are stored with their clear name and temporal offset.

MarkerWindowerException

class pySPACE.missions.support.windower.MarkerWindowerException(arg)

Bases: exceptions.Exception

__init__(arg)

__weakref__

list of weak references to the object (if defined)

LabeledWindowDef

class pySPACE.missions.support.windower.LabeledWindowDef(windef_name, classname, markername, startoffsetms, endoffsetms, excludedefs=None, includedefs=None, skipfirstms=None, jitter=None, startmarker=None, endmarker=None)

Bases: object

Labeled window definition that is to be extracted from EEG stream.

Class Components Summary

__str__()

__init__(windef_name, classname, markername, startoffsetms, endoffsetms, excludedefs=None, includedefs=None, skipfirstms=None, jitter=None, startmarker=None, endmarker=None)

__str__()

__weakref__

list of weak references to the object (if defined)

ExcludeDef

class pySPACE.missions.support.windower.ExcludeDef(markername, preexcludems, postexcludems)

Bases: object

Definition of exclude constraints for window extraction.

Class Components Summary

__str__()

__init__(markername, preexcludems, postexcludems)

__str__()

__weakref__

list of weak references to the object (if defined)

IncludeDef

class pySPACE.missions.support.windower.IncludeDef(markername, preincludems, postincludems)

Bases: object

Definition of include constraints for window extraction.

Class Components Summary

__str__()

__init__(markername, preincludems, postincludems)

__str__()

__weakref__

list of weak references to the object (if defined)

RingBuffer

class pySPACE.missions.support.windower.RingBuffer(size_max)

Generic ring buffer class

Class Components Summary

__len__()
__str__()
append(x)	append an element at the end of the buffer
get()	return a list of elements from the oldest to the newest

__init__(size_max)

append(x)

append an element at the end of the buffer

get()

return a list of elements from the oldest to the newest

__str__()

__len__()

RingBufferFull

class pySPACE.missions.support.windower.RingBufferFull(n)

Bases: pySPACE.missions.support.windower.RingBuffer

Generic ring buffer when full

Class Components Summary

append(x)
get()

__init__(n)

append(x)

get()

WindowFactory

class pySPACE.missions.support.windower.WindowFactory

Bases: object

Factory class to create window definition objects with static methods

This WindowFactory provides static methods in order to read a given Windower specification file, which should be a valid YAML specification of a window defs, and returns a list of the containing window definitions.

Author:Jan Hendrik Metzen (jhm@informatik.uni-bremen.de) Created:2008/11/25

Class Components Summary

create_window_defs(window_specs)	Reads from the given file, which should be a valid
default_windower_spec()
window_definitions_from_yaml(yaml_file)

__weakref__

list of weak references to the object (if defined)

static default_windower_spec()

static window_definitions_from_yaml(yaml_file)

static create_window_defs(window_specs)

Reads from the given file, which should be a valid YAML specification of a window defs and returns a list of the window definitions