Paradigm Scripting¶
Introduction¶
The paradigm is programmed via a so-called paradigm file. A paradigm file contains regular Python code and must define the Paradigm class, which must derive from the ParadigmBase parent class. Its code is automatically loaded when YAGA starts. A typical skeleton looks like this:
from yaga_modules.paradigm_base import ParadigmBase
import yaga_modules.graphic_objects as GO
import yaga_modules.audio_objects as AO
import yaga_modules.signal_processing as SP
class Paradigm(ParadigmBase):
def __init__(self, paradigm_variables):
super().__init__(paradigm_variables)
[…]
Creation of a Presentation Objects¶
YAGA supports the timed presentation of several graphical or auditory objects, e.g., text, circles, or beep tones. A presentation object needs to be created within the __init__ constructor of the Paradigm class and registered to YAGA. A simple text object can be created as:
info_text = self.registerObject(GO.Text('prepare for task'))
In this example, GO.Text refers to the Text class defined in the graphic_objects module (GO). The call of registerObject is obligatory and registers the object to YAGA.
Timing of Presentation Objects¶
Once an object has been created and registered, state changes of the object can be scripted, i.e., timed. For that purpose, the ParadigmBase parent class provides the instance variable script. script must be a list of ScriptItem objects.
A ScripItem object specifies:
- a list of actions to execute when triggered
- the trigger (time, signal or LSL event marker)
- an informative name which is sent as an LSL event marker when the ScriptItem is triggered
Demo Paradigm¶
The paradigm demo.py implements a simple paradigm where study participants execute a task after a Go cue. The paradigm shows an instruction text and a traffic light on the screen. The traffic light represents the Go cue. After the traffic light, a countdown starts. While the countdown is running, the study participants would be asked to execute a certain task. The demo paradigm file shows the creation of text, image, and countdown objects and the set up of the paradigm sequence with script items.
A trial comprises the following script items: “trial_start”, “traffic_ligh_red”, “traffic_ligh_yellow”, “traffic_ligh_green”, “countdown”, “trial_end”. All script items are executed sequentially. Each script item optionally specifies the actions to be executed when the script item is triggered. Note that the actions must be specified as objects, not as actual method calls (i.e., no parentheses). The script list contains a total of 5 trial sequences, and is built using a for loop.
The demo paradigm file demonstrates three possible trigger types: absolute time, relative time, and signal-based. An absolute time trigger is used by the first “trial_start” script item, which is triggered 3s after program start. The subsequent “trial_start” script items use a relative time trigger, and are triggered 5s to 10s after the previous “trial_end” script item. A relative time trigger is specified by setting the time_type property to "rel" and the rel_name property to the name of the referenced script item. A signal-based trigger is used by the “trial_end” script item. This script item is triggered when the countdown object elicits the signal specified by the “wait_for_signal” property (COUNTDOWN_FINISHED).
The full source code of the demo paradigm file:
import numpy as np
from yaga_modules.paradigm_base import ParadigmBase, ScriptItem
import yaga_modules.graphic_objects as GO
class Paradigm(ParadigmBase):
task_name = "demo"
def __init__(self, paradigm_variables):
super().__init__(paradigm_variables, lsl_recorder_remote_control=False, lsl_recorder_host="localhost")
number_of_trials = 5
pre_paradigm_interval = 3 # seconds
post_paradigm_interval = 3 # seconds
inter_trial_interval_min = 5 # seconds
inter_trial_interval_max = 10 # seconds
demo_task_duration = 5 # seconds
# graphical objects
instruction_text = self.registerObject(GO.Text("prepare for demo task", scale_x=0.1, scale_y=0.1, color="white"))
countdown = self.registerObject(GO.Countdown(counter_start=demo_task_duration, scale_x=0.5, scale_y=0.5, color="white"))
traffic_light_green = self.registerObject(GO.Image("traffic_light_green.png", scale_x=0.2, scale_y=0.2*1.7))
traffic_light_yellow = self.registerObject(GO.Image("traffic_light_yellow.png", scale_x=0.2, scale_y=0.2*1.7))
traffic_light_red = self.registerObject(GO.Image("traffic_light_red.png", scale_x=0.2, scale_y=0.2*1.68))
# paradigm sequence definition
self.script = []
for trial_idx in range(number_of_trials):
# the first trial has an absolute time trigger, the subsequent trials have a relative time trigger
if trial_idx == 0:
trial_start_time = pre_paradigm_interval
trial_start_time_type = "abs"
trial_start_time_ref = ""
else:
trial_start_time = np.random.uniform(inter_trial_interval_min, inter_trial_interval_max)
trial_start_time_type = "rel"
trial_start_time_ref = "trial_end"
self.script.append(ScriptItem(name="trial_start", time=trial_start_time, time_type=trial_start_time_type, rel_name=trial_start_time_ref, actions=[instruction_text.activate]))
self.script.append(ScriptItem(name="traffic_light_red", time=3, time_type="rel", rel_name="trial_start", actions=[instruction_text.deactivate, traffic_light_red.activate]))
self.script.append(ScriptItem(name="traffic_light_yellow", time=5, time_type="rel", rel_name="trial_start", actions=[traffic_light_red.deactivate, traffic_light_yellow.activate]))
self.script.append(ScriptItem(name="traffic_light_green", time=7, time_type="rel", rel_name="trial_start", actions=[traffic_light_yellow.deactivate, traffic_light_green.activate]))
self.script.append(ScriptItem(name="countdown", time=9, time_type="rel", rel_name="trial_start", actions=[traffic_light_green.deactivate, countdown.activate]))
self.script.append(ScriptItem(name="trial_end", wait_for_signal=GO.Countdown.COUNTDOWN_FINISHED, actions=[]))
self.script.append(ScriptItem(time=post_paradigm_interval, time_type="rel", rel_name="trial_end", actions=[]))
Script Items¶
The paradigm sequence is defined by script items, which must be listed in the script instance variable of your Paradigm class. The script items are processed one by one in the order in which they appear in the instance variable script. A script item can be triggered by a time event, by a signal event generated by a graphical object, or by an external LSL marker event. You can also combine different trigger types. The script item is then triggered by whichever event comes first. A script item can be defined as:
ScriptItem(name='trial_start', time=10, actions=[info_text.activate])
In this example, the script item will call the method activate on the graphical object info_text at second 10 after the program start.
The name of the script item is sent as an event marker in the LSL stream yaga when the item is triggered. This allows recording programs to save the paradigm state together with the LSL data streams (see Integration with LSL).
Note that the execution of script items is synchronised to the screen refresh rate (often 60 Hz).
Actions¶
The actions parameter of a ScriptItem expects a list of methods which are called when the action item is triggered. The available actions depend on the presentation object and are listed in Supported Graphical Objects and Auditory Objects.
The actions must be specified as objects and not as method calls, i.e., without subsequent parentheses. This is to ensure that actions are executed when the script item is triggered and not at the program start when the initialisation method is executed.
Some actions accept parameters. You can specify parameters by using the partial function from the functools module. The partial function allows to fix arguments and generates a new function object. This new function object is then executed when the action item is triggered.
First import partial:
from functools import partial
Then use partial to create a new function object with a fixed set of parameters, e.g.:
ScriptItem(name='trial_start', time=10, actions=[targets.activate, partial(targets.setActiveTarget, 1)])
In this example, targets.setActiveTarget(1) is executed when the script item is triggered.
Time Triggers¶
YAGA supports two types of time triggers: absolute time triggers and relative time triggers. A trigger specified with an absolute time is triggered at the specified time after the program start. A trigger specified with a relative time is triggered at the specified time relative to another script item.
An absolute time trigger is the default. For example, to trigger a script item 10s after the program start, write:
ScriptItem(name='trial_start', time=10, actions=[info_text.activate])
To specify a relative time trigger, set time_type to “rel” and provide the name of the reference event with rel_name. For example, to trigger a script item 5s to 10s after the last “trial_end” event, write:
ScriptItem(name='trial_start', time=np.random.uniform(5, 10), time_type='rel', rel_name='trial_end', actions=[info_text.activate\])]
Signal Triggers¶
A script item can wait for trigger signals generated by graphical objects. To trigger a script item with a signal, set wait_for_signal to the respective signal. The possible trigger signals are listed in Supported Graphical Objects. For example:
ScriptItem(name='trial_end', wait_for_signal=GO.Countdown.COUNTDOWN_FINISHED, actions=[])
LSL Event Marker Triggers¶
A script item can be triggered by an external LSL event marker generated by other programs. To listen to an LSL event marker stream, call the method listenForLSLMarkers in the paradigm file. For example, to listen for markers in the first channel of the stream streamA:
self.listenForLSLMarkers("streamA", lsl_marker_channel=0)
The streams are expected to be event marker streams, i.e., they have an irregular sampling rate. The markers are usually variable-length ASCII strings.
After setting up an LSL event listener, you can trigger script items on LSL events by specifying the marker with the parameter wait_for_lsl_marker. For example:
ScriptItem(name='trial_end', wait_for_lsl_marker=”target_reached”, actions=[])
The data type of the specified marker must correspond to the data type of the LSL stream (usually a string).
Command Line Variables¶
YAGA supports three general-purpose variables, which can be specified when you start YAGA.py (var1, var2, var3). For example:
python yaga.py --paradigm YOURPARADIGM --var1 someStringOrNumber
The general purpose variables are accessible in the paradigm file:
class Paradigm(ParadigmBase):
def __init__(self, paradigm_variables):
var1 = paradigm_variables['var1']
Moreover, one can specify the subject code (string) and session number (integer) with the command line parameters --subject and --session, respectively. These parameters are accessible as paradigm_variables['subject'] and paradigm_variables['session'] in the paradigm file.
Graphical Objects¶
Graphical objects are objects shown on the computer screen, such as text or feedback bars. They can be controllable with LSL streams, and can feature animations.
To use graphical objects, one needs to import the graphic_objects module first:
import yaga_modules.graphic_objects as GO
A graphical object must be created in the initialisation method of the paradigm file and registered to YAGA. For example, to create a text object:
text = self.registerObject(GO.Text('prepare for task'))
Base Methods¶
All graphical objects support the following methods which can be specified as actions in a ScriptItem:
| method | parameters | value type | description |
|---|---|---|---|
| activate | - | - | show object on screen |
| deactivate | - | - | hide object on screen |
The position, scale, angle, and color of graphical objects can be changed with the following methods:
| method | parameters | value types | description |
|---|---|---|---|
| updatePos | pos_x, pos_y, depth | double, double, integer | set x/y position and depth |
| updateScale | scale_x, scale_y | double, double | set x/y size |
| updateRot | angle | double | rotate object (in degrees) |
| updateColor | color | string or 3-tuple | set color |
Screen Coordinates¶
The screen coordinates are independent of the monitor resolution. The centre of the screen corresponds to the x/y position 0. The bottom edge corresponds to a y position of -1. The top edge corresponds to an y position of 1. The coordinates of the left and right edges depend on the screen ratio. With a typical screen ratio of 16:9, the left and right edges have the x coordinate -1.78 and 1.78, respectively ((ytop - ycenter) x 16/9).
Depth of Objects¶
The depth parameter must be an integer value and specifies the depth position of objects. Objects with a smaller depth value are in front of objects with a higher depth value (the default depth is 0).
Colour of Objects¶
Many objects allow changing their colour. The colour must be specified as an RGB tuple with values from (0, 0, 0) to (255, 255, 255) or as a colour name (string). The following color names are supported: black, white, red, lime, blue, yellow, cyan, magenta, silver, gray, maroon, olive, green, purple, teal, navy, gold, orange, and darkorange.
Animated Objects¶
Some graphical objects support animations. Animations can be started or stopped using an action (startAnimation, stopAnimation) in a ScriptItem. Furthermore, animations can generate signals which can be used as triggers for ScriptItems (e.g., trigger actions at the end of an animation).
Supported Graphical Objects¶
Typical graphical elements used in neuroscience experiments are implemented. The Python classes representing these elements are described in the following.
Image Class¶
Loads an image file from the hard disk and displays it on the computer screen. The image must be in the subfolder resources/images. All common image formats are supported.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| file | string | file name of image |
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
Ball Class¶
Displays a filled circle on the computer screen.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| color | string or 3-tuple | fill color |
Box Class¶
Displays a filled box on the computer screen.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
| color | string or 3-tuple | fill color |
Cross Class¶
Displays a cross on the screen.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| line_width | double | cross line size |
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
| color | string or 3-tuple | cross color |
Text Class¶
Displays a text box on the computer screen. Optionally, a value can be read from an LSL stream and displayed on the screen as a rounded number.
To read a number from an LSL stream, call the method controlStateWithLSLStream. For example:
text = self.registerObject(GO.Text())
text.controlStateWithLSLStream(“the-stream”, channels=[1])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| text | string | text to display |
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
| color | string or 3-tuple | text color |
| background_color | string or 3-tuple | text box background color |
| frame_color | string or 3-tuple | text box frame color |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateColor | color | string or 3-tuple | set text color |
| updateBackgroundColor | color | string or 3-tuple | set text box background color |
| updateFrameColor | color | string or 3-tuple | set text box frame color |
| updateText | text | string | set text |
RandomNumber Class¶
Display a random number in a text box on the computer screen. The number is updated after each call of the method activate according to a discrete normal distribution.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| interval | list of 2 integers | a random number is drawn from the specified closed interval |
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
| color | string or 3-tuple | text color |
| background_color | string or 3-tuple | text box background color; set to None to deactivate |
| frame_color | string or 3-tuple | text box frame color; set to None to deactivate |
Countdown Class¶
An animated countdown is displayed in a text box on the computer screen. The countdown is restarted after each call of the method activate.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| counter_start | integer | initial value of the countdown |
| counter_stop | integer | final value of the countdown |
| counter_interval | double | interval time in seconds |
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| scale_x | double | horizontal size |
| scale_y | double | vertical size |
| angle | double | rotation angle |
| color | string or 3-tuple | text color |
| background_color | string or 3-tuple | text box background color |
| frame_color | string or 3-tuple | text box frame color |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| Countdown.COUNTDOWN_FINISHED | elicited at the end of the countdown |
Bar Class¶
Displays a bar for 1D feedback on the screen. The fill level of the bar is controlled with a channel from an LSL stream. Optionally, a horizontal target line can be specified. The target line position can be fixed at object initialization, updated with a call of updateTargetValue, or controlled with an LSL stream. If the target is controlled with an LSL stream, the target position can be continuously updated (see parameter target_online_control) or updated at discrete time points with the action updateTargetValueFromLSLStream.
To set up the control of the fill level, call the method controlStateWithLSLStream and specify one control channel. For example:
bar = self.registerObject(GO.Bar(pos_x=0, pos_y=0))
bar.controlStateWithLSLStream(“the-stream”, channels=[0])
To optionally control the vertical target position continuously with the same LSL stream, set target_online_control to true, and specify a second control channel. For example:
bar = self.registerObject(GO.Bar(pos_x=0, pos_y=0, target_online_control=true))
bar.controlStateWithLSLStream(“the-stream”, channels=[0, 1])
The vertical target position can also be controlled with a separate LSL stream by using controlStateWithLSLStreams. The second specified channel then refers to the second stream. For example:
bar = self.registerObject(GO.Bar(pos_x=0, pos_y=0, target_online_control=true))
bar.controlStateWithLSLStream([“stream-A”, "stream-B"], channels=[0, 1])
Instead of continuously updating the target position from an LSL stream, the target position can also be updated when a script item is triggered. This can be achieved by specifying the action updateTargetValueFromLSLStream in the respective ScriptItem objects. Note that you still need to set up the control with controlStateWithLSLStream or controlStateWithLSLStreams. For example:
bar = self.registerObject(GO.Bar(pos_x=0, pos_y=0, target_online_control=false))
bar.controlStateWithLSLStream(“the-stream”, channels=[0, 1])
[...]
ScriptItem(name='update_bar', time=10, actions=[bar.updateTargetValueFromLSLStream])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| bar_width | double | width of the bar |
| bar_height | double | height of the bar when fully raised |
| frame_width | double | thickness of the frame around the bar |
| target_width | double | width of the target line |
| target_height | double | height of the target line |
| bar_color | string or 3-tuple | color of the bar |
| frame_color | string or 3-tuple | color of the bar frame |
| target_color | string or 3-tuple | color of the target line |
| low_value | double | LSL value corresponding to an empty bar |
| high_value | double | LSL value corresponding to a fully filled bar |
| target_value | double | - target value or fill level (low_value <= target <= high_value) - if set to None, no target will be displayed |
| target_online_control | bool | - if true control the vertical target position continuously with an LSL stream - a second control channel for the target position must be specified when calling controlStateWithLSLStream or controlStateWithLSLStreams - if false, the target position can be updated with the action updateTargetValueFromLSLStream |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| updateTargetValueFromLSLStream | - | - | - update the target value with the most recent value from an LSL stream - it is necessary to call controlStateWithLSLStream(s) before |
BarWithRampTarget Class¶
Displays a feedback [bar]](paradigm_scripting.md#bar-class) with a ramp animation of the target. The target will transit through the following phases:
- pre phase: target line is at the initial position
- ramp up phase: target line moves upwards
- hold phase: target line is at the hold position
- ramp down phase: target line moves downwards
- post phase: target line is back to the initial position
An additional horizontal line indicating the final ramp target will be displayed (target information) if the target animation is configured to stop below the top of the bar.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| bar_width | double | width of the bar |
| bar_height | double | height of the bar when fully raised |
| frame_width | double | thickness of the frame around the bar |
| target_width | double | width of the target line |
| target_height | double | height of the target line |
| bar_color | string or 3-tuple | color of the bar |
| frame_color | string or 3-tuple | color of the bar frame |
| target_color | string or 3-tuple | color of the target line |
| target_info_color | string or 3-tuple | color of the target information |
| pre_phase_duration | double | time duration of the pre phase |
| ramp_up_phase_duration | double | time duration of the ramp up phase |
| hold_phase_duration | double | time duration of the hold phase |
| ramp_down_phase_duration | double | time duration of the ramp down phase |
| post_phase_duration | double | time duration of the post phase |
| low_value | double | LSL value corresponding to a fully contracted bar |
| high_value | double | LSL value corresponding to a fully raised bar |
| start_value | double | the initial position of the target |
| ramp_value | double | the position of the target in the hold phase |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| startAnimation | - | - | start ramp animation |
| stopAnimation | - | - | stop ramp animation |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| BarWithRampTarget.BAR_FINISHED | elicited after the end of the post phase |
BarWithSinusTarget Class¶
Displays a feedback bar with a sinus oscillation animation of the target. The target will transit through the following phases:
- pre phase: target line is at the initial position
- ramp up phase: target line moves upwards
- sinus phase: target line oscillates up and down
- ramp down phase: target line moves downwards
- post phase: target line is back to the initial position
An additional horizontal line indicating the oscillation center will be displayed (target information) if the oscillation center is configured to be below the top of the bar.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| bar_width | double | width of the bar |
| bar_height | double | height of the bar when fully raised |
| frame_width | double | thickness of the frame around the bar |
| target_width | double | width of the target line |
| target_height | double | height of the target line |
| bar_color | string or 3-tuple | color of the bar |
| frame_color | string or 3-tuple | color of the bar frame |
| target_color | string or 3-tuple | color of the target line |
| target_info_color | string or 3-tuple | color of the target information |
| pre_phase_duration | double | time duration of the pre phase |
| ramp_up_phase_duration | double | time duration of the ramp up phase |
| sinus_phase_duration | double | time duration of the oscillation phase |
| sinus_frequency | double | oscillation frequency in Hz |
| sinus_amplitude | double | oscillation amplitude |
| ramp_down_phase_duration | double | time duration of the ramp down phase |
| post_phase_duration | double | time duration of the post phase |
| low_value | double | LSL value corresponding to a fully contracted bar |
| high_value | double | LSL value corresponding to a fully raised bar |
| start_value | double | the initial position of the target |
| ramp_value | double | the center position of the sinus oscillations |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| startAnimation | - | - | start ramp animation |
| stopAnimation | - | - | stop ramp animation |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| BarWithSinusTarget.BAR_FINISHED | elicited after the end of the post phase |
Arrow Class¶
Displays an arrow for 2D feedback on the computer screen. The endpoint of the arrow is controlled with two channels from an LSL stream. Optionally, a target can be visualised as a disk above the arrow's origin. The vertical target position can be fixed at object initialization, updated with a call of updateTargetValue, or controlled with an LSL stream. If the target is controlled with an LSL stream, the target position can be continuously updated (see parameter target_online_control) or updated at discrete time points with the action updateTargetValueFromLSLStream.
To set up the control of the arrow’s endpoint, call the method controlStateWithLSLStream and specify the LSL stream and two control channels. For example:
arrow = self.registerObject(GO.Arrow(pos_x=0, pos_y=0))
arrow.controlStateWithLSLStream(“the-stream”, channels=[0, 1])
To optionally control the vertical target position continuously with the same LSL stream, set target_online_control to true, and specify a third control channel. For example:
arrow = self.registerObject(GO.Arrow(pos_x=0, pos_y=0, target_online_control=true))
arrow.controlStateWithLSLStream(“the-stream”, channels=[0, 1, 2])
The vertical target position can also be controlled with a separate LSL stream by using controlStateWithLSLStreams. The third specified channel then refers to the second stream. For example:
arrow = self.registerObject(GO.Arrow(pos_x=0, pos_y=0, target_online_control=true))
arrow.controlStateWithLSLStreams([“stream-A”, "stream-B"], channels=[0, 1, 0])
Instead of continuously updating the target position from an LSL stream, the target position can also be updated when a script item is triggered. This can be achieved by specifying the action updateTargetValueFromLSLStream in the respective ScriptItem objects. Note that you still need to set up the control with controlStateWithLSLStream or controlStateWithLSLStreams. For example:
arrow = self.registerObject(GO.Arrow(pos_x=0, pos_y=0, target_online_control=false))
arrow.controlStateWithLSLStream(“the-stream”, channels=[0, 1, 2])
[...]
ScriptItem(name='update_arrow', time=10, actions=[arrow.updateTargetValueFromLSLStream])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| angle | double | rotation angle of the arrow |
| arrow_length | double | length of the arrow when fully extended |
| line_width | double | width of the arrow |
| head_size | double | head size of the arrow |
| target_size | double | size of the target |
| arrow_color | string or 3-tuple | color of the arrow |
| target_color | string or 3-tuple | color of the target |
| low_value | double | LSL value corresponding to a fully contracted arrow |
| high_value | double | LSL value corresponding to a fully extended arrow |
| target_value | double | - target value (vertical target position) - if set to None, no target will be displayed |
| target_online_control | bool | - if true, control the vertical target position continuously with an LSL stream - a third control channel for the target position must be specified when calling controlStateWithLSLStream or controlStateWithLSLStreams - if false, the target position can be updated with the action updateTargetValueFromLSLStream |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| updateTargetValueFromLSLStream | - | - | - update the target value with the most recent value from an LSL stream - it is necessary to call controlStateWithLSLStream(s)* before |
ArrowWithRampTarget Class¶
Displays a feedback arrow with a ramp animation of the target disk. The target will transit through the following phases:
- pre phase: target disk is at the initial position
- ramp up phase: target disk moves upwards
- hold phase: target disk is at the final position
- ramp down phase: target disk moves downwards
- post phase: target disk is back to the initial position
An additional disk indicating the final ramp target position will be displayed (target information).
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| angle | double | rotation angle |
| arrow_length | double | length of the arrow when fully extended |
| line_width | double | width of the arrow |
| head_size | double | head size of the arrow |
| target_size | double | size of the target |
| target_info_size | double | size of the target information |
| arrow_color | string or 3-tuple | color of the arrow |
| target_color | string or 3-tuple | color of the target |
| target_info_color | string or 3-tuple | color of the target information |
| pre_phase_duration | double | time duration of the pre phase |
| ramp_up_phase_duration | double | time duration of the ramp up phase |
| hold_phase_duration | double | time duration of the hold phase |
| ramp_down_phase_duration | double | time duration of the ramp down phase |
| post_phase_duration | double | time duration of the post phase |
| low_value | double | LSL value corresponding to a fully contracted arrow |
| high_value | double | LSL value corresponding to a fully extended arrow |
| start_value | double | the initial position of the target |
| ramp_value | double | - the center position of the sinus oscillation - when this value is lower than the high_value, an additional hold target will be displayed |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| startAnimation | - | - | start ramp animation |
| stopAnimation | - | - | stop ramp animation |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| ArrowWithRampTarget.BAR_ARROW_FINISHED | elicited after the end of the post phase |
ArrowWithSinusTarget Class¶
Displays a feedback arrow Class with a ramp animation of the target disk. The target will transit through the following phases:
- pre phase: target disk is at the initial position
- ramp up phase: target disk moves upwards
- sinus phase: target disk is oscillating up & down
- ramp down phase: target disk moves downwards
- post phase: target disk is back to the initial position
An additional disk indicating the position of the oscillation centre will be displayed (target information).
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| angle | double | rotation angle |
| arrow_length | double | length of the arrow when fully extended |
| line_width | double | width of the arrow |
| head_size | double | head size of the arrow |
| target_size | double | size of the target |
| target_info_size | double | size of the target information |
| arrow_color | string or 3-tuple | color of the arrow |
| target_color | string or 3-tuple | color of the target |
| target_info_color | string or 3-tuple | color of the target information |
| pre_phase_duration | double | time duration of the pre phase |
| ramp_up_phase_duration | double | time duration of the ramp up phase |
| sinus_phase_duration | double | time duration of the oscillation phase |
| sinus_frequency | double | oscillation frequency in Hz |
| sinus_amplitude | double | oscillation amplitude |
| ramp_down_phase_duration | double | time duration of the ramp down phase |
| post_phase_duration | double | time duration of the post phase |
| low_value | double | LSL value corresponding to a fully contracted arrow |
| high_value | double | LSL value corresponding to a fully extended arrow |
| start_value | double | the initial position of the target |
| ramp_value | double | - the center position of the sinus oscillation - when this value is lower than the high_value, an additional hold target will be displayed |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| updateTargetValue | target_value | double | update the target value |
| startAnimation | - | - | start ramp animation |
| stopAnimation | - | - | stop ramp animation |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| ArrowWithSinusTarget.BAR_ARROW_FINISHED | elicited after the end of the post phase |
SpikeVis Class¶
Visualise discrete events, such as spikes, with flashing feedback disks. A SpikeVis object can be used as a feedback element for, e.g., the onset and termination of spike trains.
SpikeVis supports multiple feedback disks and must have as many LSL control channels as feedback disks. The LSL control channels are expected to provide 0/1 values. To set up the control, call the method controlStateWithLSLStream. For example:
spikes = self.registerObject(GO.SpikeVis(pos_x=0, pos_y=0, number_of_units=3))
spikes.controlStateWithLSLStream(“the-stream”, channels=[0, 1, 2])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| number_of_units | integer | number of feedback disks; must correspond to the number of LSL control channels |
| size | double | size of a feedback disk |
| spacing | double | distance between feedback disks |
| flash_duration | double | time in seconds a feedback disk flashes (activates) when a “1” is read from the corresponding LSL control channel |
| active_color | string or 3-tuple | color of the feedback disk when inactive |
| inactive_color | string or 3-tuple | color of the feedback disk when it is active |
ReachTargets Class¶
Graphical object to implement target reaching experiments. It displays one or more disk targets which must be reached and held with a 2D cursor. The targets will be arranged in a circle around the centre. The cursor is controlled with 2 LSL channels.
To set up the control, call the method controlStateWithLSLStream. For example:
targetreaching = self.registerObject(GO.ReachTargets(pos_x=0, pos_y=0, number_of_targets=3))
targetreaching.controlStateWithLSLStream(“the-stream”, channels=[0,1])
An optional start target can be specified. Such a start target will be placed in the centre and must be reached before the actual target.
The cursor must stay within the correct target area for a certain period (dwell time) to generate a target-reached signal. This target-reached signal can then be used to trigger a script item.
To activate the ReachTargets object at second 10 and set the active (current) target to target 1, create a script item such as:
ScriptItem(name='trial_start', time=10, actions=[targets.activate, partial(targets.setActiveTarget, 1)])
To end the trial when the target reached signal is generated or a 30s timeout occurs, create a script item such as:
ScriptItem(name='trial_end', time=30, time_type='rel', rel_name='trial_start', wait_for_signal=GO.ReachTargets.TARGET_REACHED, actions=[targets.deactivate])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pox_x | double | horizontal position |
| pos_y | double | vertical position |
| depth | integer | depth position |
| radius | double | radius of the targets around the center |
| number_of_targets | integer | number of targets |
| dwell_time | double | dwell or hold time of targets |
| start_target | bool | show a start target |
| target_rotation | double | rotate targets (in degrees) |
| target_size | double | target size |
| cursor_size | double | cursor size |
| target_active_color | string or 3-tuple | color of the active (current) target disk |
| target_inactive_color | string or 3-tuple | color of the inactivate target disks |
| target_reached_color | string or 3-tuple | color of the target disk when the cursor is within the target area |
| cursor_color | string or 3-tuple | color of the cursor |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| setActiveTarget | selected_target_idx | integer | index of the active (current) target |
ScriptItem Trigger Signals:
| signal | description |
|---|---|
| ReachTargets.START_TARGET_REACHED | optional start target has been reached |
| ReachTargets.TARGET_REACHED | active (current) target has been reached |
Auditory Objects¶
Auditory objects generate sounds. They can be controllable with LSL streams.
To use auditory objects, one needs to import the audio_objects module first:
import yaga_modules.audio_objects as AO
An auditory object must be created in the initialisation method of the paradigm file and registered to YAGA. For example, to create a beep object:
beep = self.registerObject(AO.Beep())
The Python classes representing the auditory objects are described in the following.
Beep Class¶
Generates a beep sound, e.g., to attract the attention of the subject.
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| beep_frequency | double | frequency of the beep in Hz |
| beep_amplitude | double | sound level |
| beep_duration | double | duration of a beep in seconds |
| beep_channels | string | output channels: "both”, “left”, or “right” |
| delay | double | - delay the presentation of the beep - use a delay of 0.2s or more to minimise jitter |
ScriptItem Actions:
| method | parameters | value types | description |
|---|---|---|---|
| beep | - | - | generate beep |
SpikeSound Class¶
Generate a feedback signal for discrete events like spikes. SpikeSound supports multiple event or spike signals which can be associated with a specific sound frequency or output channel. Optionally, the sound frequencies can be dynamically modulated by the instantaneous firing rates. The events are read from an LSL stream which is expected to provide 0/1 values. To set up the LSL control call the method controlStateWithLSLStream. For example, to generate sounds for two spike signals using different sound frequencies and output channels:
spikes = self.registerObject(AO.SpikeSound(beep_frequencies=\[800, 2000\], beep_channels=\['left', 'right'\]))
spikes.controlStateWithLSLStream(“the-stream”, channels=\[0, 1\])
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| beep_frequencies | list of doubles | - list of frequencies in Hz - must have the same length as number of LSL channels - values must be between 100 Hz and 10 kHz |
| beep_channels | list of strings | - list of output channels (“both”, “left”, or “right”) - must have the same length as number of LSL channels |
| beep_duration | double | duration of each spike sound |
| downsample | integer | downsample factor; generate a spike sound for every x^th^ spike |
| dynamic_frq | bool | if true, modulate the sound frequency by the instantaneous firing rate |
| dynamic_frq_factor | integer | the final sound frequency is the base frequency (from beep_frequencies) plus the instantaneous firing rate multiplied by this factor |
| dynamic_max_frq | double | limit the smoothed instantaneous firing rate to this frequency |
| dynamic_mov_avg | integer | - calculate the instantaneous firing rate as the inverse of the interspike interval smoothed with a moving average window of length dynamic_mov_avg - set to None if exponential smoothing is used |
| dynamic_exp_avg_alpha | double | - calculate the instantaneous firing rate as the inversive of the interspike interval smoothed with exponential smoothing using a smoothing factor of dynamic_exp_avg_alpha - set to None if the moving average is used |
Online Control of Objects with LSL streams¶
Parameters of graphical and auditory objects can be controlled with LSL streams. The LSL streams can be read directly or processed online (scaling, filtering, etc.). The parameter updates are synchronous to the screen refresh rate.
Control of Graphical Objects¶
The position, scale, colour or state of graphical objects can be controlled by one or more LSL streams. For that, call the respective control method listed below.
Methods for controlling a property with a single LSL stream:
| method | parameters | value types | description |
|---|---|---|---|
| controlPosWithLSLStream | lsl_stream_name, channels, aggregation_mode | string, list of integers, string | control the x/y position of an object with two channels from a single LSL stream |
| controlScaleWithLSLStream | lsl_stream_name, channels, aggregation_mode | string, list of integers, string | control the x/y size of an object with two channels from a single LSL stream |
| controlColorWithLSLStream | lsl_stream_name, channel, aggregation_mode, neg_color, pos_color, neutral_color |
string, integer, string, string, string, string |
- interpolate between colors using one channel from a single LSL stream: when the value of the LSL channel is in the interval [-1, 0): interpolate between the negative and neutral color when the value of the LSL channel is in the interval [0, 1]: interpolate between the neutral and positive color - colors must be specified by their name (string) |
| controlStateWithLSLStream | lsl_stream_name, channels, aggregation_mode | string, list of integers, string | state updates are specific to the graphical object |
- the channels parameter of controlPosWithLSLStream and controlScaleWithLSLStream must be a list comprising two channel indices
- the channel parameter of controlColorWithLSLStream must be a channel index
- the channels parameter of controlStateWithLSLStream must be a list with one or more channel indices (the number of channels depends on the actual graphical object)
- for an explanation of the aggregation_mode parameter see Aggregation Mode
- note that channel indexing starts with a 0 in Python
Methods for controlling a property with multiple LSL stream:
| method | parameters | value types | description |
|---|---|---|---|
| controlPosWithLSLStreams | lsl_stream_names, channels, aggregation_mode | list of strings, list of integers, string | control the x/y position of an object with two channels from two separate LSL streams |
| controlScaleWithLSLStreams | lsl_stream_names, channels, aggregation_mode | list of strings, list of integers, string | controls the x/y size of an object with two channels from two separate LSL streams |
| controlStateWithLSLStreams | lsl_stream_names, channels, aggregation_mode | list of strings, list of integers, string | currently only supported by Bar Class and Arrow Class |
Control of Auditory Objects¶
Auditory objects support the following methods to set up LSL control.
Methods for controlling a property with one or more LSL stream:
| method | parameters | value types | description |
|---|---|---|---|
| controlWithLSLStream | stream name, channels, aggregation_mode | string, list of integers, string | currently only supported by SpikeSound Class |
| controlWithLSLStreams | stream names, channels, aggregation_mode | list of string, list of integers, string | not yet supported by any auditory object |
Aggregation Mode¶
The LSL streams used to control the parameters of graphical or auditory objects are read and processed just before a screen refresh. All updates are therefore synchronised with the screen refresh rate, which is often 60 Hz.
- when the sampling frequency of an LSL stream is lower than the screen refresh rate, the LSL stream is upsampled to the screen frequency using a sample & hold strategy
- when the sampling frequency of an LSL stream is higher than the screen frequency, the LSL stream is downsampled strategy specified by aggregation_mode is used
The following sample aggregation strategies are supported:
| aggregation mode value | description |
|---|---|
| last (default) | the most recent LSL sample is used, the rest is discarded |
| sum | the LSL samples since the last read-out are summed up |
| mean | the LSL samples since the last read-out are averaged |
Signal Processing¶
LSL streams can be processed when used to control parameters of graphical or auditory objects. YAGA signal processing is applied to the original LSL stream, i.e., before any re-sampling to the screen refresh rate (see Aggregation Mode).
To use signal processing on LSL streams, import the signal_processing module:
import yaga_modules.signal_processing as SP
Signal processing objects can be added to the signal processing pipeline by calling the method addSignalProcessingToLSLStream. In the following example, a feedback bar is set to be controlled by an LSL stream and the LSL stream is filtered with a Butterworth low-pass filter and scaled:
bar = self.registerObject(GO.Bar())
bar.controlStateWithLSLStream('the-stream', channels=[10])
butter = SP.ButterFilter(4, 5)
scaler = SP.Scaler(scale=2)
bar.addSignalProcessingToLSLStream(butter, channels=[10])
bar.addSignalProcessingToLSLStream(scaler, channels=[10])
The first parameter of addSignalProcessingToLSLStream is the signal processing object. The second parameter is the channel list. The number of supported channels depends on the concrete signal processing object.
Important note: Make sure that signal processing objects which have a state (e.g., ButterFilter) exist only in one signal processing pipeline. If you need the same signal processing method applied to another LSL stream, create a new signal processing object.
The following signal processing objects are supported by YAGA.
Constant¶
Set channels to a constant value.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| value | double | set signals to the given value |
CopyChannel¶
Copy values from one channel to another channel.
Number of supported channels: two or more
Operates channel-wise: no
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| channel_in | integer | index of the source channel |
| channel_out | integer | index of the destination channel |
ButterFilter¶
Filters stream with a Butterworth filter.
Number of supported channels: one or more
Operates channel-wise: Yes
Stateful: Yes
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| order | integer | order of the filter |
| cutoff_frqs | double or list of doubles | critical frequency or frequencies: - low/highpass: a scalar - bandpass/stop: a 2-element list |
| filter_type | string | filter type, possible values: lowpass, highpass, bandpass, bandstop |
MovAvg¶
Moving average filter.
Number of supported channels: one or more
Operates channel-wise: Yes
Stateful: Yes
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| window_length | integer | length of the moving window in seconds |
Angle¶
Calculates the angle in radians between the x-axis and the point given by (x,y). The first channel represents the x-coordinate, and the second channel represents the y-coordinate.
Number of supported channels: two
Operates channel-wise: no
Stateful: no
Integrate¶
Integration over samples.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: yes
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| factor | double | multiply samples by a factor before integration |
Diff¶
Calculates the difference between two consecutive samples.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: yes
Sum¶
Calculates the sum over channels.
Number of supported channels: two or more
Operates channel-wise: no
Stateful: no
Mean¶
Calculates the average over channels.
Number of supported channels: two or more
Operates channel-wise: no
Stateful: no
StdDev¶
Calculates the standard deviation over channels.
Number of supported channels: two or more
Operates channel-wise: no
Stateful: no
Scaler¶
Multiplies channels by a scaling factor and adds offsets before and after scaling.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| scale | double | scale samples by this factor |
| pre_offset | double | add pre_offset before scaling |
| post_offset | double | add post_offset after scaling |
LinearMap¶
Linearly maps signals from the interval [in_val1, in_val2] to the interval [out_val1, out_val2].
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type |
|---|---|
| in_val1 | double |
| in_val2 | double |
| out_val1 | double |
| out_val2 | double |
Limit¶
Limit signals to minimum and maximum values.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| min_val | double | minimum value |
| max_val | double | maximum value |
Abs¶
Calculates the absolute value.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Power¶
Raises signals to the given power.
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| exponent | double | exponent (power) |
EuclidNorm¶
Calculate the Euclidean norm over channels.
Number of supported channels: one or more
Operates channel-wise: no
Stateful: no
MaxEuclidNormalizationXDF¶
Normalise channels by the maximum Euclidean norm found from the data channels in the specified XDF file (e.g., force normalisation).
Find maximum:
- load the XDF file
- apply a median window filter to the selected channels of the data stream
- subtract a common or channel-specific offset
- calculate the 2-norm over all selected channels
- find the maximum norm value over all trials
Online processing:
- subtract offset from signals
- divide by the found maximum norm
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: no
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| xdf_file | string | XDF file to load |
| data_stream_name | string | name of data stream over which the Euclidean norm is calculated (e.g., 2D force data) |
| marker_stream_name | string | name of the event stream |
| start_marker | string | name of a trial start event |
| data_stream_channels | list of integers | channel indices of the data stream over which the Euclidean norm is calculated (if None, use all channels) |
| offset | double or list of double | - subtract an offset value before calculating the Euclidean norm - to specify channel-specific offsets, specify offsets as a list with one offset value per channel - set to None to subtract the minimum of each channel |
| filter_window_length | integer | filter data stream with a median filter using the specified window size in samples (must be odd) |
MaxAvgPowerNormalizationXDF¶
Normalise channels by the maximum power found from the data channels in the specified XDF file (e.g., EMG power normalisation).
Find maximum:
- load the XDF file
- apply a Butterworth bandpass filter to the selected channels of the data stream (pre-filter)
- calculate the signal power of all selected channels
- apply a moving average filter (post-filter)
- calculate the median signal power over all selected channels
- find the 90% percentile of the power over all trials
Online processing:
- apply the pre-filter (bandpass)
- calculate the signal power
- apply the post-filter (moving average)
- calculate the median over channels
- divide signals by found 90% percentile of the power
Number of supported channels: one or more
Operates channel-wise: yes
Stateful: yes
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| xdf_file | string | XDF file to load |
| data_stream_name | string | name of data stream over which the Euclidean norm is calculated (e.g., 2D force data) |
| marker_stream_name | string | name of the event stream |
| start_marker | string | name of a trial start event |
| end_marker | string | name of a trial end event |
| data_stream_channels | list of integers | channel indices of the data stream over which the Euclidean norm is calculated (if None, use all channels) |
| prefilter_order | Butterworth bandpass filter order | |
| prefilter_cutoff_frqs | Butterworth bandpass cutoff frequencies | |
| postfilter_win_length | - apply a moving average filter with the specified windows length in samples - set to 1 to deactivate filter |
FlappyBirdController¶
Implements a Flappy Bird style 2D control with discrete events like spikes. As input, a 1D signal comprising 0/1 values is expected (e.g., spiking activity). The generated output is a 2D position signal, which can be used to control the position of other graphical objects.
The control works as follows:
- an input activity increases the position on the currently controlled axis
- a short pause in the input activity switches the control between the x and y axis
- the x and y positions decrease at a constant velocity
Note: The signal processing object expects 2 channels, but only the activity in the first channel is evaluated. The channel values will be replaced by the calculated x/y position.
Number of supported channels: two
Operates channel-wise: no
Stateful: yes
Object Initialisation Parameters:
| parameter | value type | description |
|---|---|---|
| pos_increment | double | each discrete event (spike) increases the position on the currently controlled axis by this value |
| negative_vel | double | decrease the position on both axes by this velocity; however, recent spiking activity prevents the currently controlled axis from this change |
| switch_interval | double | minimum pause interval to switch between x and y axis |
| x_max | double | limit x position to this value |
| y_max | double | limit y position to this value |