Difference between revisions of "Interacting with NIC"

From Neuroelectric's Wiki
Jump to: navigation, search
(Controlling NIC with Matlab: the MatNIC toolkit)
 
(162 intermediate revisions by 8 users not shown)
Line 2: Line 2:
  
 
== About Synchronization: general principles ==
 
== About Synchronization: general principles ==
[[File:Master slave synchronization.png|200px|thumb|left| The Master system's clock is the only one used in the whole system. The master system either gathers the data from the slave systems to provide a single output or sends its clock so the slave systems can provide their outputs using that clock]]
+
[[File:Markers through TCP setting.png|200px|thumb|left| The Master system's clock is the only one used in the whole system. The master system either gathers the data from the slave systems to provide a single output or sends its clock so the slave systems can provide their outputs using that clock]]
 
To coordinate different data sources or events to a single clock is known as synchronization.
 
To coordinate different data sources or events to a single clock is known as synchronization.
  
Line 10: Line 10:
  
 
NIC implements an algorithm that compensates this latency and finds out the offset between the clock from Enobio and the one from the host where NIC runs. When NIC receives information from third-party applications that need to be synchronized with the EEG data, like markers that signal when external events occur, it compensates the timestamp of the received data so it is aligned with the EEG data streaming.
 
NIC implements an algorithm that compensates this latency and finds out the offset between the clock from Enobio and the one from the host where NIC runs. When NIC receives information from third-party applications that need to be synchronized with the EEG data, like markers that signal when external events occur, it compensates the timestamp of the received data so it is aligned with the EEG data streaming.
 +
 +
The relative values of the timestamps recorded in the easy files are corresponding to the unadjusted clock on the Necbox acquisition device. So each sample is separated 2 ms form the previous one according to the necbox clock.
 +
 +
The absolute Unix time is started reading the clock of the PC (the Necbox does not have a realtime clock), so that if you start a recording at 10:00 am, the easy file will start with the corresponding Unix time of 10:00am and then the rest of the data will be added to the 10:00am according to the unadjusted necbox clock.
  
 
In the scenario described above, when NIC collects markers that are sent by other applications, another clock to be synchronized is introduced in the system. This is the clock from the host that sends the markers. NIC provides two ways of gathering such markers. Both of them are described in the following sections. The first one does not provide any synchronization mechanism, this is the reception of markers using a TCP/IP server where the clients connect to and send their markers. This method might be useful when the time synchronization requirements do not need accuracy under the 100 ms.
 
In the scenario described above, when NIC collects markers that are sent by other applications, another clock to be synchronized is introduced in the system. This is the clock from the host that sends the markers. NIC provides two ways of gathering such markers. Both of them are described in the following sections. The first one does not provide any synchronization mechanism, this is the reception of markers using a TCP/IP server where the clients connect to and send their markers. This method might be useful when the time synchronization requirements do not need accuracy under the 100 ms.
Line 18: Line 22:
 
<!--talk about hardware set up to manually determine this deviation and fix it through advanced NIC properties-->
 
<!--talk about hardware set up to manually determine this deviation and fix it through advanced NIC properties-->
  
== Sending Markers to NIC from other software or hardware ==
+
== TCP ==
 +
===Receiving data streams using TCP/IP===
 +
 
 +
TCP (Transmission Control Protocol) is a connection-oriented protocol for transferring data reliably in either direction between a pair of users. The NIC software has a TCP/IP server that streams the EEG data received from Enobio. Up to 5 clients can connect to that server simultaneously in order to receive the EEG data ans perform the desired operations in real time.
 +
 
 +
 
 +
[[File:Markers_through_TCP_setting.png|200px|thumb|left| NIC settings for configuring the sending of markers through the TCP/IP connection]]
 +
 
 +
The software clients that want to receive the EEG data in real time from NIC need to connect to the '''TCP/IP port 1234''' of the host where the NIC software is running. Once the client software is connected to the server, it will receive the EEG data streaming according to the following format:
 +
-------------------------------------------------------------------------------------------
 +
|                    Channel 1                  | ... |            Channel N            |
 +
-------------------------------------------------------------------------------------------
 +
| (MSB) Byte#1 | Byte#2 | Byte#3 | (LSB) Byte#4 | ... | Byte#1 | Byte#2 | Byte#3 | Byte#4 |
 +
-------------------------------------------------------------------------------------------
 +
Each EEG sample is sent as a two-complement 4 byte value. The unit of the EEG sample is nano volts and its range is from -400000000 to +400000000 nV. The most significant byte is sent first. The following code in 'C' shows how to decode the streaming from the received bytes to EEG sample values. The example assumes that the computer architecture is little-endian.
 +
// byte3 = 0xF7, byte2 = 0x8F, byte1 = 0x99, byte0 = 0x61
 +
signed int32_t sample = 0;
 +
sample += byte3;
 +
sample = sample << 8;
 +
sample += byte2;
 +
sample = sample << 8;
 +
sample += byte1;
 +
sample = sample << 8;
 +
sample += byte0;
 +
// sample = -141584031 nV
 +
 
 +
The client will receive first the four bytes from channel 1, then the next four bytes from channel 2 and so on till receiving the four bytes from the last channel of Enobio (8 or 20 depending of the type of Enobio/Starstim NIC handles). Then channel 1 bytes are receiving again.
 +
 
 +
It is also possible to receive the markers that NIC collects using the TCP/IP connection. By enabling this feature from the NIC settings the markers are sent along with the EEG streaming through the same TCP/IP connection. In this case the markers are sent as a four-bytes integer after the last EEG channel. The most significant byte is sent first. The data streaming will have the following format then:
 +
 
 +
-----------------------------------------------------------------------------------------------------------------------------------
 +
|                    Channel 1                  | ... |            Channel N            |              Marker              |
 +
-----------------------------------------------------------------------------------------------------------------------------------
 +
| (MSB) Byte#1 | Byte#2 | Byte#3 | (LSB) Byte#4 | ... | Byte#1 | Byte#2 | Byte#3 | Byte#4 | Byte#1 | Byte#2 | Byte#3 | Byte#4 |
 +
-----------------------------------------------------------------------------------------------------------------------------------
 +
 
 +
 
 +
Those markers correspond to the ones received by NIC from third-party software using [[Interacting_with_NIC#Sending_Markers_using_TCP.2FIP | TCP/IP]] or [[Interacting_with_NIC#Sending_markers_using_LSL | LSL]], the manual markers that can be inserted while recording by pressing the keys from 1 to 9 and the stimulation events in the case of the Starstim device such as start and stop stimulation and when any parameter is changed from MatNIC.
 +
 
 +
===Example:===
 +
 
 
=== Sending Markers using TCP/IP ===
 
=== Sending Markers using TCP/IP ===
 
<!-- talk about the marker server: port number, protocol -->
 
<!-- talk about the marker server: port number, protocol -->
NIC provides a server that other software can connect to in order to send markers. Those received markers are synchronized with the EEG streaming for further analysis.
 
  
This NIC server is running in the '''TCP/IP port 1234'''. Up to five clients can simultaneously send markers to NIC by making a TCP/IP connection to this port.
+
NIC provides a TCP server that other softwares can connect to in order to send markers. Those received markers are synchronized with the EEG streaming and written in the penultimate column of the .easy files for further analysis.
 +
 
 +
You need to create a TCP client inside your external software. The connection of the TCP client to the NIC TCP server is done using the standard TCP/IP connection protocols available in different programming languages. Up to five clients can simultaneously send markers to NIC by making a TCP/IP connection to this port. The TCP clients can be located in the same machine running NIC or they can also be in another machine located in the same network as the machine running NIC.
 +
 
 +
If the TCP client is located in an external machine in the same network as the machine running NIC, you just need the to connect to the IP of the machine running NIC, shown in the right bottom corner of NIC, and the '''TCP/IP port 1234'''. If the TCP client is located at the same machine running NIC, the IP to which you have to connect is the local IP 127.0.0.1 and port 1234. Once the client is connected, you'll see a progress bar in the right bottom corner of NIC showing the connection.
  
 
Once a client is connected it needs to send the following string in order to send a marker:
 
Once a client is connected it needs to send the following string in order to send a marker:
 
  <TRIGGER>XXXX</TRIGGER>
 
  <TRIGGER>XXXX</TRIGGER>
Where XXXX can represent any integer number different from zero (from -2147483647 to +2147483647). This marker will be co-registered in the output files generated by NIC to the corresponding EEG sample. For instance, in the output tabulated text file, the column just after the timestamp one is filled with zeros if no markers are received. When a marker is received its corresponding number is set to that column. See the following example:
+
Where XXXX can represent any integer number different from zero (from -2147483647 to +2147483647). This marker will be co-registered in the output files generated by NIC to the corresponding EEG sample.  
 +
 
 +
 
 +
For instance, in the output tabulated text file, the column just after the timestamp one is filled with zeros if no markers are received. When a marker is received its corresponding number is set to that column. See the following example:
  
 
  ...
 
  ...
Line 39: Line 89:
 
  ...
 
  ...
  
[[File:Markers audio signal.png|200px|thumb|left| Plot of markers received by NIC and an audio signal recorded by Enobio]]
 
Please take as an example [[Media:Matlab Markers Example.zip | ''this'']] Matlab code which connects to NIC to send markers every time a tone is played back through the sound card. If you connected the output of the computer sound card to one of the Enobio electrodes you would be able to see the alignment between the markers and the played tones.
 
  
=== Sending markers using LSL ===
 
<!-- talk about LSL client: string and integer markers -->
 
NIC is compliant with the [https://code.google.com/p/labstreaminglayer/ Lab Streaming Layer (LSL)] protocol so makers can be co-registered with the EEG signal by setting up a LSL marker outlet (see this [https://code.google.com/p/labstreaminglayer/source/browse/LSL/liblsl/examples/C/SendStringMarkersC/SendStringMarkersC.c example]).
 
  
The LSL handles both the networking and time-synchronization isues between the sender and receiver hosts obtaining reliability on order of 1 ms (see the time-synchronization validation [https://code.google.com/p/labstreaminglayer/wiki/TimeSynchronizationValidation tests]).
+
Those markers correspond to the ones received by NIC from third-party software using [[Interacting_with_NIC#Sending_Markers_using_TCP.2FIP | TCP/IP]] or [[Interacting_with_NIC#Sending_markers_using_LSL | LSL]], the manual markers that can be inserted while recording by pressing the keys from 1 to 9 and the stimulation events in the case of the StarStim device such as start and stop stimulation and when any parameter is changed from MatNIC.
 +
 
 +
=== Examples ===
 +
 
 +
Please take as an example a [[Media:RamReplayTCP.zip | Matlab code]] to do the following interactions:
 +
 
 +
-Connects the TCP server
 +
 
 +
-Disconnects the TCP server
  
[[File:NIC LSL settings.png|200px|thumb|left| NIC settings for configuring the reception of markers through LSL]]
+
-Reads the EEG signal for t number of seconds
NIC needs little configuration in order to receive the markers from a LSL outlet present in the local network. When the LSL marker outlet sends integer-type markers only the name of the outlet needs to be configured in NIC.
 
  
Please go to "''EEG Setup -> Settings -> Markers from Lab Streaming Layer''" and set the name that your LSL marker outlet has. NIC will automatically look for a marker outlet with this name and will connect to it. If the outlet sends integer-type integers then no further configuration is needed. All the received makers will be co-registered along with the EEG signal to the output files.
+
-An example to read data with a pause to do some processing
  
In case the outlet sends string-type markers then there are some considerations that have to be taken into account. The LSL outlet sending string-type markers has to format them as XML tags. The following example is taken from the string markers that the Presentation software sends when the LSL extension manager is installed (see the [[Event_Related_Potentials_(ERPs)#Presentation | Working with ERPs]] section). You can see that NIC will decode the string looking for the tag that is configured in the "''EEG Setup -> Settings -> Markers from Lab Streaming Layer''" settings, ''ecode'' in this case. The marker number 37 will be registered at the reception of this string:
+
- Send via TCP a marker
<pevent><etype>Picture</etype>'''<ecode>37</ecode>'''<unc>209.638092041016</unc>test</pevent>
 
  
<!-- talk about TTL hardware triggering -->
+
Please take an example to plot EEG data from the different channels and save the data
  
== Receiving data streams from NIC ==
+
Please take also an example [[Media:Matlab Markers Example.zip | ''this'']] Matlab code which connects to NIC to send markers every time a tone is played back through the sound card. If you connected the output of the computer sound card to one of the Enobio electrodes you would be able to see the alignment between the markers and the played tones.
=== Receiving data streams using TCP/IP ===
 
The NIC software has a TCP/IP server that streams the EEG data received from Enobio. Up to 5 clients can connect to that server simultaneously in order to receive the EEG data ans perform the desired operations in real time.
 
  
The software clients that want to receive the EEG data in real time from NIC need to connect to the '''TCP/IP port 1234''' of the host where the NIC software is running. Once the client software is connected to the server, it will receive the EEG data streaming according to the following format:
+
The following link [http://wiki.neuroelectrics.com/images/7/76/Eprime2NIC_example.zip eprime2NIC_example.zip] also contains an example on how to send TCP markers from E-Prime to NIC.
-------------------------------------------------------------------------------------------
 
|                    Channel 1                  | ... |            Channel N            |
 
-------------------------------------------------------------------------------------------
 
| (MSB) Byte#1 | Byte#2 | Byte#3 | (LSB) Byte#4 | ... | Byte#1 | Byte#2 | Byte#3 | Byte#4 |
 
-------------------------------------------------------------------------------------------
 
Each EEG sample is sent as a two-complement 4 byte value. The unit of the EEG sample is nano volts and its range is from -400000000 to +400000000 nV. The most significant byte is sent first. The following code in 'C' shows how to decode the streaming from the received bytes to EEG sample values. The example assumes that the computer architecture is little-endian.
 
// byte3 = 0xFF, byte2 = 0x8F, byte1 = 0x99, byte0 = 0x61
 
signed int32_t sample = 0;
 
sample += byte3;
 
sample = sample << 8;
 
sample += byte2;
 
sample = sample << 8;
 
sample += byte1;
 
sample = sample << 8;
 
sample += byte0;
 
// sample = -141584031 nV
 
  
The client will receive first the four bytes from channel 1, then the next four bytes from channel 2 and so on till receiving the four bytes from the last channel of Enobio (8 or 20 depending of the type of Enobio/Starstim NIC handles). Then channel 1 bytes are receiving again.
+
== LSL ==
  
The following links are a Java and a Matlab example clients that connects to NIC and receive the EEG streaming: [[Media:Java_TCP_Enobio_Client.zip | Java client]], [[Media:Matlab_TCP_Enobio_Client.zip | Matlab client]]
 
  
=== Receiving data streams using LSL ===
+
=== Receiving and sending data streams using LSL ===
 
<!-- talk about receiving the data stream using the LSL -->
 
<!-- talk about receiving the data stream using the LSL -->
  
NIC streams the received EEG data from Enobio usign the Lab Streaming Layer. NIC creates a LSL outlet with the following settings:
+
NIC streams the received EEG data from Enobio and Starstim using the Lab Streaming Layer. NIC creates a LSL outlet with the following settings:
  Name: Enobio
+
  Name: (choosen in NIC EEG settings)
 
  Type: EEG
 
  Type: EEG
  Channel count: 8 or 20 depending of the Enobio NIC handles
+
  Channel count: 8, 20 or 32 depending of the Enobio NIC handles
 
  Nominal sample rate: 500
 
  Nominal sample rate: 500
 
  Channel format: float_32
 
  Channel format: float_32
  Unique source ID: The Enobio type plus its mac address
+
  Unique source ID: The Enobio/Starstim type plus its mac address
An LSL client software needs to connect to this outlet in order to receive the EEG streaming data. The received values are expressed in nanovolts and its range is from -400000000 to +400000000.
+
An LSL client software needs to connect to this outlet in order to receive the EEG streaming data. The received values are expressed in microvolts and its range is from -400000 to +400000.
  
 
Using LSL is possible to access to the accelerometer data too. The outlet the LSL clients need to connect to has the following settings:
 
Using LSL is possible to access to the accelerometer data too. The outlet the LSL clients need to connect to has the following settings:
  Name: Enobio
+
  Name: (choosen in NIC EEG settings)
 
  Type: Accelerometer
 
  Type: Accelerometer
 
  Channel count: 3
 
  Channel count: 3
 
  Nominal sample rate: 100
 
  Nominal sample rate: 100
 
  Channel format: float_32
 
  Channel format: float_32
  Unique source ID: The Enobio type plus its mac address plus the "Acc" string
+
  Unique source ID: The device type plus its mac address plus the "Acc" string
 +
 
 +
Using LSL is possible to access markers generated from another NIC. The outlet the LSL clients need to connect to has the following settings:
 +
Name: (choosen in NIC EEG settings)
 +
Type: Markers
 +
Channel count: 1
 +
Nominal sample rate: n/a
 +
Channel format: Int_32
 +
Unique source ID: The device type plus its mac address plus the "Marker" string
 +
 
 +
NIC streams some markers when starts or stops stimulation with the following settings:
 +
Name: (choosen in NIC EEG settings)
 +
Type: Markers
 +
Channel count: 1
 +
Nominal sample rate: n/a
 +
Channel format: Int_32
 +
Unique source ID: The device type plus its mac address plus the "Marker" string
 +
 
 +
NIC streams signal quality level for each channel:
 +
Name: (choosen in NIC EEG settings)
 +
Type: Quality
 +
Channel count: 8,20 or 32
 +
Nominal sample rate: 1
 +
Channel format: Float_32
 +
Unique source ID: The device type plus its mac address plus the "Marker" string
 +
 
 +
The names of the Outlets, are for the version 1.3.12. For previous versions, the name of the outlet is "NIC".
  
==  Controlling NIC with Matlab: the MatNIC toolkit ==
+
=== Sending markers using LSL ===
 +
<!-- talk about LSL client: string and integer markers -->
 +
 
 +
NIC is compliant with the [https://code.google.com/p/labstreaminglayer/ Lab Streaming Layer (LSL)] protocol so makers can be co-registered with the EEG signal by setting up a LSL marker outlet (see this [https://code.google.com/p/labstreaminglayer/source/browse/LSL/liblsl/examples/C/SendStringMarkersC/SendStringMarkersC.c example]). The received LSL markers are synchronized with the EEG streaming data and written in the penultimate column of the .easy files for further analysis.
 +
 
 +
The external software sending the LSL markers can be located in the same machine running NIC or it can also be in another machine located in the same network as the machine running NIC. If you are using two machines, you can connect both of them using an Ethernet cable or by connecting both of them in the same Wifi network.
 +
 
 +
The LSL handles both the networking and time-synchronization isues between the sender and receiver hosts obtaining reliability on order of 1 ms (see the time-synchronization validation [https://code.google.com/p/labstreaminglayer/wiki/TimeSynchronizationValidation tests]).
 +
 
 +
==== Sending integer markers using LSL ====
 +
[[File:NIC LSL settings.png|200px|thumb|right| NIC settings for configuring the reception of markers through LSL]]
 +
NIC needs little configuration in order to receive the markers from a LSL outlet present in the local network. When the LSL marker outlet sends integer-type markers, only the name of the outlet needs to be configured in NIC. Please go to "''EEG Setup -> Settings -> Markers from Lab Streaming Layer''" and set the name that your LSL marker outlet has, as shown in the figure on the right. NIC will automatically look for a marker outlet with this name and will connect to it. If the outlet sends '''integer-type markers''' then no further configuration is needed. All the received makers will be co-registered along with the EEG signal to the output files.
 +
 
 +
==== Sending string-type markers using LSL ====
 +
 
 +
In case the outlet sends '''string-type markers''' then there are some considerations that have to be taken into account. The LSL outlet sending string-type markers has to format them as XML tags. The following example is taken from the string markers that the Presentation software sends when the LSL extension manager is installed (see the [[Event_Related_Potentials_(ERPs)#Presentation | Working with ERPs]] section). You can see that NIC will decode the string looking for the tag that is configured in the "''EEG Setup -> Settings -> Markers from Lab Streaming Layer''" settings, ''ecode'' in this case. The marker number 37 will be registered at the reception of this string:
 +
  <pevent><etype>Picture</etype>'''<ecode>37</ecode>'''<unc>209.638092041016</unc>test</pevent>
 +
 
 +
<!-- talk about TTL hardware triggering -->
 +
 
 +
=== Examples ===
 +
The following links are a [[Media:LSL Script Example.rar | Python]] and a [[Media:Matlab lsl scripts.zip | Matlab]] example clients that connects to NIC in order to:
 +
 
 +
-Read from the device EEG data
 +
 
 +
-Read from the device markers
 +
 
 +
-Send to the device markers
 +
 
 +
== Sending TTL pulses ==
 +
NIC can also receive a TTL signal and display it through one of its EEG channels using our [http://www.neuroelectrics.com/products/accessories/ext-ttl-trigger-adapter/ TTL adapter].
 +
 
 +
This option requires to have a [http://en.wikipedia.org/wiki/Parallel_port#Pinouts parallel port] in the computer running the external software. If you don't have a parallel port, you can also install a [https://es.wikipedia.org/wiki/PCI_Express/ PCI Expresss] card emulating a parallel port.  Please see the user manual of our [https://www.neuroelectrics.com/api/downloads/NE_TTL_Trigger_Receiver_UserManual_201_06.pdf TTL adapter] for detailed information on how to set up the connections.
 +
 
 +
The following pins of the TTL adapter should be connected to the following pins from the parallel port:
 +
* Pin 1 (Vcc 5V) from the adapter should be connected to any data pin (pins 2-9) of the parallel port. This pin should always be activated at  5V.
 +
* Pin 2 (TTL input) from the adapter should be connected to another data pin (pins 2-9) of the parallel port. This pin should only be activated whenever a TTL pulse is sent.
 +
* Pin 3 (ground) from the adapter should be connected to any ground pin (pins 18-25) of the parallel port.
 +
 
 +
The TTL Input should be between (3.3V-5V).
 +
 
 +
NOTE: the TTL signal will be synchronized with the EEG streaming data and you'll see the TTL pulses in the EEG channel you have used to connect it, but the TTL pulses will not be "written" in the penultimate column of the .easy files as markers. The "detection of the edges" of the TTL pulses in the EEG channel for offline analysis is responsability of the user.
 +
 
 +
As an example, in the following figure is displayed a connection between the parallel port and the TTL adapter:
 +
 
 +
                                    [[File:Figure_1_Parallel_port.png| 500 px]]
 +
 
 +
Another option could be that the TTL trigger receiver is powered by 5V from an USB and TTL trigger is sent by an external device by the BNC connector. To integrate this it could be created a connection like in the following image:
 +
 
 +
[[File:Figure2.png| 500 px]]  [[File:Exampleofarealcable.jpg| 500 px]]
 +
 
 +
Every time a pulse is sent, it can be seen in NIC software as in the following image:
 +
 
 +
                                                [[File:Pulses.png| 500 px]]
 +
 
 +
 
 +
To try the TTL adapter before using it in a real subject we can do the following connections using the Tesboard. In the following example we will receive the pulse in channel P7:
 +
 
 +
[[file:TTL_with_the_testboard_to_test.jpg| 500 px]]
 +
 
 +
== Sending commands to NIC ==
 
<!-- talk about the features of NIC being controlling (Enobio and StarStim) using a command-based protocol-->
 
<!-- talk about the features of NIC being controlling (Enobio and StarStim) using a command-based protocol-->
 
<!-- talk about MatNIC as a set of routines that wrap this protocol to provide the functionalities of command NIC from Matlab -->
 
<!-- talk about MatNIC as a set of routines that wrap this protocol to provide the functionalities of command NIC from Matlab -->
 
<!-- provide examples -->
 
<!-- provide examples -->
  
NIC can be remotely commanded from a third-party software through a set of commands that can be sent using a TCP/IP connection. NIC listens to the '''TCP/IP port 1235''' for incoming connections. The clients that connect to that port can command the following actions:
+
NIC can be remotely commanded from Matlab using the [http://www.neuroelectrics.com/products/software/matnic-remote-stimulation-client/ ''MatNIC toolkit''] through a set of commands that can be sent using a TCP/IP connection.  
  /---------------------------------------------------------------------\
+
 
  | Action             | Device            | Command code              |
+
MatNIC Remote stimulation client is a Matlab toolkit for agile, programmatic control of NE devices for advanced researchers. MatNIC functions give you full control of all Neuroelectrics devices. With MatNIC you can, for example, modify in real-time any stimulation parameter, or receive and process EEG data.
  |---------------------------------------------------------------------|
+
 
  | Start EEG streaming | Enobio & StarStim | 0xF0                      |
+
For more information, visit the [https://www.neuroelectrics.com/wiki/index.php?title=MatNIC_Matlab_Toolkit '''MatNIC section'''] and if you are interested, please [http://www.neuroelectrics.com/contact '''contact us'''] for pricing and more information.
  | Stop EEG streaming  | Enobio & StarStim | 0xF1                      |
+
 
  | Start Stimulation  | StarStim          | 0xFB                      |
+
The MatNIC zip file contains:
  | Abort Stimulation  | StarStim          | 0xFA                      |
+
- The MatNIC Manual, an explanation on how to use the Matlab functions to remotely control NIC and
  | Load template      | StarStim          | String terminated with    |
+
- RemoteStimulatioClient, a folder with all the Matlab functions for remotely controlling NIC, and an example on how to use them.
                                            either '\0', '\n' or '\r' |
+
 
  \---------------------------------------------------------------------/
+
NIC listens to the '''TCP/IP port 1235''' for incoming connections. The clients that connect to that port can command the following actions:
 +
 
 +
With MatNIC you can command the following actions:
 +
  /--------------------------------------------------\
 +
  | Action                       | Device            |
 +
  |--------------------------------------------------|
 +
| Load Protocol                | Enobio & StarStim |
 +
  | Start recording EEG         | Enobio & StarStim |
 +
  | Stop recording EEG           | Enobio & StarStim |
 +
  | Start Stimulation protocol   | StarStim          |
 +
| Pause Stimulation protocol  | Starstim          |  
 +
  | Abort Stimulation protocol   | StarStim          |
 +
| Online tACS Frequency Change | StarStim          |
 +
  | Online tACS Amplitude change | StarStim          |
 +
| Online tDCS Amplitude change | StarStim          |
 +
| Online tRNS Amplitude change | StarStim          |
 +
| Request status              | Enobio & StarStim |
 +
  \--------------------------------------------------/
 +
 
 +
The previous actions are examples of the toolkit but we provide more actions to control the device and the NIC software.
  
 
NIC responds to those commands with a set of status commands to indicate whether the commands are successfully processed, the stimulation is ready to be started and so on. The following table shows all the possible status value that NIC might send.
 
NIC responds to those commands with a set of status commands to indicate whether the commands are successfully processed, the stimulation is ready to be started and so on. The following table shows all the possible status value that NIC might send.
  
  /---------------------------------------------------------------------\
+
  /--------------------------------------------------------\
  | Status             | Device            | Status code              |
+
  | Status                             | Device            |
  |---------------------------------------------------------------------|
+
  |--------------------------------------------------------|
  | EEG streaming is ON | Enobio & StarStim | 0xF2                    |
+
| Remote control allowed            | Enobio & StarStim |
  | EEG streaming is OFF | Enobio & StarStim | 0xF3                    |  
+
| Remote control rejected            | Enobio & StarStim |
  | Template not loaded | StarStim          | 0xF9                    |
+
| Device is idle                    | Enobio & StarStim |
  | Stimulation is ready | StarStim          | 0xFC                    |
+
  | EEG streaming is ON               | Enobio & StarStim |
  | to be started       |                   |                          |
+
  | EEG streaming is OFF               | Enobio & StarStim |  
  | Stimulation is ON   | StarStim          | 0xFD                    |
+
  | Protocol not loaded               | StarStim          |
  | Stimulation is OFF   | StarStim          | 0xFE                    |
+
  | Protocol loaded                    | StarStim          |
  \---------------------------------------------------------------------/
+
  | Stimulation is ready to be started | StarStim          |
 +
  | Stimulation is ON                 | StarStim          |
 +
  | Stimulation is OFF                 | StarStim          |
 +
  \--------------------------------------------------------/
 +
 
 +
== Sending Markers using the keyboard ==
 +
 
 +
You can also send manual events using the number keys (1-9) of your keyboard. To configure the codes for each marker, you can go to the list of protocols in NIC --> Select one --> SETTINGS, as described in page 22 and 23 of NIC's [https://www.neuroelectrics.com/documentation/ user manual].
 +
 
  
'''MatNIC''' wraps all these commands and status code in a set of Matlab functions that allow remotely controlling NIC. The MatNIC zip file ([http://www.neuroelectrics.com/contact '''contact us'''] for more info) contains:
+
[[File:Settings markers.PNG| 500 px]]
- The MatNIC manual (also available in our Downloads Area), an explanation on how to use the Matlab functions to remotely control NIC and
+
 
- RemoteStimulatioClient, a folder with all the Matlab functions for remotely controlling NIC, and an example on how to use them.
+
Once you start with the streaming of the EEG data, if you press a number key they appear in the time domain as in the following image:
  
To run the example please make the following call:
+
  [[File:Sending_makers_to_NIC2.PNG| 500 px]]
  >> ret = RemoteStimulationClient (templateName, '127.0.0.1', 1235)
 
where templaName is set with a name of an already defined stimulation template NIC, then the host where NIC is running (in the example above we use the localhost IP, so NIC is running in the same computer) and finally the port 1235 which is where the NIC server runs.
 
Here is the header of the example client code:
 
function [ret] = RemoteStimulationClient (templateName, host, port)
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
% RemoteStimulationClient
 
%
 
% This function implements a client to connect to the NIC Remote
 
% Stimulation Server. The server is running on the machine where the NIC
 
% application runs on the port 1235.
 
%
 
% The NIC Remote Server waits for the client to send the name of the
 
% template that the client wants to use. If the NIC user accepts the
 
% remote control session, the template is loaded and the client is
 
% informed when the system is ready to start the stimulation.
 
%
 
% The client then might send the code to start the stimulation or the one
 
% to abort it. The abort command might be sent in any moment of the
 
% stimulation session. If the stimulation session is aborted locally from
 
% the NIC GUI the connection is automatically closed so the client needs
 
% to connect, send the template and wait again for the permission. When
 
% the stimulation script execution is finished the system prepares itself
 
% to start again a new stimulation so the code informing about that is
 
% received.
 
%
 
% The client might also send commands to start and stop receiving the EEG
 
% streaming. The data is received through a separate server on the port
 
% 1234. The 8/20 channel samples are sent in 4 bytes in two's complement.
 
% The MSB byte is sent first.
 
%
 
% The following are the functions to connect, send the commands to the
 
% server and read the status the server sends:
 
% - NICRemoteStimulationServerConnect
 
% - NICRemoteStimulationServerReadStatus
 
% - NICRemoteStimulationServerLoadTemplate
 
% - NICRemoteStimulationServerStartStimulation
 
% - NICRemoteStimulationServerAbortStimulation
 
% - NICRemoteStimulationServerStartEEG
 
% - NICRemoteStimulationServerStopEEG
 
%
 
% The following are the status that the server might send to the client
 
% and can be read with the NICRemoteStimulationServerReadStatus function:
 
%
 
% STATUS              | TYPE        |      CODE
 
% -------------------------------------------------------
 
% EEG streaming is ON  |  Byte        |      0xF2
 
% -------------------------------------------------------
 
% EEG streaming is OFF |  Byte        |      0xF3
 
% -------------------------------------------------------
 
% Template not loaded  |  Byte        |      0xF8
 
% -------------------------------------------------------
 
% Template loaded      |  Byte        |      0xF9
 
% -------------------------------------------------------
 
% Stimulation is ready |  Byte        |      0xFC
 
%  to be started        |              |
 
% -------------------------------------------------------
 
% Stimulation is ON    |  Byte        |      0xFD
 
% -------------------------------------------------------
 
% Stimulation is OFF  |  Byte        |      0xFE
 
% -------------------------------------------------------
 
%
 
% Input:
 
% templateName: Name of the template to be loaded on NIC.
 
% host: Name or IP of the host where NIC is running.
 
% port: Port number where the NIC server runs.
 
%
 
% Output:
 
% ret: Zero or positive number if the function completes successfully. A
 
% negative number otherwise:
 
% -1: The connection to the host and port number did not succeed.
 
% -2: Error writing template name.
 
% -3: Error reading from server.
 
% -4: Template not loaded.
 
% -5: Error writing command to the server.
 
%
 
% Author: Javier Acedo (javier.acedo@starlab.es)
 
% Company: Neuroelectrics
 
% Created: 16 Jan 2013
 
% Known issues: None
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 

Latest revision as of 06:47, 11 July 2022

In this page we describe how you can interact with NIC (Neuroelectrics Instrument Controller, the software for control with NE devices) using other software.

About Synchronization: general principles

The Master system's clock is the only one used in the whole system. The master system either gathers the data from the slave systems to provide a single output or sends its clock so the slave systems can provide their outputs using that clock

To coordinate different data sources or events to a single clock is known as synchronization.

When using Enobio at least there are two different clocks that need to be synchronized: The clock of the Enobio wireless sensor which is in charge of sampling the EEG data recorded by the electrodes, and the clock of the host where NIC runs and receives the EEG data from the Bluetooth connection.

NIC uses the clock from the Enobio sensor as master. The Enobio clock is received by NIC through the data streaming. The EEG data is sampled at 500 Hz, so every sample is delayed 2 ms from the previous one. In traditional wired system the data will be received by the control software as it is sampled so the two clocks might be directly synchronized. However in wireless system such as the Enobio one, some latency might be introduced in the wireless channel due to RF interference and re-transmission of data.

NIC implements an algorithm that compensates this latency and finds out the offset between the clock from Enobio and the one from the host where NIC runs. When NIC receives information from third-party applications that need to be synchronized with the EEG data, like markers that signal when external events occur, it compensates the timestamp of the received data so it is aligned with the EEG data streaming.

The relative values of the timestamps recorded in the easy files are corresponding to the unadjusted clock on the Necbox acquisition device. So each sample is separated 2 ms form the previous one according to the necbox clock.

The absolute Unix time is started reading the clock of the PC (the Necbox does not have a realtime clock), so that if you start a recording at 10:00 am, the easy file will start with the corresponding Unix time of 10:00am and then the rest of the data will be added to the 10:00am according to the unadjusted necbox clock.

In the scenario described above, when NIC collects markers that are sent by other applications, another clock to be synchronized is introduced in the system. This is the clock from the host that sends the markers. NIC provides two ways of gathering such markers. Both of them are described in the following sections. The first one does not provide any synchronization mechanism, this is the reception of markers using a TCP/IP server where the clients connect to and send their markers. This method might be useful when the time synchronization requirements do not need accuracy under the 100 ms.

The second method uses the Lab Streaming Layer (LSL), which incorporates built-in network and synchronization capabilities that allow synchronization accuracy on 1 ms so it perfectly fits application like the one that detect ERPs signals.


TCP

Receiving data streams using TCP/IP

TCP (Transmission Control Protocol) is a connection-oriented protocol for transferring data reliably in either direction between a pair of users. The NIC software has a TCP/IP server that streams the EEG data received from Enobio. Up to 5 clients can connect to that server simultaneously in order to receive the EEG data ans perform the desired operations in real time.


NIC settings for configuring the sending of markers through the TCP/IP connection

The software clients that want to receive the EEG data in real time from NIC need to connect to the TCP/IP port 1234 of the host where the NIC software is running. Once the client software is connected to the server, it will receive the EEG data streaming according to the following format:

-------------------------------------------------------------------------------------------
|                    Channel 1                  | ... |             Channel N             | 
-------------------------------------------------------------------------------------------
| (MSB) Byte#1 | Byte#2 | Byte#3 | (LSB) Byte#4 | ... | Byte#1 | Byte#2 | Byte#3 | Byte#4 |
-------------------------------------------------------------------------------------------

Each EEG sample is sent as a two-complement 4 byte value. The unit of the EEG sample is nano volts and its range is from -400000000 to +400000000 nV. The most significant byte is sent first. The following code in 'C' shows how to decode the streaming from the received bytes to EEG sample values. The example assumes that the computer architecture is little-endian.

// byte3 = 0xF7, byte2 = 0x8F, byte1 = 0x99, byte0 = 0x61
signed int32_t sample = 0;
sample += byte3;
sample = sample << 8;
sample += byte2;
sample = sample << 8;
sample += byte1;
sample = sample << 8;
sample += byte0;
// sample = -141584031 nV

The client will receive first the four bytes from channel 1, then the next four bytes from channel 2 and so on till receiving the four bytes from the last channel of Enobio (8 or 20 depending of the type of Enobio/Starstim NIC handles). Then channel 1 bytes are receiving again.

It is also possible to receive the markers that NIC collects using the TCP/IP connection. By enabling this feature from the NIC settings the markers are sent along with the EEG streaming through the same TCP/IP connection. In this case the markers are sent as a four-bytes integer after the last EEG channel. The most significant byte is sent first. The data streaming will have the following format then:

-----------------------------------------------------------------------------------------------------------------------------------
|                    Channel 1                  | ... |             Channel N             |               Marker              |
-----------------------------------------------------------------------------------------------------------------------------------
| (MSB) Byte#1 | Byte#2 | Byte#3 | (LSB) Byte#4 | ... | Byte#1 | Byte#2 | Byte#3 | Byte#4 | Byte#1 | Byte#2 | Byte#3 | Byte#4 |
-----------------------------------------------------------------------------------------------------------------------------------


Those markers correspond to the ones received by NIC from third-party software using TCP/IP or LSL, the manual markers that can be inserted while recording by pressing the keys from 1 to 9 and the stimulation events in the case of the Starstim device such as start and stop stimulation and when any parameter is changed from MatNIC.

Example:

Sending Markers using TCP/IP

NIC provides a TCP server that other softwares can connect to in order to send markers. Those received markers are synchronized with the EEG streaming and written in the penultimate column of the .easy files for further analysis.

You need to create a TCP client inside your external software. The connection of the TCP client to the NIC TCP server is done using the standard TCP/IP connection protocols available in different programming languages. Up to five clients can simultaneously send markers to NIC by making a TCP/IP connection to this port. The TCP clients can be located in the same machine running NIC or they can also be in another machine located in the same network as the machine running NIC.

If the TCP client is located in an external machine in the same network as the machine running NIC, you just need the to connect to the IP of the machine running NIC, shown in the right bottom corner of NIC, and the TCP/IP port 1234. If the TCP client is located at the same machine running NIC, the IP to which you have to connect is the local IP 127.0.0.1 and port 1234. Once the client is connected, you'll see a progress bar in the right bottom corner of NIC showing the connection.

Once a client is connected it needs to send the following string in order to send a marker:

<TRIGGER>XXXX</TRIGGER>

Where XXXX can represent any integer number different from zero (from -2147483647 to +2147483647). This marker will be co-registered in the output files generated by NIC to the corresponding EEG sample.


For instance, in the output tabulated text file, the column just after the timestamp one is filled with zeros if no markers are received. When a marker is received its corresponding number is set to that column. See the following example:

...
26748	-27675	35631	42398	532666	64345	12376	40988	0	1382432459788
26865	-26683	35685	42450	532711	64821	12376	41046	0	1382432459790
26810	-26821	35531	41997	532821	64945	13164	41099	0	1382432459792
26749	-26995	35325	42008	532712	64377	13478	41286	0	1382432459794
26796	-27245	35932	42391	532923	64245	13620	41117	300	1382432459796  <-- Reception of the marker #300
26622	-27510	35501	42630	532876	64193	13031	40986	0	1382432459798
26751	-27912	35611	42003	532345	64344	12967	40731	0	1382432459800
...


Those markers correspond to the ones received by NIC from third-party software using TCP/IP or LSL, the manual markers that can be inserted while recording by pressing the keys from 1 to 9 and the stimulation events in the case of the StarStim device such as start and stop stimulation and when any parameter is changed from MatNIC.

Examples

Please take as an example a Matlab code to do the following interactions:

-Connects the TCP server

-Disconnects the TCP server

-Reads the EEG signal for t number of seconds

-An example to read data with a pause to do some processing

- Send via TCP a marker

Please take an example to plot EEG data from the different channels and save the data

Please take also an example this Matlab code which connects to NIC to send markers every time a tone is played back through the sound card. If you connected the output of the computer sound card to one of the Enobio electrodes you would be able to see the alignment between the markers and the played tones.

The following link eprime2NIC_example.zip also contains an example on how to send TCP markers from E-Prime to NIC.

LSL

Receiving and sending data streams using LSL

NIC streams the received EEG data from Enobio and Starstim using the Lab Streaming Layer. NIC creates a LSL outlet with the following settings:

Name: (choosen in NIC EEG settings)
Type: EEG
Channel count: 8, 20 or 32 depending of the Enobio NIC handles
Nominal sample rate: 500
Channel format: float_32
Unique source ID: The Enobio/Starstim type plus its mac address

An LSL client software needs to connect to this outlet in order to receive the EEG streaming data. The received values are expressed in microvolts and its range is from -400000 to +400000.

Using LSL is possible to access to the accelerometer data too. The outlet the LSL clients need to connect to has the following settings:

Name: (choosen in NIC EEG settings)
Type: Accelerometer
Channel count: 3
Nominal sample rate: 100
Channel format: float_32
Unique source ID: The device type plus its mac address plus the "Acc" string

Using LSL is possible to access markers generated from another NIC. The outlet the LSL clients need to connect to has the following settings:

Name: (choosen in NIC EEG settings)
Type: Markers
Channel count: 1
Nominal sample rate: n/a
Channel format: Int_32
Unique source ID: The device type plus its mac address plus the "Marker" string

NIC streams some markers when starts or stops stimulation with the following settings:

Name: (choosen in NIC EEG settings)
Type: Markers
Channel count: 1
Nominal sample rate: n/a
Channel format: Int_32
Unique source ID: The device type plus its mac address plus the "Marker" string

NIC streams signal quality level for each channel:

Name: (choosen in NIC EEG settings)
Type: Quality
Channel count: 8,20 or 32
Nominal sample rate: 1
Channel format: Float_32
Unique source ID: The device type plus its mac address plus the "Marker" string

The names of the Outlets, are for the version 1.3.12. For previous versions, the name of the outlet is "NIC".

Sending markers using LSL

NIC is compliant with the Lab Streaming Layer (LSL) protocol so makers can be co-registered with the EEG signal by setting up a LSL marker outlet (see this example). The received LSL markers are synchronized with the EEG streaming data and written in the penultimate column of the .easy files for further analysis.

The external software sending the LSL markers can be located in the same machine running NIC or it can also be in another machine located in the same network as the machine running NIC. If you are using two machines, you can connect both of them using an Ethernet cable or by connecting both of them in the same Wifi network.

The LSL handles both the networking and time-synchronization isues between the sender and receiver hosts obtaining reliability on order of 1 ms (see the time-synchronization validation tests).

Sending integer markers using LSL

NIC settings for configuring the reception of markers through LSL

NIC needs little configuration in order to receive the markers from a LSL outlet present in the local network. When the LSL marker outlet sends integer-type markers, only the name of the outlet needs to be configured in NIC. Please go to "EEG Setup -> Settings -> Markers from Lab Streaming Layer" and set the name that your LSL marker outlet has, as shown in the figure on the right. NIC will automatically look for a marker outlet with this name and will connect to it. If the outlet sends integer-type markers then no further configuration is needed. All the received makers will be co-registered along with the EEG signal to the output files.

Sending string-type markers using LSL

In case the outlet sends string-type markers then there are some considerations that have to be taken into account. The LSL outlet sending string-type markers has to format them as XML tags. The following example is taken from the string markers that the Presentation software sends when the LSL extension manager is installed (see the Working with ERPs section). You can see that NIC will decode the string looking for the tag that is configured in the "EEG Setup -> Settings -> Markers from Lab Streaming Layer" settings, ecode in this case. The marker number 37 will be registered at the reception of this string:

<pevent><etype>Picture</etype><ecode>37</ecode><unc>209.638092041016</unc>test</pevent>


Examples

The following links are a Python and a Matlab example clients that connects to NIC in order to:

-Read from the device EEG data

-Read from the device markers

-Send to the device markers

Sending TTL pulses

NIC can also receive a TTL signal and display it through one of its EEG channels using our TTL adapter.

This option requires to have a parallel port in the computer running the external software. If you don't have a parallel port, you can also install a PCI Expresss card emulating a parallel port. Please see the user manual of our TTL adapter for detailed information on how to set up the connections.

The following pins of the TTL adapter should be connected to the following pins from the parallel port:

* Pin 1 (Vcc 5V) from the adapter should be connected to any data pin (pins 2-9) of the parallel port. This pin should always be activated at   5V.
* Pin 2 (TTL input) from the adapter should be connected to another data pin (pins 2-9) of the parallel port. This pin should only be activated whenever a TTL pulse is sent.
* Pin 3 (ground) from the adapter should be connected to any ground pin (pins 18-25) of the parallel port.

The TTL Input should be between (3.3V-5V).

NOTE: the TTL signal will be synchronized with the EEG streaming data and you'll see the TTL pulses in the EEG channel you have used to connect it, but the TTL pulses will not be "written" in the penultimate column of the .easy files as markers. The "detection of the edges" of the TTL pulses in the EEG channel for offline analysis is responsability of the user.

As an example, in the following figure is displayed a connection between the parallel port and the TTL adapter:

                                   Figure 1 Parallel port.png

Another option could be that the TTL trigger receiver is powered by 5V from an USB and TTL trigger is sent by an external device by the BNC connector. To integrate this it could be created a connection like in the following image:

Figure2.png  Exampleofarealcable.jpg

Every time a pulse is sent, it can be seen in NIC software as in the following image:

                                               Pulses.png


To try the TTL adapter before using it in a real subject we can do the following connections using the Tesboard. In the following example we will receive the pulse in channel P7:

TTL with the testboard to test.jpg

Sending commands to NIC

NIC can be remotely commanded from Matlab using the MatNIC toolkit through a set of commands that can be sent using a TCP/IP connection.

MatNIC Remote stimulation client is a Matlab toolkit for agile, programmatic control of NE devices for advanced researchers. MatNIC functions give you full control of all Neuroelectrics devices. With MatNIC you can, for example, modify in real-time any stimulation parameter, or receive and process EEG data.

For more information, visit the MatNIC section and if you are interested, please contact us for pricing and more information.

The MatNIC zip file contains:
- The MatNIC Manual, an explanation on how to use the Matlab functions to remotely control NIC and
- RemoteStimulatioClient, a folder with all the Matlab functions for remotely controlling NIC, and an example on how to use them.
NIC listens to the TCP/IP port 1235 for incoming connections. The clients that connect to that port can command the following actions:
With MatNIC you can command the following actions:
/--------------------------------------------------\
| Action                       | Device            |
|--------------------------------------------------|
| Load Protocol                | Enobio & StarStim |
| Start recording EEG          | Enobio & StarStim |
| Stop recording EEG           | Enobio & StarStim |
| Start Stimulation protocol   | StarStim          |
| Pause Stimulation protocol   | Starstim          | 
| Abort Stimulation protocol   | StarStim          |
| Online tACS Frequency Change | StarStim          |
| Online tACS Amplitude change | StarStim          |
| Online tDCS Amplitude change | StarStim          |
| Online tRNS Amplitude change | StarStim          |
| Request status               | Enobio & StarStim |
\--------------------------------------------------/

The previous actions are examples of the toolkit but we provide more actions to control the device and the NIC software.

NIC responds to those commands with a set of status commands to indicate whether the commands are successfully processed, the stimulation is ready to be started and so on. The following table shows all the possible status value that NIC might send.

/--------------------------------------------------------\
| Status                             | Device            |
|--------------------------------------------------------|
| Remote control allowed             | Enobio & StarStim |
| Remote control rejected            | Enobio & StarStim |
| Device is idle                     | Enobio & StarStim |
| EEG streaming is ON                | Enobio & StarStim |
| EEG streaming is OFF               | Enobio & StarStim | 
| Protocol not loaded                | StarStim          |
| Protocol loaded                    | StarStim          |
| Stimulation is ready to be started | StarStim          |
| Stimulation is ON                  | StarStim          |
| Stimulation is OFF                 | StarStim          |
\--------------------------------------------------------/

Sending Markers using the keyboard

You can also send manual events using the number keys (1-9) of your keyboard. To configure the codes for each marker, you can go to the list of protocols in NIC --> Select one --> SETTINGS, as described in page 22 and 23 of NIC's user manual.


Settings markers.PNG

Once you start with the streaming of the EEG data, if you press a number key they appear in the time domain as in the following image:

Sending makers to NIC2.PNG