Размещено на http://www.allbest.ru/



Аннотация

платформа нейроинтерфейс серверный когнитивный

В данной статье рассматривается подход к использованию собранных данных с ЭЭГ устройства, включая артефакты, который может быть использован для решения широкого круга задач в области контроля ментального состояния пользователя, что безусловно имеет огромное значение, и может быть использовано в решении ряда задач для разных секторов, начиная от оценки внимания водителя за рулем (с целью предотвращения потери контроля на дороге), и заканчивая телемедициной (например, контроля качества сна).

Работа состоит из трех основных частей: Введение, описание подхода к решению задачи и выводы. Вводная часть посвящена актуальности проблемы, и соответственно постановке цели и задачи, а также определению объекта и предмета исследований. Описание подхода - дает детальное объяснение реализации, с учетом техническо-экономического обоснования.

Данная работа рассматривает не только проблемы, связанные с обработкой ЭЭГ сигналов, но также покрывает проблемы, связанные с анализом данных, их безопасным хранением и манипуляции.

Ключевые слова: ЭЭГ, мозговые волны, JavaScript, blockchain, большие данные, solidity, машинное обучение, ethereum, IPFS.



Abstract

This paper outlines the way in which the RAW EEG data, alongside with the artifacts, may be used to solve the fundamental problems, in the field of mental control, which is necessary for different kind of tasks and professions, varied from drivers (who can lose control on the road and got into an accident) up to the sports sector and lifestyle (controlling the sleep level).

The work consists of three major parts: Introduction, Approach description, and Conclusion. The introduction point out the relevance of the topic and the problem sets goals and objectives and defines the object and subject of research. While the Approach description gives the in-depth explanation of the implementation from technical and economic sides.

It dives into issues related not only to processing EEG itself but also covers the general problems of data analyze, its secure storing and manipulation.

Keywords: EEG, brain waves, JavaScript, blockhain, big data, solidity, machine learning, ethereum, IPFS.



Chapters

• Introduction
• 1. Scientific meaning
• 2. General Concept
• 3. Brain signal processing
• 3.1 The EEG sensor
• 3.2 Identifying Brain Activity Patterns
• 4. Data processing
• 4.1 Prediction models and pure functions
• 4.2 Server information exchange with Oauth2.0
• 4.3 Pure functions delivery
• 5. VM Security
• 6. Technological stack
• 7. Data storage
• 7.1 Decentralized solution
• 7.2 Off-chain and performance improvement
• 7.3 Features and limitations
• 7.4 IPFS as a second storage engine
• 7.5 File lifecycle
• 7.6 Security Issues
• 8. Economic model
• 9. Conclusions
• 10. References
• 11. Application


Introduction

Time passes by, and progress moves forward as well: we think, we analyze and we take decisions - but what familiar with all these processes? The answer lies in our brain. The brain could be treated as a central processor, which is responsible not only for our mental activity but also for the work of our body - from the projection of our feelings up to controlling our hormonal system. As a result, one of the main humanity challenges, which we try to overcome, is lied in understanding and interpretation of our mental state: from internal feelings, up to the recognition of specific disease.

To track the user's mental activity, the approach of using EEG technology is used. However, the current state in a field of neuroscience is not so brilliant as it could seem. Not so long time ago, the humanity started to investigate in what brain is and how does it work. One of the fascinating discoveries for the past 20 years was related to finding the common patterns, by which we can determinate the specific state. Such events are called `artifacts'.

As we have already pointed out, these artifacts can determinate only current state, but can't describe the possible changes over time. This is where NeuroIO project is.

The object of our research and product itself is devoted, mostly, to telemedicine and machine analyze fields, and dive into a way, of EEG signals and artifacts processing in short/long running tasks.

The subject of this research lies in the development of cloud-based service, which is responsible for storing and processing data, got from EEG, including artifacts processing.

The following work has been introduced on XLIV International Gagarin science conference in 2018 year as an article “Machine Learning in brain waves processing. Detecting driver's attention level through SVM”.



1. Scientific meaning

The critical point of this research is to introduce to the reader the problems, related to mental activity, and the way, of how these problems may be resolved thanks to EEG technology and machine learning.

This work has a significant meaning for the telemedicine sector, as it completely wipes edges between substantial medical organizations and general audience, by exposing them the tool for tracking and analyze their mental activity in the long/short running tasks and giving them an ability to buy devices under low cost.



2. General Concept

According to our concept, the system should track the user's mental activity in real-time and predict possible future changes based on regression analyze / Neural network result. The second point of the platform is an ability to make long-running predictions, based on already grabbed EEG data for the defined period (for example - 1 month). This full circle should cover the primary use cases, like tracking attention level or sleep quality.

The platform will be divided into several parts: mobile app, backend, admin panel and storage connector.

Mobile app

The mobile app will be responsible for interaction with user: show him his mental activity level, inform about decreased attention level and so on. Beside it, the app will also synchronize with portable EEG device, to fetch the RAW data and make short-running predictions right away, and send the results alongside with RAW data to the backend.

Backend

The backend is used to handle the incoming EEG data, individual users requests (like registration, obtaining record history and so on) and perform deep analyze incoming data.

Storage connector

The storage connector is responsible for storing and manipulating the data. It is the combination of storage engine and DAO (Data Access Object) layer. The reason for implementing the storage connector is lied in providing the client an ability to specify the storage, which will better suit his business.

Admin panel

An admin panel is a tool for an operator/business owner, who want to track his staff during his work. The panel cover the primary use cases occurred during the delivery / post-delivery processes. Thus include driver's tracking (his mental activity), recorded history and export tools, to export the recorded data.

Below you can find the general implementation scheme:

Pic. 1. General implementation scheme



3. Brain signal processing

One of the most difficult challenges regarding the developed platform lies in EEG data processing. Under the hood, the following task involves not only a strong background in neuroscience field but also a good understanding of machine learning, to find specific correlations between datasets.

The datasets are represented as certain values, which varies by its frequency. Below, you will find the basic description of each wave and its meaning:

Table 1

Wave	Frequency	Description / active state
Delta	0-4 Hz	deep sleep
Theta	4-8 Hz	general sleep
Alpha	8-12 Hz	Relax state
Beta	12-40 Hz	Main mental activity (concentration)
Gamma	40-100 Hz	Deep concentration

In case, some of these frequencies are lied out of bounds - we can come up to the conclusion, that current client/user has a problem related to specific wave. Below is presented the common cases table:

Table 2

Wave	Frequency	Too low	Too high
Delta	0-4 Hz	poor sleep	Depressants
Theta	4-8 Hz	Anxiety, poor emotional awareness, stress	ADHD, depression, hyperactivity, impulsivity, inattentiveness
Alpha	8-12 Hz	Anxiety, high stress, insomnia, OCD	Alcohol, marijuana, relaxants, some antidepressants
Beta	12-40 Hz	ADHD, daydreaming, depression, poor cognition	Coffee, energy drinks, various stimulants
Gamma	40-100 Hz	ADHD, depression, learning disabilities	Meditation

All these states are tracked not only by current obtained results but by its changes over the specific timeframe. Such type of state recognition is called `artifact'. The artifact recognition is entirely a complicated process, which involves much effort from scientific research groups.

For our solution, we've established a tight collaboration with research groups from Neurosky and Macrotellect, and we build our predictive services based on their approved researchers.

3.1 The EEG sensor

The EEG sensor detects and interprets the electrical voltages that are created by mind cells (neurons). The frequencies most ordinarily for EEG are between 1 to 40 Hz. The EEG sensor records a "crude" EEG flag, which is the continually changing contrast of potential between the positive and negative anode, and the product forms that flag by applying an assortment of digital channels to the recorded flag, keeping in mind the end goal to concentrate recurrence space data. The EEG flag has different recurrence groups, divided by certain criteria. These criteria are called "brain waves," and represent a specific band with the particular range (like delta wave, which has band 0.5-4Hz). The leading bands and their properties we will discover later.

3.2 Identifying Brain Activity Patterns

Beside just tracking the particular mental state, like a dream, or an opposite - any activity, the modern neurology science can track specific events, like blinking, or any kind of movements.

These events are called artifacts. These artifacts describe how do certain brain waves should change their frequency over a short timeframe. If during record, we see such change, which satisfies this pattern - then it means, that we've caught this event.

Artifacts

Regarding neurology artifact - is a particular kind of function, which describes specific user state. Under the hood, this function takes particular brain waves and check it changes (its frequency) in a timeframe. In case, we notice smth similar to our pattern - we can treat with this event, as it is an artifact. Below is presented a sample form of artifact's function:

(3.2.1)

(3.2.2)

The first function here - is a Fast Fourier Transform. This function will transform signals, received from EEG device from the time domain, to the frequency domain. While the second function is a simple naпve Bayes classifier. As a result, we process data in some timeframe, then analyze it and compare with some pattern - In case it will satisfy the condition, then we can be sure, that the current state describes specific mind activity (like eyes blinking, muscle work and so on).

NeuroIO brings an upper layer over exciting artifacts. In other words, we don't track artifacts events (as they are already tracked on the lower level, thanks to an API of our partners). What platform do - it tracks changes of a particular artifact, and predict it possible change over a specific timeframe. For instance, it can determinate possible attention level of the current user in next 5-10 minutes in real-time, and build the prediction model for the next several months.



4. Data processing

Artifacts processing happens in two ways: on the client side and server side. The client-side analytics Is represented as a set of low-cost models, like general linear regression. The term `low-cost' - means, that we don't need many resources to calculate results. The opposite approach would be used Neural networks, as their usage is many resources addicted, then just using simple classifiers and regression models.

One of such business cases, when we need to perform client-side predictions - is driver attention level.

When we talk about the algorithm, which will describe the user's attention level in the short run, the first thing we have to think about is defining the short period. In other words, which period do I want to predict. For our case, we will take short timeframe of 5 minutes.

The next step will be choosing the best approach to solving the issue related to predictions. According to our data, we have two base points - attention level and timestamp. As a result, we can conclude that our model will be similar to linear regression. However, we still need to pay attention to data accuracy: in case, attention level will change rapidly, for instance, due to noise issues, we can retrieve the negative result of our prediction.

To overcome this problem, we can introduce a simple neural network, where each timeframe will be represented in percentage from the end timestamp (our prediction time). This approach will help us transform our data to the suitable form.

The last step is lied in building the suitable model. According to our concept, this neural network will have one layer, where activation function will be presented as a general logistic sigmoid function.

Let us try our models on an array of 4000 points, where each point - is a time stamp, and its value is attention level, received at this time (the code is written in Javascript ES6):

const synaptic = require('synaptic'), shaman = require('shaman'), mongoose = require('mongoose'), _ = require('lodash'), brain = require('brain.js'), PointsSchema = require('./PointsSchema'), config = require('../app/config'); const init = async () => { await mongoose.connect(config.mongoUrl, { useMongoClient: true, connectTimeoutMS: 1000 }); let point = await PointsSchema.findOne({_id: '5a72e089aa5d6e0019a5d7cc'}); const data = _.chain(point.asset.points) .map(point => point.dots) .flattenDeep() .value(); let lr = new shaman.LinearRegression(data.map(c => c.created), data.map(c => c.attention)); await new Promise(res => { lr.train(() => res()); }); console.log(lr.predict(_.last(data).created)); let net = new brain.NeuralNetwork(); let trainingSet = _.chain(data) .map(dot=>({ input: [dot.created / _.last(data).created], output: [dot.attention / 100] })) .value(); net.train(trainingSet); const result = net.run([1])[0] * 100; console.log(result);

}; module.exports = init();

Here, general linear regression's result is 29.659258426465385, while neural network's result is 29.97494339942932.

As we can see, the final calculations do not much different, but the execution time is 0.5s vs. 10s. This happens thanks to training algorithm of a neural network: as we have to pass all points through the input layer, process them in hidden (to find the optimal weight) and pass to the output, while with linear regression you end up with finding a relation between depended and independent variables.

4.1 Prediction models and pure functions

NeuroIO provides a flexible API for building custom artifacts, prediction, and classification models. For instance, to describe the driver's attention level, we use SVM regression with two variables and general attention level value, provided from neurosky's device.

Below is a standard function for calculating the x value, based on previous input points:

(4.1.1)

The SVM regression has been chosen for several reasons: it is cheap to calculations, scale significantly, and have the accurate result, in case, we build short predictions (for next 10-20 minutes or less). This approach has been compared with simple RNN neural network - to obtain the most accurate results.

In our case, the x value is the attention level value (which varies from 0% up to 100%), while the y value - is a timeframe. So, by simple calculation, we can find out, how the attention level will change in next second/minute/ hour.

All these descriptions are represented as a single piece of code and called `pure functions'. The origin name has been taken from functional programming. The fundamental concept is to create the function, which will not have access to the upper context. Here, we followed the same idea: we run the private function and only obtain the output value, which should be represented in percentage.

Such approach could be described by a simple linear function, which keeps all points during the session and relies on this data, during each recalculation. In other words: previous calculation will influence results of further calculations (4.1.2):

(4.1.2)

4.2 Server information exchange with Oauth2.0

To organize the high-security level of our server-client sides information exchange, we have introduced the oauth2.0 explicit flow.

OAuth was first intended for authorization, i.e., so users could perform authentication on the external resource, like Google, to validate them and get access to the specific website (like Amazon). Oauth is more than just standard - it's a registered protocol (RFC6749).

The general idea of OAuth look like that: If a user wants to authorize a relying party (RP), like amazon.com, to access some of the user's data at an identity provider (IDP), like Google , the RP redirects the user (i.e., the user's browser) to the IDP, where the user authenticates and agrees to grant the RP access to some of her user data at the IDP. Then, along with some token (an authorization code or an access token) issued by the IDP, the user is going to be redirected back to the RP. The RP can then use the token as a credential at the IDP to access the user's data at the IDP.

However, the OAuth is only a protocol, which means that before processing this flow, the following user has to be registered on IDP. During the auth process, the IDP assigns credentials to the RP: a public OAuth client id and (optionally) a client secret. (Recall that in the terminology of the OAuth standard the term “client” stands for RP.) The RP may later use the client secret (if issued) to authenticate to the IDP. According to the implicit flow, an RP registers one or more redirection endpoint URIs (located at the RP) at an IDP. That's quite an important point, as beside the implicit flow, we can use OAuth for validation purpose instead of cookies.

Step-by-Step Protocol Flow.

To understand better the protocol flow, let's look at the diagram presented below:



Pic.2. OAuth protocol flow

Here is a more detailed explanation of the flow itself:

1) The application (RP) requests authorization to access service resources (IDP) from the user

2) If the user authorized the request, the application (RP) receives an authorization grant

3) The application (RP) requests an access token from the authorization server (IDP) (API) by presenting authentication of its own identity, and the authorization grant

4) If the application (RP) identity is authenticated and the authorization grant is valid, the authorization server (IDP) (API) issues an access token to the application. Authorization is complete.

5) The application (RP) requests the resource from the resource server (IDP) (API) and presents the access token for authentication

6) If the validation of access token has been passed, then the resource server(IDP) (API) serves the resource to the application

The actual flow of this process will differ depending on the authorization grant type in use, but this is a general idea. We will explore different grant types in a later section.

Oauth 2.0 attacks (explicit flow)

Another exciting part about OAuth is the way, how the auth system could be bypassed. We won't take into a count implicit flow issues, as we took only explicit flow as a base. These types of attacks could be divided into two categories: IDP mix-up attack and state leak attack.

IDP Mix-Up Attack

In this attack, which breaks our authorization and authentication properties, the attacker confuses an RP about which IDP the user chose at the beginning of the login/authorization process to acquire an authentication code or access token which can be used to impersonate the user or access user data.

This attack applies to the authorization code mode and the explicit mode of OAuth when the RP uses explicit user intention tracking. To launch the attack, the attacker redirects or mix up the initial request of the user. This action will lead to replacing of authority side (IDP) for which the user requests the access token. As a result, user requests auth code for the attacker IDP (Aidp), instead of original one. Then, this access token will be used as an auth code on attacker resource, where he can grab this token and use it in another app, from the user's account.

State Leak Attack

Using the state leak attack, an attacker can force a browser to be logged in under the attacker's name at an RP or force an RP to use the attacker's resource instead of the original one. This attack, which breaks our session integrity property (see Section 5.2), enables what is often called session swapping or login CSRF.

After the user has granted the access to the IdP in the authorization code mode, the user is going to be redirected to RP. This request contains state and code as parameters. The response to this request can be a page containing a link to the attacker's website or some resource located at the attacker's website. When the user clicks the link, or the resource is loaded, the user's browser sends a request to the attacker. This request contains a Referer header with the full URI of the page the user was redirected to, which in this case includes state and code.

As the state value is supposed to protect the browser's session against CSRF attacks, the attacker can now use the leaked state value to perform a CSRF attack against the victim. For example, he can redirect the victim's browser to the RP's redirection endpoint (again) and by this, overwrite the previously performed authorization. The user will then be logged in as the attacker.

How to protect

All of the described attacks could be performed only under untrusted resources, where an attacker can proxy your request or filter request packages. However, all there becomes useless, if we switch our connection to a secure one (SSL), and our web resource will be signed with a certificate from the trusted authority.

4.3 Pure functions delivery

Pure function - is a virtual function, which takes an input, process it in some way, and give back the result. This result should describe the certain value in percentage range. The fascinating thing about it is that virtual function is represented as a single piece of code and executed through native VV8 VM. Below you can find a virtual sample function:

((points, options, ctx) => { if (!_.has(ctx.global, 'lastIssue')) ctx.global.lastIssue = Date.now(); let stat = _.chain(points) .thru(acc => this.simpleStatistics.linearRegression(acc.map((c, i) => [i, c.attention])) ) .get('b') .value(); if (Date.now() >= ctx.global.lastIssue + options['artifacts:drivers:attention_check'] * 60000 && stat < options['artifacts:drivers:attention_level']) { ctx.app.showToast(ctx.locale.level_decreased); ctx.app.tts(ctx.locale.level_decreased); ctx.global.lastIssue = Date.now(); } return parseInt(stat) || _.last(points).attention; })

The flow of this function could be described via the following diagram:

Pic.3. Pure function flow

Another important note about the current flow - is how do we deliver these clear functions. Initially, these functions are not hard coded on the client side but expose from our cloud service through REST API. To make sure, that these functions haven't been modified in any way during the delivery process, we use curve25519 standard.

According to our flow, right before sending these functions to the client side, each function is going to be converted to hex. Then we calculate the checksum in md5 and sign it with client's public key + server's private one. When the client receives the response from the server side, he validates the checksum of each function, by decoding this checksum with his private key.



Pic.4. Function delivery process

Such workflow reduces the chances to replace the existing data with the wrong one.



5. VM Security

The next question which comes up after we've introduced a custom prediction models - is about the security side, or how we can make sure, that specific artifact won't gain access to the mobile device memory and private data.

To overcome this issue, we have introduced a unique environment, where your functions are executed like in a sandbox. The original concept comes from a VV8 virtual machine and is taken as a base. In other words, the artifact (or just function) is executed in an isolated scope with temp context, which makes it impossible to access any data outside this sandbox.

As we have pointed out, the VV8 has been taken as a sandbox env, which means, that our functions should be written in pure Javascript. However, there are several limitations over using the sandbox: the first one - is that we are not able to run the script in VM on another thread (like we can do on the server side), the second thing - is that functions should be pure, which means they shouldn't have access to the context of an application itself. So the backend/frontend implementation will be different.

In backend we have to use the native VM module, supplied with node.js:

const vm = require('vm'); vm.runInNewContext(code, sandbox, opts);

Where code - is our function, and sandbox - is a prepared environment. This execution won't block the main thread of the application and will be executed in the background, without blocking the main queue.

From the frontend side, the execution will look a little bit different:

(function () { return eval(artifact.logic) }).call({_, simpleStatistics})

Here, we process the code through particular command - eval, which safely convert your code (in string representation) to function, then we wrap this function into a self-executed function with already bound context. This encapsulates the function from the upper context, and prepared function then is going to be executed asynchronously (as a promise), which means it won't block the main queue.



6. Technological stack

According to our concept, we have to determinate the technological stack, on which our services will be based. Each service has its significant meaning, and as a result - has its own requirements for implementation and stack itself.

Mobile App

The mobile application will not do difficult tasks: it will grab the RAW data from EEG device, apply this data to the virtual functions, display results on charts, show the user his history, and synchronize with the backend.

From this point, it's quite important to pay attention to the workload of the application - as it doesn't require any usage of GPU, or perform difficult math - there is no particular reason to write the application in native language (for instance, for android - it's java/scala, IOS - objective-c/swift). As a result, the hybrid framework, like Cordova may be used to satisfy our needs.

Another big plus about using the hybrid framework - is the speed of development and testing the software. The hybrid framework is called “hybrid” - as you write code only once and for all supported platforms. However, the hybrid frameworks are divided into groups as well (by language and runtime environment). Here are the most popular solutions:

Xamarin

Xamarin is a hybrid SDK owned by Microsoft. The solution represents itself as the cross-platform implementations of the Common Language Infrastructure (CLI) and Common Language Specifications (often called Microsoft .NET).

The common language for the SDK is C#, on which developers can write cross-platform applications (Android, IOS, Win Phone). Although the developers claim, that the applications are work as native (under the same speed), it's no so. By design, the xamarin, use an extra layer, supplied alongside with your codebase, which serves as an integration bridge between native API and the application itself.

Ionic

The Ionic Framework is a combination of Javascript based framework and Cordova platform. The Cordova - is an SDK, which brings an ability to write the applications in Javascript/HTML/CSS stack. The Ionic is just a framework, with some predefined UI components and integration bridge between native API and Javascript. This approach brings the flexibility regarding writing custom UI, or just doing things, like porting mobile version of the website as a separate application.

React Native

React native is a brand new way of writing the hybrid applications. The main key point around it is that you describe the application logic in plain Javascript, which is interpreted to byte-code for the specific platform. As a result, react native seems to be a good option for the web developer to start coding the native mobile apps. However, we should point out, that this framework, due to it's implementation is quite restricted to performing custom operations, like connecting platform specific libs, or interacting with GPU.

To choose the right framework for us, we have to rely on specific criteria:

1) Technology understanding time, or how much time does it take until you start to code on this framework/SDK

2) Implementation time: how it will be challenging to implement my task

3) Speed: how fast will work the application

4) Support: is project maintained, how much people code on it

5) Additional plugins/libs: is there any repo, where people share open source code (this can dramatically decrease development time)

In the table below, we have compared these solutions by giving those points (1-5):

Table 3

Product	Criteria 1	Criteria 2	Criteria 3	Criteria 4	Criteria 5
Ionic	5	5	3	5	5
Xamarin	3	2	3	3	2
React native	3	3	5	3	3

According to the presented table, the Ionic has been chosen as the main framework for Mobile app.

Backend

The backend is one of the core elements in our infrastructure. Its purpose - to handle user's requests, including authorization, history synchronization, and data manipulation.

As we have already pointed out, the backend part should handle a large number of requests in the short timeframe.

First of all, we have to take the decision, do we need to support real-time and what workload do we expect. In our case, we assume, that workload should be high, and the service, which we are going to implement, should handle at least 10000 clients on the single machine. From this point, we already shouldn't take into a count, languages like Python or PHP - as they are interpreted languages and work by the call from the process.

Among other popular languages - there are Node.js, Java, and C#.

All of them has its advantages as well as disadvantages.

C#

The C# - is a powerful OOP language, supported and developed by Microsoft. It is well known from the platform.Net. However, the visible part about C# is that its designed for windows and in our case could be run only on Windows Server (which at least, brings twice more spends on infrastructure, due to licenses and Windows memory utilization). Although it has the support of Linux (through mono), this support is not official and poorly maintained.

Java

Java is an OOP language supported by Oracle. Java is a quite old language, which proved himself on high load and enterprise projects. Under the hood, there are many frameworks (like spring, or java play!), which can boost development process.

Node.js

Node.js is a platform, which is designed to run Javascript on the server side. Its based on the VV8 engine designed by Google. It has wide support, and a significant number of frameworks as well (like express, loopback or koa).

From this point, we came up to conclusion that both Java and Node.js suit our deal. However, we haven't paid attention to concurrency and memory utilization problem - which leads to other problems, like resource costs, scalability and so on.

First of all, we should keep in mind, that well-designed architecture - is the key to success. For this reason, we have to think about the services design itself.

Java Concurrency Model

Java applications tend to run as a single process. However, that process can utilize numerous "threads" (depends from CPU), in order to execute tasks in parallel. As a result, the general Java web server can use hundreds of threads. Each request to the server is handled on its thread, and usually, these threads are handled in a non-blocking way (meaning they send a callback to the main process, after accomplishing their task in the background). Furthermore, these threads may exchange with information, as they have the shared memory. This brings the flexibility and higher I/O speed regarding exchanging the information between threads. However, this feature may lead to dirty object mutation, in case both threads will try to modify the object at one time.

Node.js Concurrency Model

Node.js is an entirely different approach to handling the concurrent tasks. In Node.js, a process consists of one main thread of execution and a set of background threads. This approach is also called Event-loop, and that is why: the communication between the main thread and background threads is happened thanks to a special queue, which hold all tasks, need to be performed. The event-loop loop through these tasks - the synchronous requests (tasks) are performed in place (and may block the main I/O), while the asynchronous one will be sent to the background thread, and once it did - the callback will be sent to the event-loop, the result will be wrapped and returned to the main thread. The key point of such approach, that any variable modification happens only from the main thread, so there won't be any dirty mutation. Also, there is no need to care about the queue itself, as it handled automatically by node.js.

Java vs. Node.js compare

Returning to our task: The Java seems to be an attractive solution. However, we should remember - that it's strongly typed language (which dramatically increase development time). Furthermore, the JVM machine also requires many resources to be run. On the other hand, the Node.js provide a lightweight VV8 engine, non-strong typed language, but the code execution speed may be slower (due to the fact, that VV8 perform syntactic analysis on the go and create bindings to JS functions and variables based on data type). However, if you will check the data type, before assigning/calling function - then memory utilization will be much lower, and execution speed will be high.

Based on this conditions and language (platform) specifications - the Node.js has been chosen as backend platform.



7. Data storage

When it comes to choosing the right storage model, it's quite remarkable to start from determinating the nature of the data, which we are going to handle.

In our case, most of our data will consist of the recorded EEG signals, artifacts, and analytics. All this data is going to be pushed into the Database rapidly in the short-time delay (the speed should be closed to real-time recording). However, such quick insert in the database may cause table/collection lock. Such locks will place all records, which have to be inserted, into the separate queue. This queue may grow over time, and as a result, the database performance may degrade rapidly over time.

From this point, it's important to determinate the value of our data and choose the right model of our future storage. In case, we assume, that some of our data may be eventually consistent (for instance, we push several records to the database, but we don't aware of concurrency side and write order), then we should think about NoSQL databases, neither then choosing the SQL one solution.

The main difference between SQL and NoSQL solution lies in their consistency models - ACID and BASE. The acid acronym stays for:

1) Atomic - all operations are performed successfully. Otherwise, all changes are rollback to the previous state.

2) Consistent - the structure of database won't change after the transaction has been committed.

3) Isolated - all transactions don't contend with one another

4) Durable - the results, of the committed transaction, is permanent

While the BASE one is concentrated mostly on data write throughput, instead of keeping the consistent state:

1) Basic availability - The database appears to work most of the time.

2) Soft state - Stores don't have to be write-consistent, nor do different replicas have to be mutually consistent all the time.

3) Eventual consistency - Stores exhibit consistency at some later point (e.g., lazily at the reading time).

Write consistency can be a critical feature for the particular type of data, but we should keep in mind, that requires sophisticated locking which is typically a heavyweight pattern for most use cases.

Also, the BASE datastore stands for availability (since that's important for scale), but it doesn't offer guaranteed full replication state all the time on other nodes. Overall, the BASE consistency model doesn't provide such insurance than ACID. But from another point of view, the BASE, due to these simplified consistency rules, offer more powerful write throughput mechanism, and flexible model schemas, over restricted one in SQL.

Navigating ACID vs. BASE Trade-offs

To sum up the above research: we definitely should care about the data, it's consistency and so on. However, we also should point out, that in our case, the data could be divided into two major groups - essential and inessential one. Where essential - is user profile and long-running predictions, while inessential - is grabbed RAW data from EEG device.

This is the main reason for choosing NoSQL over SQL in our use case. Beside the fact of high write throughput ability, it's also necessary to mention about the economics of holding such warehouse.

By choosing the SQL (for instance PostgreSQL or Oracle g11), we have to care about increased lock time during writes. To overcome this issue, we have to think about vertical scale over the single node. For instance, to hold about 10000-50000 clients with write approximately every 10 seconds during the work day (8 hours), we have to roll out the node, with at least 32vCPU (Intel Xeon Scalable), 40GB RAM and network card with an ability to pass at least 500mb/s. The price of such configuration may vary from 400-500$ on popular cloud solutions from Google Cloud / Amazon. The same scenario with NoSQL database may require about 16vCPU, 10GB RAM and network card with IO about 1GB/s. This configuration will cost at least 150-250$ on the same cloud.



7.1 Decentralized solution

The next problem, which we've faced with was an ability to provide the user the persistence and immutability of his data. To cope with it, a blockchain was introduced as primary data storage (for recorded data).

The blockchain - is a decentralized and distributed ledger. This technology is well known by one of its most famous implementations - bitcoin. The origins of blockchain took its place from traditional financial systems: in most financial systems, the relationship between two clients lie in transactions. For instance, client A sends some money to client B - this action is called transaction, which at least should hold the address of user A, user B and some amount of money. However, the main issue happened to such systems - is that some transactions could be reverted, new money may appear in the system, and as a result, currency may lose its value.

To overcome the issues described above, the concept of blockchain has been introduced. The main idea about it is:

1) Keep every user action in blockchain (each transaction ever made)

2) Move trusted (validated) transactions to blocks

3) Keep chain of blocks, where each new block has a binding to the previous one

4) Maintain the blockchain through special consensus algorithm, which helps nodes to validate transactions, blocks and safely replicate the data between different nodes over the network

5) Provide the fixed amount of money right from the beginning.

So, the workflow is quite simple: from the beginning, the specific blockchain network will have the 0 block (or genesis block), and first transactions, where some accounts will hold all the money, supplied to the network.

From this point, it's important to understand how new users are going to be added to the system: according to the blockchain concept, the user is represented as a unique address. This address is generated from his public key. The public key - is the key, which is used in transaction signing and validation process. The public key is generated from the private key. The private key - also is a unique combination of symbols and numbers. It is usually about 64 characters long. The private key is used to sign the transaction.

Both private and public keys - are used for the signing transaction. This process takes its root from asymmetric cryptography algorithm family. Most blockchain implementations use elliptic curve digital signature as the backbone for safe tx validation.

To better understand how the signing process happens, let's look at the diagram below:

Pic.5. Key pair generation

On this diagram, user A generates his private and corresponding public key. Let's assume, that user B did the same. Now User A wants to send User B the encrypted message:

Pic.6. Asymmetric encryption

According to the following diagram, user A should know the user B public key, to encrypt the message and send it to user B. Under the hood, the crypto curve algorithm generates a special shared secret by user A private key and user B public key. This shared secret is used to encrypt the message:

Pic.7. Asymmetric encryption (extended)

The main idea, lied in asymmetric cryptography, is that in our case, User B will generate the same shared secret and decrypt the message, by using his private key and User A public key.

Pic.8. Message decryption process

Under the hood, the user B private key and User A public key should generate the same shared secret, and as a result, we should be able to decode the message with this shared key.

Returning to tx validation process. It may vary in different blockchain implementations, but the general form of validation usually includes

The central principle lies in 2 points: smart contracts and consensus algorithms. The first one - is for declaring relationships between users (in case you want to pass your recorded dataset to third-party, like a doctor), and the second one - is for organizing decentralized storage with proof of workflow against hijacking or modifying information.

For our purpose, an ethereum + IPFS stack has been chosen, and hyperledger fabric as an alternative.

The primary flow will look like so: for every user in the system, we generate his smart contract with his address. Only this user has access to modify its state. The contract will store only references to collected data. The collected data will be stored in IPFS. On every new push to IPFS, we receive the address of this record. This is what will be stored in the smart contract. Also, we need to point out, that this contract will also have a transparent rights delegation (so only you can push new data to smart contract, but others, who have an address can look at this data, but without performing any modifications). The basic idea of flow is presented below (pic.3.):

Pic.9. Blockchain connector flow

In the case of ethereum, the smart contract is built on solidity language. Below you will find the implementation (pic.9.):

pragma solidity ^0.4.1; contract Storage { struct item { uint timestamp; string payload; } item[] items; function add(string val) { var new_item = item(now, val); items.push(new_item); } function getByIndex(uint index) constant returns (uint timestamp, string str) { if (index >= items.length) { return (0, ""); } return (items[index].timestamp, items[index].payload); } function getIndexesByDate(uint start_date, uint end_date) constant returns (uint[]) { uint[] indexes; for(uint i = 0;i < items.length;i++){ if(items[i].timestamp >= start_date && items[i].timestamp <= end_date){ indexes.push(i); } } return indexes; } function getCount() constant returns (uint) { return items.length; } }

s a result, the storage has an anonymous data and entry points, controlled by an inner mechanism (smart contracts), which make things more straightforward regarding security - as data read/write access is managed by blockchain database, neither than by service itself. This looks like a transparent layer between storing and manipulating data without any fear of losing these points and upload history.

7.2 Off-chain and performance improvement

The blockchain seems to be desirable technology, from the security point of view. However, each technology has its limitations. One of such restrictions (and the most important for the blockchain industry) is lying in transaction processing. According to the general blockchain pattern - each transaction has to be validated before being included in the block.

This approach involves each node in the network - keep the transaction in mempool, validate it, and record to blockchain (locally). The main issue is that the number of transactions is strictly limited to specific blockchain implementation and resources for mining (validation) itself. As a result, in bitcoin, for instance, the block can't be larger than 10 megabytes and couldn't be formed earlier than once per 10 minutes. This limitation leads to slow traffic and processing of user transactions, which is critical for structures like banks, or real-time apps, where the pick of transactions could rise to millions of transactions per second.

The second issue about blockchain - is transaction fee: as for each tx you have to pay. In case, you hold a Dapp (decentralized app, for instance on smart contracts), where all actions performed from your account, each action you have to pay from your own. These become useless since 99% of apps require rapid data update over time.

As a result, these issues limit the usage of the blockchain. However, if we look from another point of view - the blockchain itself could act as permanent storage, while some of the intermediate data may be processed by the third party service. Let's take a simple example: Bob has a relationship with Alice. Alice has a connection with Tom. Let's assume, that Alice want to send some money to Bob, while Bob wants to send some money to Tom. The chain of operations will look like so: Alice->Bob->Tom.

Now let's imagine, that Alice sends to Bob 1 token (or any other cryptocurrency you wish). Now Bob is the holder of this one token. The next thing Bob is going to do is to send this token to Tom. After Bob has sent this token, Bob becomes the holder of this token. Let's have a look at the diagram, which describes this behavior:

Pic.10. Alice to Bob transfer

Pic.11. Bob to Tom transfer



The result of this action is that Alice spent one token and Tom got one token. In other words - the delta of this operation could be described as:

Pic.12. Reduced flow

In systems, where you expect the fixed amount of users (in other words - stack holders), most actions should be predictable. These transfers between a set of users, like in our example, should create a simple delta. This simple delta will replace a broad set of operations with simple delta one, and decrease costs by several times. Also, I would like to admit, that most actions performed on smart contracts have the same case: as the smart contract is just a small program executed on the blockchain, where blockchain only holds its state. The calls to the smart contract are happened as like with the general transaction. However, instead of tokens, you send the encoded action, which you want to perform on the smart contract. You can treat it as with RPC (remote procedure call).

So, now we see, that most of the actions could be reduced just to deltas. However, some system needs to process these deltas and do it on higher speed, than do general blockchain. From this moment, I would like to introduce the off chain tx processing pattern.

The general concept is damn simple: all operations are performed on the simplified version of the blockchain, which doesn't have such complicated consensus rules, and even can have a general SQL/NoSQL DB on the backend. However, in the case of ethereum, the general off chain should have the EVM (their virtual machine) under the hood, to process transactions and form blocks. These blocks are going to be saved in local storage (or just database). The consensus doesn't involve complicated validation, as with off chain we assume, that there won't be public nodes, and they will be available only in single range network. As a result, the off chain acts as a simple, centralized application.

So, the next question, which comes up - what's the point in the blockchain, if we stuck with the centralized application? The answer is simple as the pattern as well: during sync, once per a predefined timeframe, the off chain forms a block with all included transactions and counted deltas. Then off chain push changes and block header to smart contract on on-chain (or blockchain). It's the vital part, as according to this workflow, the blockchain serves as checkpoints. Each delta is recording to blockchain (which means, all stack holders will gain their tokens in a right way), and block, headers - which could be used for additional validation purpose.