IDNet: Smartphone-based Gait Recognition
with Convolutional Neural Networks
Here, we present IDNet, a user authentication framework from smartphone-acquired motion signals. Its goal is to recognize a target user from their way of walking, using the accelerometer and gyroscope (inertial) signals provided by a commercial smartphone worn in the front pocket of the user’s trousers. IDNet features several innovations including: i) a robust and smartphone-orientation-independent walking cycle extraction block, ii) a novel feature extractor based on convolutional neural networks, iii) a one-class support vector machine to classify walking cycles, and the coherent integration of these into iv) a multi-stage authentication technique. IDNet is the first system that exploits a deep learning approach as universal feature extractors for gait recognition, and that combines classification results from subsequent walking cycles into a multi-stage decision making framework. Experimental results show the superiority of our approach against state-of-the-art techniques, leading to misclassification rates (either false negatives or positives) smaller than % with fewer than five walking cycles. Design choices are discussed and motivated throughout, assessing their impact on the user authentication performance.
keywords:Biometric gait analysis, target recognition, classification methods, convolutional neural networks, support vector machines, inertial sensors, feature extraction, signal processing, accelerometer, gyroscope.
Wearable technology is advancing at a very fast pace. Many wearable devices, such as smart watches and wristbands are currently available in the consumer market and they often possess miniaturized inertial motion sensors (accelerometer and gyroscope) as well as other sensing hardware capable of gathering biological signs such as photoplethysmographic signals, skin temperature and so forth. Other wearables, such as commercial physiological monitors, deliver a number of vitals via their wireless interfaces, including electrocardiogram, heart rate, chest motion, etc. The same holds true for recent smartphones, that allow for the collection of user’s feedback and for the realtime assessment of their health condition. They also feature sophisticated sensing technology, among which we consider inertial sensors. With sensing technology growing at a fast pace, two major problems are related to the analysis of wearable data and to the authentication of the mobile users who provide it, so that we can assess with reasonably high accuracy whether the data sources are genuine. Notably, certifying the data sources is a necessary step toward the widespread use of this technology in the medical field and, in this paper, we develop the user authentication technology that is required to make this possible. A great deal of work has been carried out on gait recognition in the last decade sprager2015inertial (). In general, biometric gait recognition can be grouped into three main categories: 1) computer vision based, 2) floor sensor based and 3) wearable sensor based gafurov2007survey (). Most of the recent work belongs to the first category, where image and video analysis are performed to infer the user identity zeng2014silhouette (); luo2016robust (); xing2016complete (); chen2016uncooperative (); choudhury2015robust (). Nevertheless, user authentication from wearables is a sensible approach in those scenarios where the deployment of cameras in not possible.
In this paper, we propose IDNet (IDentification Network), a new system for the authentication of mobile users from smartphone-acquired motion data. As shown in whittle2007analysis (); chan2012evaluating (), modern phones possess highly accurate inertial sensors, which allow for non-obtrusive gait biometrics. IDNet leverages deep Convolutional Neural Networks (CNN) razavian2014computer () and tools from machine learning, such as Support Vector Machines (SVM) Bishop-2007 (), combining them in an innovative fashion. Specifically, we develop algorithms for 1) walking cycle extraction, 2) feature identification and, finally, 3) user authentication. CNNs are used as universal feature extractors to discriminate gait signatures from different subjects. Single- as well as multi-stage classifiers are finally combined with CNNs to authenticate the user through the accumulation of scores from subsequent walking cycles. As shown in Section 4, our solution authenticates the target user with high accuracy and outperforms state-of-the-art techniques such as thang2012gait (); nickel2012authentication (); watanabe2014influence (); choi2014biometric (); ren2013smartphone (); Sprager-15 (). The main contributions of this paper are:
The design and validation of an original preprocessing techniques that includes: a robust algorithm for the extraction of walking cycles and an original transformation to move smartphone acquired motion signals into an orientation invariant reference system. Subsequent processing is carried out within this reference system, as this considerably improves authentication results, see Section 3. As opposed to making motion data orientation independent, previous papers either use data acquired from a sensor in a known and fixed position choi2014biometric (); ren2013smartphone (); chan2013smart (); thang2012gait (); nickel2012authentication (); huang2012gait (); nickel2011scenario (); nickel2011using (), or use orientation independent features at the cost of losing information about the direction of the forces kobayashi2011rotation ().
The design of a new CNN-based feature extraction tool, which is trained only once on a representative set of users and then used at runtime as a universal feature extractor, see Section 4. Note that with CNNs, statistical features are automatically extracted as part of the CNN training phase (automatic feature engineering) as opposed to the selection of predefined and often arbitrary features, as commonly done in the literature watanabe2014influence (); choi2014biometric (); chan2013smart (); nickel2012authentication ().
The combination of CNN-extracted features with a one-class SVM classifier scholkopf2001SVM (), which is solely trained on the target subject, see Section 5. The resulting SVM scores are then accumulated across multiple walking cycles to get higher accuracies, through a new multi-stage identification framework, see Section 6.
The coherent integration of these techniques into the IDNet authentication framework, that uses smartphone-acquired accelerometer and gyroscope motion data. We also show that the integration of gyroscope data provides further performance improvements, see Section 4.2.
2 Related Work
Interest in gait analysis began in the 60’s, when walking patterns from healthy people, termed as normal patterns, were investigated by Murray et al. murray1964walking (). These measurements were performed through the analysis of photos acquired using interrupted light photography. Murray compared normal gait parameters against those from pathologic gaits murray1967gait () and showed that gait is unique to each individual. Since then, human identification through gait recognition has been enjoying a growing interest. Most recent works are based on computer vision nixon2006human (); gafurov2007survey (). Currently, multi-view gait recognition problem and condition invariance (e.g., clothing or carried items, walking speed, view angle, etc.) are of special interest choudhury2015robust (). Many new approaches have been studied to improve recognition performance, such as 3D body estimation luo2016robust (), complete canonical correlation analysis xing2016complete (), sparse coding and hypergraph learning chen2016uncooperative (). However, mobile devices are becoming increasingly sophisticated and can provide high quality inertial measurements. Multiple activities can thus be analyzed using wearable sensors data, and exploited, e.g., for task identification kwapisz2010cell (). A thorough review of the latest developments in this area can be found in Sprager’s work sprager2015inertial ().
Our interest in this paper is in human gait identification through smartphone inertial sensors. Ailisto et al. mantyjarvi2005identifying () were the first to look at this problem and they did it through accelerometer data. In their paper, they used a triaxial accelerometer worn on a belt with fixed orientation: the -axis pointed forward, the -axis to the left and the -axis was aligned with the direction of gravity. Only data points from the and axes were used for identification purposes. Gait cycle extraction was performed through a simple peak detection method, and a template was built for each subject. User identification employed a template matching technique, for which different methods were explored: temporal correlation, frequency domain analysis and data distribution statistics.
In derawi2010unobtrusive (), Derawi et al. proposed more robust preprocessing, cycle detection and template comparison algorithms. Data were acquired using a mobile phone worn on the hip, and only the vertical -axis was considered for motion analysis. Dynamic Time Warping (DTW) DTW-2005 () was used as the distance measure, to ensure robustness against non-linear temporal shifts. This scheme was also tested in nickel2011scenario (), where majority voting and cyclic rotation were compared as inference rules. In a further paper nickel2011using (), Hidden Markov Models (HMM) were explored. Accelerometer data were split into windows of fixed length, which were then utilized to train the HMMs. Good identification results were obtained, but at the cost of long authentication phases ( seconds).
Classification algorithms based on machine learning were also investigated. Either gait cycles extraction juefei2012gait () or fixed windows lengths nickel2012authentication () are possible signal segmentation methods. After that, a feature extraction technique is applied to each segment and statistical measures such as mean, standard deviation, root mean square, zero-crossing rate or histogram bin counts, are commonly used. However, more advanced features are required for better results, like cepstral coefficients, which are widely used for speech recognition nickel2012authentication (), or features extracted through frequency analysis, i.e., using Fourier thang2012gait () or wavelet transforms juefei2012gait (). Supervised algorithms are typically used for classification, including -Nearest Neighbours (-NN) choi2014biometric (); nickel2012authentication (); chan2013smart (); Sprager-15 (), Support Vector Machines (SVM) juefei2012gait (); chan2013smart (); watanabe2014influence (), Multi Layer Perceptrons (MLP) chan2013smart (); watanabe2014influence () and Classification Trees (CT) chan2013smart (); watanabe2014influence ().
Accelerometer based gait analysis has also interest in the medical field. Using time-frequency analysis, Huang et al. showed that signals acquired by a waist-worn device on a patient with cervical disc herniation differed before and after the surgery huang2012gait (). In chan2013smart (), classification algorithms were used to discriminate a group of subjects with non-specific chronic low back pain from healthy subjects. Complex parameters, e.g., dynamic symmetry and cyclic stability of gait, were extracted by Jiang et al. jiang2013possibility (). However, their evaluation requires to place sensors on the legs, and fine gait details are difficult to extract from signals acquired by a single waist-worn sensor.
We stress that in most of the related work the acquisition system was placed according to a controlled and well known orientation on the subject body. In real scenarios, this is however unlikely to be the case. It is thus important to implement an algorithm whose performance is invariant to the smartphone orientation, which is somewhat unconstrained (and unknown). This makes the phone reference system with good probability misaligned with respect to the direction of motion and the definition of subject specific and time invariant templates impossible. To deal with this, two different approaches can be used. The first consists in the extraction of features that are rotation invariant (e.g., correlation matrices of Fourier transforms kobayashi2011rotation () or gait dynamic images zhong2014sensor ()). The second relies on the transformation of inertial signals watanabe2014influence (), projecting them into a new orientation invariant three-dimensional reference system, which is extracted directly from the data. Here, we adopt the latter approach. Another distintive feature of our work is that we use an original processing pipeline exploiting automatic feature extraction through convolutional neural networks and a scoring algorithm combining one-class support vector machines and multi-step decision analysis.
3 Signal Processing Framework
The aim of IDNet is to correctly recognize a subject from his/her way of walking, through the acquisition of inertial signals from a standard smartphone. The proposed processing workflow is shown in Fig. 1. Walking data is first acquired, then we perform some preprocessing entailing:
pre-filtering to remove motion artifacts (Section 3.1),
the extraction of walking cycles (Section 3.2),
a transformation to move the raw walking data into a new orientation independent reference system (Section 3.3),
a normalization to represent each walking cycle (accelerometer and gyroscope data) through fixed length, zero mean and unit variance vectors (Section 3.4).
After this, the walking cycles are ready to be processed to identify the user. The standard approach, called “Classical Machine Learning” entails the computation of a number of pre-established statistical features, the most informative of which are selected and used to train a classifier. Various machine learning techniques are usually exploited to this purpose, and are trained through a supervised approach. Hence, the classification performance is assessed and the whole process is usually iterated for a further feature selection phase. In this way, the features that are used for the classification task are progressively refined until a final feature set is attained. Note that statistical features are often assessed by the designer through educated guesses and a trial and error approach.
As opposed to this, with IDNet we advocate the use of convolutional neural networks (see Sections 4 and 5). These have been successfully used by the video processing community CNN_video_analysis_2014 () but to the best of our knowledge have never been exploited for the analysis of inertial data from wearable devices. One of the main advantages of this approach is that statistical features are automatically assessed by the CNN as a result of a supervised training phase. In Section 4, the CNN is trained to act as a universal feature extractor, whereas in Section 5 a one-class SVM is trained as the final classifier. Once the CNN is trained, our system operates assuming that the smartphone only has access to the walking patterns of the target user (i.e., the legitimate user) and the SVM is solely trained using his/her walking data. Our system is based on the premise that, at runtime, the CNN should be capable of producing discriminant features for unseen users and the one-class SVM, once trained on the target, should reliably detect impostors, although their walks were not used for training.
The processing blocks are described in higher details in the following subsections.
Notation: With we mean a column vector with elements , where is the transpose operator. returns the number of elements in vector . , whereas is the L2-norm operator. If , we define their inner product as and their entrywise product as . Vector with . Matrices are denoted by uppercase and bold letters. For example, if , we define a matrix as , whose rows contain the three vectors. In addition, element of matrix is denoted by . With we mean a 3D vector and is the corresponding 3D versor . For any two 3D vectors and we indicate their cross-product as . The gravity vector is referred to as . With we mean a time series, where is the discrete time index. For acceleration data, the boldface letter is reserved for vectors, for time series and for matrices. The same notation is adopted for the gyroscope data, using , and , respectively for vectors, time series and matrices.
3.1 Data Acquisition and Filtering
A proper dataset is key to the successful design and testing of identity recognition algorithms. Some datasets are publicly available. The largest one was acquired by the Institute of Scientific and Industrial Research (ISIR) at Osaka University (OU) ngo2014largest (). It contains motion data collected from subjects using four motion sensors: three inertial sensors were placed on a belt, with triaxial accelerometer and gyroscope, and a smartphone was worn in the center back waist, and only measured triaxial accelerometer data. Despite the high number of participants, the main problem with this dataset is that motion data was acquired in a controlled environment, and for each subject only two short data sequences are available, which are not enough for deep network training. Furthermore, smartphone’s gyroscope data is not provided. Other datasets are available, but featuring a much smaller number of participants. Casale et al. collected accelerometer data from a smartphone positioned in the chest pocket from users walking over a predefined path casale2012personalization (). In tilmanne2008adatabase (), a motion capture suit was used to acquire data from subjects walking in a small area at different speeds and with direction changes. However, due to the acquisition environment and conditions, these data are more suitable for human gait modeling rather than for user identification. Frank et al. collected data from a mobile phone in the pocket of individuals at McGill University, performing two separate minute walks on two different days frank2010data (). Also in this case gyroscope data is not provided. All these databases do not meet our requirements. In fact, for proper training we need long data collection phases, preferably from different days and with devices freely worn in the user’s front pockets. Hence, we decided to acquire our own motion traces, which are publicly available at http://signet.dei.unipd.it/human-sensing/.
Specifically, we acquired motion data from subjects, during a period of six months using Android smartphones worn in the right front pocket of the users’ trousers. The following devices were used: Asus Zenfone 2, Samsung S3 Neo, Samsung S4, LG G2, LG G4 and a Google Nexus 5. Several acquisition sessions of about five minutes were performed for each subject, in variable conditions, e.g., with different shoes and clothes. We asked each subject to walk as she/he felt comfortable with, to mimic real world scenarios. For the data acquisition, we developed an Android inertial data logger application, which saves accelerometer, gyroscope and magnetometer signals into non-volatile memory and then automatically transfers them to an Internet server for further processing. The magnetometer signal is not used for identification purposes. In general, IDNet can be used carrying the device in other positions but we remark that each requires a dedicated training.
In Fig. 2, we plot the power of accelerometer and gyroscope signals at different frequencies through the Welch’s method Welch-67 (), considering a full walking trace and setting the Hanning window length to s, with half window overlap. Most of the signal power is located at low frequencies, mostly below Hz (where the power is at least dB smaller than the maximum). The raw inertial signals were acquired using an average sample frequency ranging between and Hz (depending on the smartphone model), which is more than appropriate to capture most of the walking signal’s energy.
At the first block of IDNet processing chain, due to the non-uniform sampling performed by the smartphone operating system, we apply a cubic Spline interpolation to represent the input data through evenly spaced points ( points/second). Hence, a low pass Finite Impulse Response (FIR) filter with a cutoff frequency of Hz is used for denoising and to reduce the motion artifacts that may appear at higher frequencies. In fact, given the power profile of Fig. 2, the selected cutoff frequency only removes noise and preserves the relevant (discriminative) information about the user’s motion.
In the following, with and we respectively mean the filtered and interpolated acceleration and gyroscope time series along axis , where is the sample number. The same notation holds for axes and .
3.2 Extraction of Walking Cycles
For the extraction of walking cycles we use a template-based and iterative method that solely considers the accelerometer’s magnitude signal. This signal is in fact inherently invariant to the rotation of the smartphone and, as such, allows for the precise assessment of walking cycles regardless of how the user carries the device in her/his front pocket. For each sample the acceleration magnitude is computed as:
To identify the template, a reference point in has to be located. To do so, inspired by ren2013smartphone () we first pass through a low-pass filter with cutoff frequency Hz. Thus, we detect the first minimum of this filtered signal, which corresponds to the heel strike teixeira2009distributed (), and the corresponding index is called . This minimum is then refined by looking at the original signal in a neighbourhood of and picking the minimum value of in that neighborhood. This identifies a new index for which is a local minimum. As an example, in Fig. 4, we show this minimum through a red vertical (dashed-dotted) line. As a second step, we pick a window of one second centered in , which in Fig. 4 is represented through two vertical blue (dashed) lines. Now, the samples of falling between the two blue lines define the first gait template, which we call , with samples, where corresponds to the number of samples measured in one second.
The extracted template is then iteratively refined and, at the same time, used to identify subsequent walking cycles. To this end, we first define the following correlation distance, for any two real vectors and of the same size we have:
The template is then processed with the acceleration magnitude through the following Eq. (3), leading to a further metric , where is the sample index:
As can be seen from Fig. 4, the function exhibits some local minima, which are promptly located by comparing with a suitable threshold and performing a fast search inside the regions where . The indices corresponding to these minima determine the optimal alignments between the template and . In particular, the second of these identifies the beginning of the second gait cycle. From these facts we have that:
the samples between the second and the third minima correspond to the second gait cycle. It is thus possible to locate accelerometer and gyroscope vectors for this walking cycle, which are respectively defined as: , , and , , , still expressed in the coordinate system of the smartphone. We remark that the number of samples does not necessarily match the template length and usually differs from cycle to cycle, as it depends on the length and duration of walking steps.
A second template is obtained by reading samples starting from the second minimum.
At this point, a new template is obtained through a weighted average of the old template and the new one :
where for the results in this paper we used . The new template is then considered for the extraction of the next walking cycle and the procedure is iterated. Note that this technique makes it possible to obtain an increasingly robust template at each new cycle.
A template matching approach exploiting a similar rationale was used in ren2013smartphone (), where the authors employed the Pearson product-moment correlation coefficient between template and . The main differences between ren2013smartphone () and our approach are: we obtain the template in a neighborhood of , using a fixed number of samples , whereas they take the samples between two adjacent minima of (which may then differ in size for different cycles). In Eq. (4), a discrete-time filter is utilized to refine the template at each walking cycle, making it more robust against speed changes. In previous work ren2013smartphone (), the template is instead kept unchanged up to a point when minima cannot be longer detected, and a new template is to be obtained.
Finally, a normalization phase is required to represent all the cycles through the same number of points , as this is required by the following feature extraction and classification algorithms. Before doing this, however, a transformation of accelerometer and gyroscope signals is performed to express these inertial signals in a rotation invariant reference system, as described next.
3.3 Orientation Independent Transformation
To evaluate the new orientation invariant coordinate system, three orthogonal versors , , are to be found, whose orientation is independent of that of the smartphone and aligned with gravity and the direction of motion. Specifically, our aim is to express accelerometer and gyroscope signals in a coordinate system that remains fixed during the walk, with versor pointing up (and parallel to the user’s torso), versor pointing forward (aligned with the direction of motion) and tracking the lateral movement and being orthogonal to the other two. This entails inferring the orientation of the mobile device carried in the front pocket from the acceleration signal acquired during the walk. To this end, we adopt a technique similar to those of pca-heading-2009 (); heading-2015 ().
Gravity is the main low frequency component in the accelerometer data, and will be our starting point for the transform. Moreover, although it is a constant vector, it continuously changes when represented in the coordinate system of the smartphone, due to the user’s mobility and the subsequent change of orientation of the device. So, even the gravity vector is not constant when expressed through the smartphone coordinates. As the first axis of the new reference system, we consider the mean direction of gravity within the current walking cycle. Let be the number of samples in the current cycle , with . We recall that, with , and we mean the acceleration samples in the current cycle along the three axes and , with , whereas with , and we indicate the gyroscope samples in the same cycle , with . The gravity vector within cycle is estimated as:
The first versor of the new system is obtained as:
Now, we define the acceleration matrix of size , whose rows corresponds to , and . Likewise, the gyroscope matrix is , whose rows corresponds to , and . The projected acceleration and gyroscope vectors along axis are:
where the new vectors have the same size . By removing this component from the original accelerometer signal, we project the latter on a plane that is orthogonal to . This is the horizontal plane (parallel to the floor). We represent this flattened acceleration data through a new matrix of size , where , and are vectors of size that describe the acceleration on the new plane:
Analyzing this flattened acceleration, we see that during a walking cycle it is unevenly distributed on the horizontal plane. Also, the acceleration points on this plane are dispersed around a preferential direction, which has the highest excursion (variance). Here, we assume that the direction with the largest variance in our measurement space contain the dynamics of interest, i.e., it is parallel to the direction of motion, as it was also observed and verified in previous research pca-heading-2009 (). Given this, we pick this direction as the second axis (versor ) of the new reference system. This is done by applying the Principal Component Analysis (PCA) PCA-64 () on the projected points, which finds the direction along which the variance of the measurements is maximized. The PCA procedure is as follows:
Find the empirical mean along each direction , and (rows , and of the flattened acceleration matrix ). Store the mean in a new vector of size ., i.e.:
Subtract the empirical mean vector from each column of matrix , obtaining the new matrix :
Compute the sample autocovariance matrix :
The eigenvalues and the corresponding eigenvectors of are evaluated. The eigenvector associated with the maximum eigenvalue identifies the direction of maximum variance in the dataset (i.e., the first principal component of the PCA transform).
Hence, versor is evaluated as:
Accelerometer and gyroscope data are then projected along through the following equations: and . Being placed on a plane that is orthogonal to , these two versors are also orthogonal. The third axis is then obtained through a cross product:
and the new accelerometer and gyroscope data along this axis are respectively obtained as: and . The transformed vectors and , along with the magnitude vectors and are the output of the Orientation Independent Transformation block of Fig. 1.
An example of this transform is shown in Fig. 5, where accelerometer and gyroscope data from two different walks from the same subject are plotted. These signals were acquired carrying the phone in the right front pocket of the subject’s trousers using two different orientations. As highlighted in the figure, our transform makes walking data rotation invariant. In fact, subject-specific gait patterns emerge in the new coordinate system (see the red colored patterns in the right plots).
Each gait cycle has a different duration, which depends on the walking speed and stride length. So, considering the accelerometer and gyroscope data collected during a full walking cycle, we remain with variable-size acceleration and gyroscope vectors, which are now expressed in the new orientation invariant coordinate system discussed in Section 3.3. However, since feature extraction and classification algorithms require -sized vectors for each cycle, where has to be fixed, a further adjustment is necessary. We cope with this cycle length variability through a further Spline interpolation to represent all walking cycles through vectors of samples each. This specific value of was selected to avoid aliasing. In fact, assuming a maximum cycle duration of seconds, which corresponds to a very slow walk, and a signal bandwidth of Hz, a number of samples samples/cycle is required. Amplitude normalization was also implemented, to obtain vectors with zero mean and unit variance, as this leads to better training and classification performance. This results in a total of eight -sized vectors for each walking cycle, which are inputted into the feature extraction and classification algorithms of the following sections.
4 Convolutional Neural Network
In this section, we present the chosen Convolutional Neural Network (CNN) architecture for IDNet (Section 4.1), its optimization, training and quantitative comparison against gait authentication techniques from the literature (Section 4.2).
4.1 CNN Architecture
CNNs are feed-forward deep neural networks differing from fully connected multilayer networks for the presence of one or more convolutional layers. At each convolutional layer, a number of kernels is defined. Each of them has a number of weights, which are convolved with the input in a way that the same set of weights, i.e., the same kernel, is applied to all the input data, moving the convolution operation across the input span. Note that, as the same weights are reused (shared weights), and each kernel operates on a small portion of the input signal, it follows that the network connectivity structure is sparse. This leads to advantages such as a considerably reduced computational complexity with respect to fully connected feed forward neural networks. For more details the reader is referred to lecun1998convolutional (). CNNs have been proven to be excellent feature extractors for images krizhevsky2012imagenet () and here we prove their effectiveness for motion data. The CNN architecture that we designed to this purpose is shown in Fig. 6. It is composed of a cascade of two convolutional layers, followed by a pooling and a fully-connected layer. The convolutional layers perform a dimensionality reduction (or feature extraction) task, whereas the fully-connected one acts as a classifier. Accelerometer and gyroscope data from each walking cycle are processed according to the algorithms of Section 3. We refer to the input matrix for a generic walking cycle to as , where all the vectors are normalized to samples (see Section 3.4). In detail, we have (CL = Convolutional Layer, FL = Fully-connected Layer):
CL1 The first convolutional layer implements one dimensional kernels (x samples) performing a first filtering of the input and processing each input vector (rows of ) separately. This means that at this stage we do not capture any correlation among different accelerometer and gyroscope axes. The activation functions are linear and the number of convolutional kernels is referred to as .
CL2 With the second convolutional layer we seek discriminant and class-invariant features. Here, the cross-correlation among input vectors is considered (kernels of size x samples) and the output activation functions are non-linear hyperbolic tangents. Max pooling is applied to the output of CL2 to further reduce its dimensionality and increase the spatial invariance of features scherer2010evaluation (). With we mean the number of convolutional kernels used for CL2.
FL1 This is a fully connected layer, i.e., each output neuron of CL2 is connected to all input neurons of this layer (weights are not shared). Hyperbolic tangent activation functions are used at the output neurons. FL1 output vector is termed , and contains the features extracted by the CNN.
FL2 Each output neuron in this layer corresponds to a specific class (one class per user), for a total of neurons, where is the number of subjects considered for the training phase. The dimensional output vector is obtained by a softmax activation function, which implies that , and (stochastic vector). Also, can be thought of as the probability that the current data matrix belongs to class (user) .
The network is trained in a supervised manner for a total of subjects solving a multi-class classification problem, where each of the input matrices in the dataset is assigned to one of mutually exclusive classes. The target output vector has binary entries and is encoded using a -of- coding scheme, i.e., they are all zero except for that corresponding to the subject that generated the input data.
4.2 CNN Optimization and Results
In this section, we propose some approaches for the optimization of the CNN, quantify its classification performance and compare it against classification techniques from the literature. As said above, the output of layer FL2 is the stochastic vector , whose -th entry , , can be seen as the probability that the input pattern belongs to user , i.e., , where is the vector containing all the CNN weights, is the current input matrix (walking cycle) and if belongs to class and otherwise. If is the set of all training examples, we define the batch set as . Let and denote the corresponding output vector by and its -th entry by . The corresponding target vector is . The CNN is then trained through a stochastic gradient descend algorithm which minimizes a categorical cross-entropy loss function , defined as (Bishop-2007, , Eq. (5.24) of Section 5.2):
During training, Eq. (14) is iteratively minimized, by rotating the walking cycles (training examples) in the batch set so as to span the entire input set . Training continues until a stopping criterion is met (see below).
Walking patterns from subjects are used to train the CNN, and the same number of cycles is considered for each of them, for a total of training cycles. randomly chosen walking cycles from each subjects are used to obtain a test set . The remaining cycles are split into training and validation sets, with , , , where all the sets have null pairwise intersection and are built picking input patterns from evenly at random. Set is used to terminate the training phase, and termination occurs when the loss function evaluated on does not decrease for twenty consecutive training epochs. After that, the network weights which led to the minimum validation loss are used to assess the CNN performance on set . This is done through an accuracy measure, defined as the number of walking cycles correctly classified by the CNN divided by the total number of cycles in . In the following graphs, we show the mean accuracy obtained averaging the test set performance over ten different networks, all of them trained through the just explained approach by considering subjects from our dataset and cycles per subject.
As a first set of results, we look at the impact of (neurons in layer ) and of the number of convolutional kernels in CL1 and CL2. Since the last layer FL2 acts as a classifier, can be seen as the number of features extracted by the CNN. In general, a too small can lead to poor classification results; too many features, instead, would make the state space too big to be effectively dealt with (curse of dimensionality) hanka1997curse (). Besides , we also investigate the right number of kernels to use within each convolutional layer. Three networks are considered by picking different pairs. For network 1 we use , network 2 has and network 3 has . In Fig. 8, we show the accuracy performance of these networks as a function of . From this plot, it can be seen that at least neurons have to be used at the output of FL1 and that the accuracy performance stabilizes around , leading to negligible improvements as grows beyond this value. As for the number of kernels, we conclude that small networks (network 1) perform worse than bigger ones (networks 2 and 3), but increasing the number of kernels beyond that used for network 2 does not lead to appreciable improvements. Hence, for the results of this paper we used with .
Comparison against existing techniques: in Fig. 8, the accuracy is plotted against for our CNN-based approach and four selected authentication algorithms from the literature, featuring classifiers based on Classification Trees (CT) quinlan1993c45 (), Naive Bayes (NB) friedman1997bayesian (), -Nearest Neighbors (-NN) cover1967nearest () and Support Vector Machines (SVM) cortes1995support ().111For SVM, we considered a linear kernel, as it outperformed polynomial and radial basis function ones (results are omitted in the interest of space). A one-versus-all strategy was used solve the considered multiclass problem for the binary classifers. These techniques were used in a large number of papers including choi2014biometric (); nickel2012authentication (); juefei2012gait (); chan2013smart (); watanabe2014influence (). For their training, features were extracted from the signal samples in , including their variance, mean trend, windowed mean difference, variance trend, windowed variance difference, maxima and minima, spectral entropy, zero crossing rate and bin counts. These features, were then utilized to train the selected classifiers in a supervised manner. Note that, while the CNN automatically extracts its features (vector ), with previous schemes these are manually selected based on experience.
From Fig. 8, we see that the CNN-based algorithm delivers better accuracies across the entire range of . Also, the accuracy increases with an increasing until it saturates and no noticeable improvements are observed. While a higher is always beneficial, a higher number of cycles also entails a longer acquisition time, which we would rather avoid. For this reason, for the following results we have used as it provided a good tradeoff between accuracy and complexity across all our experiments.
To illustrate the superiority of CNN features with respect to manually extracted ones, in the following we conduct an instructive experiment. We consider CNN as a feature extraction block, by removing the output vector and using the inner feature vector to train the above classifiers from the literature (CT, NB, -NN and SVM). The corresponding accuracy results are provided in Fig. 10. All the classifiers perform better when trained using CNN features, with typical improvements in the test accuracy of more than %. For instance, for a -NN classifier trained with cycles per subject, the accuracy increases from % (manually extracted features) to (CNN features). The best performance is provided by the combined use of CNN features and SVM.
A last consideration is in order. Most of the previous papers only used accelerometer data, but our results show that using both gyroscope and accelerometer provides further improvements, see Fig. 10.
5 One-Class Support Vector Machine Training
In this section, we further extend the IDNet CNN-based authentication chain through the design of an SVM classifier which is trained solely using the motion data of the target subject. This is referred to as One-Class Classification (OCC) and is important for practical applications where motion signals of the target user are available, but those belonging to other subjects are not. More importantly, with this approach the classification framework can be extended to users that were not considered in the CNN training.
5.1 Revised Classification Architecture
Due to the generalization properties of convolutional deep networks, once trained, the CNN can be used as a universal feature extractor, providing meaningful features even for subjects that were not included in the training. To take advantage of this, we discard the output neurons of FL2 and utilize the CNN as a dimensionality reduction tool that, given an input matrix , returns a user dependent feature vector . The CNN is then trained only once considering the optimizations of Section 4.2. All its weights and biases are then precomputed and will not be modified at classification time. Considering the diagram of Fig. 6, at the output of the CNN we obtain the feature vector . We then apply a feature selection block to reduce the number of features from to (dimensionality reduction). PCA is used to accomplish this task and the new feature vector is called . Hence, we have , where is the PCA transform.
A One-class Support Vector Machine (OSVM) is then used as the classification algorithm (Section 5.2). It defines a boundary around the feature (training) vectors belonging to the target subject. At runtime, as a new walking cycle is processed, the OSVM takes the feature vector and outputs a score, which is a distance measure between the current feature vector and the SVM boundary (Bishop-2007, , Chapter 7). As we discuss shortly, this score relates to the likelihood that the current walking cycle belongs to the target user.
5.2 One-Class SVM Design
Next, we design the OSVM block of Fig. 6. It differs from a standard binary SVM classifier as the SVM boundary is built solely using patterns from the positive class (target user). The strategy proposed by Schölkopf is to map the data into the feature space of a kernel, and to separate them from the origin with maximum margin scholkopf1999support (). The corresponding minimization problem is similar to that of the original SVM formulation cortes1995support (). The trick is to use a hyperplane (in the space transformed by a suitable kernel function) to discriminate the target vectors. OSVM takes as input the reduced feature vector and we use the following Radial Basis Function (RBF) kernel, that for any is defined as:
where is a feature map and is the RBF kernel parameter, which intuitively relates to the radius of influence that each training vector has for the space transformation. With we mean the number of training points (feature vectors), and are the hyperplane parameters in the transformed domain (through Eq. (15)) and is the vector of slack variables, which are introduced to deal with outliers. Given this, the following quadratic program is defined to separate the feature vectors in the training set, , from the origin:
is one of the most important parameters and sets an upper bound on the fraction of outliers and a lower bound on the fraction of Support Vectors (SV) scholkopf1999support (). The decision function for a generic feature vector is defined as , is obtained solving Eq. (16), and only depends on the training vectors through the following relations:
Now, , and only some of the training vectors have . These are the support vectors associated with the classification problem and are the only ones who count in the definition of the SVM boundary. is the score associated with vector . It weighs the distance from the SVM boundary, i.e., is greater than zero if resides inside the boundary, zero if it lies on it and negative otherwise.
Hence, the SVM is trained using a set of feature vectors from the target user, obtaining the SVM boundary (and the related decision function) through Eq. (17). After training, we test the performance of the obtained SVM classifier considering feature vectors from the positive class (target user) and the negative one (any other user). Note that the vectors used for this test were not considered during the SVM training.
As it is customary for binary classification approaches, the two most important metrics to assess the goodness of a classifier are the precision and the recall. The precision is the fraction of true positives, i.e., the fraction of patterns identified of the target class that in fact belong to the target user, while the recall corresponds to the fraction of target patterns that are correctly classified out of the entire positive class of samples Fmeasure-2003 (). Often, these two metrics are combined into their harmonic mean, which is called F-measure and is used as the single quality parameter.
In Fig. 11, the F-measure is plotted as a function of the two SVM parameters and . As seen from this plot, the area where the classifier’s performance is maximum is quite ample. This is good as it means that even selecting and once for all at design stage, the performance of the SVM classifier is not expected to change much if the signal statistics changes or a new target user is considered. In other words, this relatively weak dependence on the parameters entails an intrinsic robustness for the classifier. For the results that follow we have used and .
Two last considerations are in order. The first relates to the PCA transformation and in particular to how many and which principal components have to be retained for the output feature vectors. In fact, as pointed out in tax2003artificial (), two options are possible to go from the CNN-extracted feature vector to . The first is to retain the entries of the transformed vector (expressed in the PCA basis) that correspond to the principal components with highest variance, whereas a second option is to retain those with the smallest. Fig. 13 shows the F-measure of the OSVM classifier as a function of for (number of CNN-extracted features). From this plot we see that picking in general provides better results and also that considering the principal components with lowest variance provides better results for this class of problems. This is in accordance with tax2003artificial ().
The last consideration regards the amount of feature vectors belonging to the target user that should be used for the OSVM training. Note that this number is related to the walking time required for a new subject to train his/her personal authentication system. To perform this analysis, a fixed number of cycles were randomly extracted from the whole target dataset and were used to train the OSVM. The remaining walking cycles were used as the positive test set. In Fig. 13 we show the F-measure as a function of this number of cycles. From these results, it follows that increasing the number of cycles beyond leads to little improvement. This number corresponds to about minutes of walking activity, distributed among different acquisition sessions. Multiple sessions are recommended to account for some statistical variation due to wearing different clothes.
Once all the model’s parameters are defined, the OSVM score can be analyzed. Let be the estimated probability density function (pdf) of the OSVM score , provided that the walking cycle belongs to a user of class with . Empirical pdfs from our dataset are provided in Fig. 14.
6 Multi-Stage Authentication
The so far discussed processing pipeline returns a score for each walking cycle. However, as seen in Fig. 14, when a score falls near the point where the two pdfs intersect, there is a high uncertainty about the identity of the user who generated it. In IDNet, we resolve this indetermination by jointly considering the scores from successive walking cycles. Let be a sequence of subsequent OSVM scores from the same subject, where and is the walking cycle index. From our previous analysis, can be thought of as a random process having probability density function , , and our objective is to reliably estimate from the scores in . Toward this, we assume that subsequent scores belong to the same user and that they are independent and identically distributed (i.i.d), i.e., they are independently drawn from , with unknown.
For the estimation of we use Wald’s probability ratio test (SPRT) Wald-1947 (); Tartakovsky-2015 (). We define the two hypotheses , meaning that the sequence belongs to the target user (class ), and , meaning that another user generated it (class ). Hence, we assess which one of these is true through SPRT sequential binary testing. That is, we keep measuring new scores and use them to decrease our uncertainty about . Considering samples , the final decision takes on two values or , where , means that hypothesis is accepted and therefore the alternative hypothesis is rejected. Owing to our assumptions (i.i.d. scores, generated by the same subject), for scores the joint pdf is:
Defining , the likelihood ratio of the sequence truncated at index , , is
and applying the logarithm, we get:
If we wait a further step before making a decision, from Eq. (20) the new log-likelihood is conveniently obtained as . The SPRT test starts from time , obtaining one-class OSVM scores for each successive walking cycle. After cycles, the cumulative log-likelihood ratio is , with . Two suitable thresholds and are defined and the test continues to the next cycle if , is accepted if , whereas is accepted if . Moreover, defining as the probability of accepting when is true and that of accepting when is true, and can be approximated as: and , see Wald-1947 ().
6.1 Experimental Results
The motion data from subjects was used to train the CNN feature extractor, with , and . One user out of the remaining was considered as the target user and as the negatives for the final tests. The following results are obtained through a leave-one-out cross-validation approach for the sessions of the target user, i.e., out of twelve sessions, eleven are used for training and one for the final tests. The session that is left out is rotated and the final results are averaged across all trials. The authentication results of the multi-stage framework are shown in Fig. 15. False positive rates (i.e., a user is mistakenly authenticated as the target) and false negative ones (i.e., the target is not recognized) are smaller than % for an appropriate choice of the SPRT thresholds ( and ). Also, a reliable authentication requires fewer than five walking cycles in % of the cases. This means that the framework is very accurate and at the same time fast. We remark that the best authentication results that were obtained in previous papers lead to error rates ranging from to % thang2012gait (); nickel2012authentication (); watanabe2014influence (); choi2014biometric (); ren2013smartphone (); Sprager-15 (). A comparison with these approaches is very difficult to carry out due to the different datasets (e.g., number of subjects and walking time), acquisition settings (e.g., smartphone or sensors location). The reader can nevertheless refer to Section 4.2 for a fair comparison between our single-step classification framework and classical feature extraction techniques on our dataset.
As for our assumptions, in light of the small number of cycles required, it is reasonable to presume that the same subject generates the scores in . For the i.i.d. assumption, we extended the decision framework to the first-order autoregressive model of (Tartakovsky-2015, , Chapter 3, p. 158), which allows tracking the correlation across successive cycles. However, this did not lead to any appreciable performance improvement and only implied a higher complexity. The reason is that scores are lightly correlated in time.
In this paper we have proposed IDNet, a user authentication framework for inertial signals acquired from smartphones. Various schemes performing manual feature extraction and using the selected features for user classification have appeared in the recent literature. In sharp contrast with these, IDNet exploits convolutional neural networks, as they allow for an automatic feature engineering and have excellent generalization capabilities. These deep neural networks are then used as universal feature extractors to feed classification techniques, combining them with one-class support vector machines and a novel multi-stage decision algorithm. With our framework, the neural network is trained once for all and subsequently utilized for new users. The one-class classifier is solely trained using motion data from the target subject; it returns a score weighing the dissimilarity of newly acquired data with respect to that of the target. Subsequent scores are then accumulated through a multi-stage decision approach. Experimental results show the superiority of IDNet against prior work, leading to misclassification rates smaller than % in fewer than five walking cycles. Design choices and the optimization of the various processing blocks were discussed and compared against existing algorithms.
- (1) S. Sprager, M. B. Juric, Inertial sensor-based gait recognition: A review, Sensors 15 (9) (2015) 22089.
- (2) D. Gafurov, A survey of biometric gait recognition: Approaches, security and challenges, in: Annual Norwegian computer science conference, Oslo, Norway, 2007.
- (3) W. Zeng, C. Wang, F. Yang, Silhouette-based gait recognition via deterministic learning, Pattern Recognition 47 (11) (2014) 3568–3584.
- (4) J. Luo, J. Tang, T. Tjahjadi, X. Xiao, Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis, Pattern Recognition 60 (2016) 361–377.
- (5) X. Xing, K. Wang, T. Yan, Z. Lv, Complete canonical correlation analysis with application to multi-view gait recognition, Pattern Recognition 50 (2016) 107–117.
- (6) X. Chen, J. Xu, Uncooperative gait recognition: Re-ranking based on sparse coding and multi-view hypergraph learning, Pattern Recognition 53 (2016) 116–129.
- (7) S. D. Choudhury, T. Tjahjadi, Robust view-invariant multiscale gait recognition, Pattern Recognition 48 (3) (2015) 798–811.
- (8) M. W. Whittle, Gait Analysis: An Introduction, 4th ed., Elsevier: Edinburgh, UK, 2008.
- (9) H. Chan, H. Zheng, H. Wang, R. Sterritt, Evaluating and overcoming the challenges in utilizing smart mobile phones and standalone accelerometer for gait analysis, in: IET Irish Signals and Systems Conference (ISSC 2012), Maynooth, Ireland, 2012.
- (10) A. S. Razavian, H. Azizpour, J. Sullivan, S. Carlsson, CNN Features Off-the-Shelf: An Astounding Baseline for Recognition, in: IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, Ohio, US, 2014.
- (11) C. Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
- (12) H. M. Thang, V. Q. Viet, N. D. Thuc, D. Choi, Gait identification using accelerometer on mobile phone, in: International Conference on Control, Automation and Information Sciences (ICCAIS), Saigon, Vietnam, 2012.
- (13) C. Nickel, T. Wirtl, C. Busch, Authentication of smartphone users based on the way they walk using k-nn algorithm, in: International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), Piraeus-Athens, Greece, 2012.
- (14) Y. Watanabe, Influence of Holding Smart Phone for Acceleration-Based Gait Authentication, in: International Conference on Emerging Security Technologies (EST), Houston, Texas, US, 2014.
- (15) S. Choi, I. H. Youn, R. LeMay, S. Burns, J. H. Youn, Biometric gait recognition based on wireless acceleration sensor using k-nearest neighbor classification, in: International Conference on Computing, Networking and Communications (ICNC), Honolulu, Hawaii, US, 2014.
- (16) Y. Ren, Y. Chen, M. C. Chuah, J. Yang, Smartphone based user verification leveraging gait recognition for mobile healthcare systems, in: IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON), New Orleans, Louisiana, US, 2013.
- (17) S. Sprager, M. B. Juric, An Efficient HOS-Based Gait Authentication of Accelerometer Data, IEEE Transactions on Information Forensics and Security 10 (7) (2015) 1486–1498.
- (18) H. Chan, H. Zheng, H. Wang, R. Sterritt, D. Newell, Smart mobile phone based gait assessment of patients with low back pain, in: Ninth International Conference on Natural Computation (ICNC), San Diego, California, US, 2013.
- (19) G.-S. Huang, C. C. Wu, J. Lin, Gait analysis by using tri-axial accelerometer of smart phones, in: International Conference on Computerized Healthcare (ICCH), Hong Kong, China, 2012.
- (20) C. Nickel, M. O. Derawi, P. Bours, C. Busch, Scenario test of accelerometer-based biometric gait recognition, in: International Workshop on Security and Communication Networks (IWSCN), Gjøvik, Norway, 2011.
- (21) C. Nickel, C. Busch, S. Rangarajan, M. Mobius, Using hidden markov models for accelerometer-based biometric gait recognition, in: IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA), Penang, Malaysia, 2011.
- (22) T. Kobayashi, K. Hasida, N. Otsu, Rotation invariant feature extraction from 3-D acceleration signals, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, 2011.
- (23) B. Schölkopf, J. C. Platt, J. C. Shawe-Taylor, A. J. Smola, R. C. Williamson, Estimating the support of a high-dimensional distribution, Neural Computation 13 (7) (2001) 1443–1471.
- (24) M. P. Murray, A. B. Drought, R. C. Kory, Walking patterns of normal men, The Journal of Bone & Joint Surgery 46 (2) (1964) 335–360.
- (25) M. P. Murray, Gait as a total pattern of movement: Including a bibliography on gait., American Journal of Physical Medicine & Rehabilitation 46 (1) (1967) 290–333.
- (26) T. Nixon, M. S. ans Tieniu, C. Rama, Human identification based on gait, Springer, 2006.
- (27) J. R. Kwapisz, G. M. Weiss, S. A. Moore, Cell phone-based biometric identification, in: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), 2010.
- (28) J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S. M. Makela, H. A. Ailisto, Identifying users of portable devices from gait pattern with accelerometers, in: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Philadelphia, Pennsylvania, US, 2005.
- (29) M. O. Derawi, C. Nickel, P. Bours, C. Busch, Unobtrusive user-authentication on mobile phones using biometric gait recognition, in: 6th International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), Darmstadt, Germany, 2010.
- (30) E. Keogh, C. Ratanamahatana, Exact indexing of dynamic time warping, Knowledge and Information Systems 7 (3) (2005) 358–386.
- (31) F. Juefei-Xu, C. Bhagavatula, A. Jaech, U. Prasad, M. Savvides, Gait-id on the move: Pace independent human identification using cell phone accelerometer dynamics, in: Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), Washington DC, US, 2012.
- (32) S. Jiang, B. Zhang, G. Zou, D. Wei, The possibility of normal gait analysis based on a smart phone for healthcare, in: IEEE International Conference on Green Computing and Communications (GreenCom), Internet of Things (iThings), and Cyber, Physical and Social Computing (CPSCom), Beijing, China, 2013.
- (33) Y. Zhong, Y. Deng, Sensor orientation invariant mobile gait biometrics, in: IEEE International Joint Conference on Biometrics (IJCB), Clearwater, FL, USA, 2014.
- (34) A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, L. Fei-Fei, Large-scale Video Classification with Convolutional Neural Networks, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, US, 2014.
- (35) T. T. Ngo, Y. Makihara, H. Nagahara, Y. Mukaigawa, Y. Yagi, The largest inertial sensor-based gait database and performance evaluation of gait-based personal authentication, Pattern Recognition 47 (1) (2014) 228–237.
- (36) P. Casale, O. Pujol, P. Radeva, Personalization and user verification in wearable systems using biometric walking patterns, Personal and Ubiquitous Computing 16 (5) (2012) 563–580.
- (37) J. Tilmanne, R. Sebbe, T. Dutoit, A database for stylistic human gait modeling and synthesis (2008).
J. Frank, S. Mannor, D. Precup,
Mobile phone gait recognition data (2010).
- (39) P. D. Welch, The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms, IEEE Transactions on Audio and Electroacoustics 15 (2) (1967) 70–73.
- (40) T. Teixeira, D. Jung, G. Dublon, A. Savvides, Pem-id: Identifying people by gait-matching using cameras and wearable accelerometers, in: ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), Como, Italy, 2009.
- (41) K. Kunze, P. Lukowicz, K. Partridge, B. Begole, Which Way Am I Facing: Inferring Horizontal Device Orientation from an Accelerometer Signal, in: IEEE International Symposium on Wearable Computers, Linz, Austria, 2009.
- (42) Z.-A. Deng, G. Wang, Y. Hu, D. Wu, Heading Estimation for Indoor Pedestrian Navigation Using a Smartphone in the Pocket, MDPI Sensors 15 (9) (2015) 21518–21536.
- (43) C. R. Rao, The Use and Interpretation of Principal Component Analysis in Applied Research, Sankhyā: The Indian Journal of Statistics 26 (4) (1964) 329–358.
- (44) Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series, in: The Handbook of Brain Theory and Neural Networks, MIT Press, 1998, pp. 255–258.
- (45) A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems 25, 2012, pp. 1106–1114.
- (46) D. Scherer, A. Müller, S. Behnke, Evaluation of pooling operations in convolutional architectures for object recognition, in: 20th International Conference on Artificial Neural Networks (ICANN), Thessaloniki, Greece, 2010.
- (47) R. Hanka, T. P. Harte, Computer Intensive Methods in Control and Signal Processing: The Curse of Dimensionality, Birkhäuser Boston, 1997, Ch. Curse of Dimensionality: Classifying Large Multi-Dimensional Images with Neural Networks, pp. 249–260.
- (48) J. R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, California, US, 1993.
- (49) N. Friedman, D. Geiger, M. Goldszmidt, Bayesian network classifiers, Machine Learning 29 (2) (1997) 131–163.
- (50) T. Cover, P. Hart, Nearest neighbor pattern classification, IEEE Transactions on Information Theory 13 (1) (1967) 21–27.
- (51) C. Cortes, V. Vapnik, Support-vector networks, Machine Learning 20 (3) (1995) 273–297.
- (52) B. Schölkopf, R. C. Williamson, A. J. Smola, J. Shawe-Taylor, J. C. Platt, et al., Support vector method for novelty detection, Neural Information Processing Systems (NIPS) 12 (1999) 582–588.
- (53) D. R. Musicant, V. Kumar, A. Ozgur, Optimizing F-Measure with Support Vector Machines, in: 16-th International FLAIRS Conference, FLAIRS, St. Augustine, Florida, US, 2003.
- (54) D. M. J. Tax, K. R. Müller, Artificial Neural Networks and Neural Information Processing, Springer, Berlin, Heidelberg, 2003, Ch. Feature Extraction for One-Class Classification, pp. 342–349.
- (55) A. Wald, Sequential analysis, Dover, New York, NY, US, 1947.
- (56) A. Tartakovsky, I. Nikiforov, M. Basseville, Sequential Analysis Hypothesis Testing and Changepoint Detection, CRC Press, 2015.