GraphRQI: Classifying Driver Behaviors Using Graph Spectrums

GraphRQI: Classifying Driver Behaviors Using Graph Spectrums

Rohan Chandra, Uttaran Bhattacharya, Trisha Mittal, Xiaoyu Li, Aniket Bera and Dinesh Manocha
University of Maryland Cornell University

We present a novel algorithm (GraphRQI) to identify driver behaviors from road-agent trajectories. Our approach assumes that the road-agents exhibit a range of driving traits, such as aggressive or conservative driving. Moreover, these traits affect the trajectories of nearby road-agents as well as the interactions between road-agents. We represent these inter-agent interactions using unweighted and undirected traffic graphs. Our algorithm classifies the driver behavior using a supervised learning algorithm by reducing the computation to the spectral analysis of the traffic graph. Moreover, we present a novel eigenvalue algorithm to compute the spectrum efficiently. We provide theoretical guarantees for the running time complexity of our eigenvalue algorithm and show that it is faster than previous methods by 2 times. We evaluate the classification accuracy of our approach on traffic videos and autonomous driving datasets corresponding to urban traffic. In practice, GraphRQI achieves an accuracy improvement of up to over prior driver behavior classification algorithms. We also use our classification algorithm to predict the future trajectories of road-agents.

I Introduction

Autonomous driving is an active area of research [ad1, ad2] and includes many issues related to perception, navigation, and control [ad2]. While some of the earlier work was focused on autonomous driving in mostly empty roads or highways, there is considerable recent work on developing technologies in urban scenarios. These urban scenarios consist of a variety of road-agents corresponding to vehicles, buses, trucks, bicycles, and pedestrians. In order to perform collision-free navigation, it is important to track the location of these road-agents [chandra2019roadtrack] and predict their trajectory in the near future [pendleton2017perception, schwarting2018planning, social-lstm, social-gan, chandra2019traphic, chandra2019robusttp]. These tasks can be improved with a better understanding and prediction of driver behavior and performing behavior-aware planning [ernest, ernest_nav, schwarting2018planning, paden2016survey]. For example, drivers can navigate dangerous traffic situations by identifying the aggressive drivers and avoiding them or keeping a distance. At the same time, there is a clear distinction between such driver behaviors and maneuver-based road-agent behaviors [pseudobehavior1, honda, pseudobehavior2, pseudobehavior3]. According to studies in traffic psychology [humanfactor1, humanfactor2, humanfactor3], driver behavior corresponds to labels for various driving traits, such as aggressive and conservative driving [humanfactor1, humanfactor2, humanfactor3]. These studies further show that traffic flow and road-agent trajectories are significantly affected by driver behaviors. For instance, an aggressive road-agent overtaking a vehicle may cause the vehicle to slow down, or an overly cautious driver may drive slowly and cause congestion.

Our goal is to classify the drivers according to their behaviors in a given traffic video. The two most widely used behavior labels used to classify drivers are aggressive and conservative [rohanref3, rohanref4, rohanref5]. Recent studies have also explored more granular behaviors, such as reckless, impatient, careful and timid [rohanref6-reckless, rohanref7-timid]. For instance, impatient drivers show different traits (e.g., over-speeding, overtaking) than reckless drivers (e.g., driving in the opposite direction to the traffic) [rohanref6-reckless], but both can be regarded as aggressive. Similarly, timid drivers may drive slower than the speed limit [rohanref7-timid], while careful drivers obey the traffic rules. However, both of them are marked as conservative drivers. In this paper, we consider a total of six different classes of driver behavior — impatient, reckless, threatening, careful, cautious, and timid [ernest]. For the rest of the paper, we collectively refer to the first three as aggressive driving, while the last three are categorized as part of conservative driving.

Figure 1: GraphRQI predicts the behaviors of road-agents in dense and heterogeneous traffic from one of the following classes–impatient, reckless, threatening, careful, cautious, timid. Our approach is up to 25% more accurate than prior behavior prediction methods.
Figure 2: Overview of GraphRQI: At any instance of traffic, we use the current trajectories to form an unweighted and undirected graphical representation. We can then compute the adjacency matrix, the degree matrix, and the Laplacian matrix of the graph. Note that the principal leading submatrix of the Laplacian can be obtained from the Laplacian of the traffic from previous time-steps. We use our eigenvalue algorithm to compute the spectrum of the Laplacian, which is then used for training via a multi-layer perceptron to predict the driver behavior.

Since the behavior of individual drivers directly influences the trajectories of all other drivers in their neighborhood, we can represent the drivers as the nodes of a sparse, undirected, and unweighted graph, and the interactions between the drivers through edges between the corresponding nodes. In our formulation, we analyze the spectral properties of such a graph for behavior classification. However, there are a few challenges in calculating the spectrum of such graphs. First, traffic flow is dynamic, therefore the number of edges in the graph changes at each time-step. Second, the number of nodes in the graph tends to increase monotonically with each time-step, since retaining information on past interactions is useful for predicting current and future behaviors of other drivers [social-lstm]. This is a problem for current state-of-the-art methods that perform graph spectral analysis  [cg], as the computational complexity grows at least as , where is the number of nodes in the graph. In order to classify the behaviors, we need to design efficient methods to perform spectral analysis of such traffic graphs.

Main Results: We present a novel algorithm to classify driver behaviors from road-agent trajectories. Our approach is general, and we don’t make any assumption about traffic density, heterogeneity, or the underlying scenarios. The trajectories are extracted from traffic videos or other sensor data, and the resulting behaviors are classified among the three aggressive or three conservative behaviors described above. In order to efficiently classify the behavior, we present two novel algorithms:

  1. A classification algorithm, GraphRQI, that takes as input road-agent trajectories, converts them into a dynamic-graph-based representation of the traffic, and predicts driver behaviors as one of six labels from the spectrum of the dynamic graph.

  2. An eigenvalue algorithm that updates the spectrum of the dynamic traffic graph with current nodes in time, up from required by prior methods.

Finally, we demonstrate that our driver behavior classification also enables more accurate prediction of future trajectories and maneuvers of drivers with different behavioral characteristics and their effect on the traffic in their neighborhood. We have compared our algorithm with prior classification schemes and observe the following benefits:

  • Accuracy improvement: We report improvement of up to and in the weighted classification accuracy on the TRAF dataset [chandra2019traphic] and the Argoverse dataset [argo], respectively.

  • Speed improvement: With up to drivers, the runtime of our eigenvalue algorithm is milliseconds which is a speed of up to seconds over prior works for calculating the spectrums of dynamic traffic graph.

Ii Related Work

Ii-a Driver Behavior and Intent Prediction

There is a large body of research on driver behavior and intent prediction. Furthermore, many studies have been performed that provides insights into factors that contribute to different driver behavior. Feng et al. [ernestref2] proposed five driver characteristics (age, gender, personality via blood test, and education level) and four environmental factors (weather, traffic situation, quality of road infrastructure, and other cars’ behavior), and mapped them to 3 levels of aggressiveness (driving safely, verbally abusing other drivers, and taking action against other drivers). Rong et al. [rohanref3] presented a similar study but instead used different features such as blood pressure, hearing, and driving experience to conclude that aggressive drivers tailgate and weave in and out of traffic. Dahlen et al. [rohanref5] studied the relationships between driver personality and aggressive driving using the five-factor-model [big5]. Aljaafreh et al. [ernestref8] categorized driving behaviors into four classes: Below normal, Normal, Aggressive, and Very aggressive, in accordance with accelerometer data. Social Psychology studies [ernestref9, ernestref10] have examined the aggressiveness according to the background of the driver, including age, gender, violation records, power of cars, occupation, etc. Mouloua et al. [ernestref11] designed a questionnaire on subjects’ previous aggressive driving behavior and concluded that these drivers also repeated those behaviors under a simulated environment. However, the driver features used by these methods cannot be computed easily for autonomous driving in new and unknown environments, which mainly rely on visual sensors.

Several methods have analyzed driver behavior using visual and other information. Murphey et al. [ernestref12] conducted an analysis on the aggressiveness of drivers and observed that longitudinal (changing lanes) jerk is more related to aggressiveness than progressive (along the lane) jerk (i.e., rate of change in acceleration). Mohamad et al. [ernestref13] detected abnormal driving styles using speed, acceleration, and steering wheel movement, which indicated the direction of vehicles. Qi et al. [ernestref14] studied driving styles with respect to speed and acceleration. Shi et al. [ernestref15] pointed out that deceleration is not very indicative of the aggressiveness of drivers, but measurements of throttle opening, which are associated with acceleration, were more helpful in identifying aggressive drivers. Wang et al. [ernestref16] classified drivers into two categories, aggressive and normal, using speed and throttle opening captured by a simulator. Sadigh et al. [ernestref17] proposed a data-driven model based on Convex Markov Chains to predict whether a driver is paying attention while driving. Some recent approaches also utilize RL (reinforcement learning) and imitation learning techniques to teach networks to recognize various driver intentions [qi2018intent, codevilla2018end], while others have applied RNN and LSTM-based networks to perform data-driven driver intent prediction [zyner2018recurrent, zyner2017long]. In our case, we exploit the spectrum of dynamic traffic graphs to predict driver behaviors, which is complementary to these approaches and can be combined with such methods.

Ii-B Supervised Learning Using Graphs

Bruna et al. [bruna2013spectral] introduce supervised learning using convolutions on graphs via graph convolutional networks (GCNs). Kipf et al. [kipf2016semi] extend GCNs to self-supervised learning. However, these methods are restricted to static graphs. In order to model the temporal nature of traffic using graphs, Yu et al. [stgcn] used a Spatio-temporal GCN (STGCN). The authors use recurrent neural networks to compute the temporal features of the dynamic traffic network. TIMERS [zhang2018timers] incrementally computes the spectrum using incremental Singular Value Decomposition (SVD) [incsvd]. The method implemented a restart on the SVD when the error increases above a threshold. Li et al. [li2017attributed] proposed an algorithm for computing the spectrum for node classification tasks that support addition and deletions of edges.

Iii Background

Symbol Description
Number of vehicles
Number of non-zero elements in a vector or matrix.
Set of indices corresponding to non-zero entries in a vector .
Set of columns of , indexed by elements of .
The Laplacian matrix in a sequence of sub-degree matrices.
The dimension number of rows (or columns)) of the laplacian matrix .
A binary vector of length such that
Eigenvector matrix
Diagonal matrix with eigenvalues on the diagonal.
Table I: Notations used in the paper.
Metric SVD IncrementalSVD [incsvd] RestartSVD [zhang2018timers] TruncatedSVD CG [cg] GraphRQI
Space -
Table II: Analytical Comparison of GraphRQI with classical and state-of-the-art eigenvector algorithms. In practice, our observed runtime is milliseconds which is a speed of up to seconds over prior works for calculating the spectrums of dynamic traffic graph.

Our overall objective is to classify all drivers in a traffic video into one of six behavior classes — impatient, reckless, threatening, careful, cautious, and timid. We assume that the trajectories of all the road-agents in the video are provided to us as the input. Given this input, we first construct a traffic graph at each time-step, with the number of road-agents during that time-step being the number of nodes and undirected, unweighted edges connecting the neighboring road-agents during that time-step. Next, we show that the spectrum of the Laplacian matrix of the graphs contains pertinent information to classify the nodes (drivers) into corresponding behavior classes (Section IV).

In this section, we give an overview of a graph spectrum and a brief overview of graph topology. Notations used frequently throughout the rest of the paper are presented in Table I.

Figure 3: (left) Conservative vehicles tend to adhere to a uniform speed. Consequently, they form a uniform neighborhood that results in smooth topology in the traffic graph with short edges. (right) Aggressive drivers tend to display maneuvers such as tailgating, over-speeding, overtaking, and weaving. These maneuvers cause the road-agent to form sharper subgraphs in the traffic graph with long edges.

Iii-a Graph Spectrum

Given a graph , with a set of vertices and a set of undirected, unweighted edges , , we define the entries of its adjacency matrix as if and , otherwise. The corresponding degree matrix of the graph is a diagonal matrix with entries . The Laplacian matrix of the graph has the entries,

The matrix of eigenvectors of is called the spectrum of . It can be computed based on eigenvalue of .

Iii-B Graph Topology

The topology of a vertex in a graph refers to the arrangement of the adjacent neighborhood of the vertex. For example, if and , then we have a star topology [star]. The spectrum of the Laplacian matrix can be used to compute the topology of a vertex. Let denote the column of the matrix . Then the entry of is given by,


Thus, the entry of equals the sum of edge values of all edges with as the common node. Since the arrangement of vertices depend on the edge lengths between them, this determines the topology of the graph.

Iv Behavior Classification Using GraphRQI

In this section, we first show the construction of traffic graphs. Next, we present our eigenvalue algorithm. Finally, we present an analysis of our eigenvalue algorithm.

Iv-a Computation of Traffic Graphs

2:for  in range do
3:     Initialize
5:     while  do
8:      return
Algorithm 1 Our eigenvalue algorithm computes the eigenvectors, of a Laplacian, .

At the -th time-step, we obtain the -nearest neighbors [knn] of each road-agent using their current trajectories. In our experiments, we heuristically choose . We initialize an adjacency matrix with all zeros, and assign if road-agents are neighbors. We then form the Laplacian matrix (ala Section III-A) at the time-step, . As road-agents observe new neighbors, we update the Laplacian matrix using the following update:

where is the leading principal sub-matrix of of order and is as defined in Table I. For our experiments, we reset the Laplacian matrix after to avoid buffer overflow.

Iv-B Eigenvalue Algorithm

Our eigenvalue algorithm is built upon the classical RQI algorithm [rqi] that computes an eigenvector that corresponds to an approximation of a given eigenvalue of a given matrix. However, the dominant step of the RQI consists of matrix inversion that generally requires operations, and so the applicability of RQI to a sequence of dynamic ( or time-varying) matrices largely depends on two factors — computational complexity of matrix inversion, and the length of the sequence. For a sequence of dynamic Laplacian matrices, , the main advantage of the GraphRQI approach is to be able to compute the eigenvector matrix, , very efficiently by combining the following optimizations:

  • Recursively exploit sub- matrix information.

  • Exploit the sparsity and symmetry of Laplacian matrices to compute inverse Laplacian matrices efficiently.

We give the pseudo-code of our eigenvalue algorithm in Algorithm 1. At each time-step , we compute eigenvectors of . For each eigenvector, we perform an iterative process. We begin by initializing a random vector. Next, we iteratively perform the following update rule until it converges to an eigenvector. For the eigenvector, the update rule is given as:


where sm refers to the Sherman-Morrison formula [sm] that computes the inverse of a sum of a matrix and an outer product, , where is a sparse vector, is the approximate eigenvalue corresponding to which we compute an eigenvector, . , and are the Laplacian, spectrum, and corresponding diagonal eigenvalue matrix of the previous time-step. is a sparse vector where, if the entry of is denoted by , then the road-agent observes a new neighbor if .

Eigenvalue Algorithms (Runtime) GraphRQI + ML model (Accuracy)
Dataset SVD IncrementalSVD [incsvd] RestartSVD [zhang2018timers] TruncatedSVD CG GraphRQI LogReg SVM LSTM GraphRQI
TRAF 34.2ms 67ms 1,644ms 37ms 2,365ms 16.9ms 69.1% 74.2% 71.4% 78.3%
ARGO 16.9ms 45ms 1,091ms 35ms 2,328ms 10ms 69.9% 75.0% 70.8% 89.9%
Table III: Ablation study on TRAF and ARGO datasets. We performa running time analysis of several eigenvalue algorithms. All experiments were performed on an 8 Core Intel Xeon(R) W2123 CPU clocked at 3.60GHz with 32 GB RAM to compute the eigenvectors for a matrix, where is the number of road-agents. We also compare the accuracy of different supervised learning machine learning models and report the weighted classification accuracy.

Iv-C Graph Spectrum Analysis and Behavior Classification

Graph topologies form the basis for our driver behavior classification approach. Behavioral traffic psychology studies show that aggressive drivers tend to display maneuvers such as tailgating, over-speeding, overtaking, and weaving [rohanref3]. From a topological perspective, all four maneuvers are similar (Figure 3, right) with a sharper graph structure with larger edge lengths. On the other hand, conservative vehicles tend to adhere to a uniform speed, forming a smoother topology with smaller edge lengths (Figure 3, left). Thus, topologies for aggressive behavior are different from the topologies for conservative behavior. We, therefore, compute the spectrum of the graph that describes the topologies of traffic at various time-step to classify aggressive and conservative behaviors.

In Table II, we compare the time and space complexities of several state-of-the-art eigenvalue algorithms. For each of these methods, the input is a Laplacian matrix. We now state the theoretical guarantees for the running time and storage cost of our eigenvalue algorithm in the following theorems. All proofs are provided in the supplementary material***

Theorem IV.1.

Given a sequence, , our eigenvalue algorithm for converges cubically with a running time complexity of for each Laplacian at time-step , where , with a storage cost of .

Informally, Theorem B states that for a Laplacian matrix , the compuatation for each eigenvector requires operations, which is fewer than quadratic runtime algorithms that require operations, since . For a random intial iterate, , our eigenvalue algorithm converges cubically, that is, at each iteration, .

Note that some eigenvalue algorithms make several assumptions in terms of running time complexity analysis. For example, the conjugate gradient method [cg] requires operations. The success of conjugate gradient assumes the matrix is either well-conditioned or pre-conditioned by a preconditioning matrix, for which the running time becomes . IncrementalSVD [incsvd] requires the matrix to be low-rank. In the general case, IncrementalSVD requires which becomes for low rank matrices with rank . Our eigenvalue algorithm makes no such assumptions. We now state a lemma, related to our Laplacian matrix, that is used to prove Theorem B.

Lemma IV.2.

The complexity of computing the inverse of a Laplacian at time-step , , grows as .

The current best-case running-time complexity for inverting a matrix is using a variation of the Coppersmith–Winograd algorithm [inversion]. We show that we can achieve an even lower runtime complexity for inverting our matrix in , where .

In our case, the graphs are constructed from the road-agent trajectories as described in Section IV-A. We then pass the spectrum of the traffic graphs as inputs to train a Multi-Layer Perceptron(MLP) [mlp] for behavior classification. The MLP is parametrized by a weight vector, . Our goal is to learn the optimal parameters that minimize the squared error loss,


where is the number of vehicles, is the row of the eigenvector matrix, and corresponds to the feature vector of the vehicle. The corresponding label of the vehicle is denoted as . In our experiments, is around for each traffic video.

V Implementation and Performance

We evaluate the performance of our algorithm on two open-source datasets, TRAF and Argoverse, described in Section V-A. In Section V-B, we list other classification algorithms that are used to compare GraphRQI using standard classification evaluation metrics. We report our results and analysis in Section V-C. We perform exhaustive ablation experiments to motivate the advantages of of GraphRQI in Section V-D. Finally, in Section V-E we show how driver behavior knowledge can be useful and applied to road-agent trajectory prediction.

Dataset Method Weighted Accuracy
TRAF DANE [dane] 68.1%
Cheung et al. [ernest] 63%
GraphRQI 78.3%
ARGO DANE [dane] 65.5%
Cheung et al. [ernest] 62.5%
GraphRQI 89.9%
Table IV: We report weighted classification accuracy on TRAF and ARGO datasets. Our GraphRQI considerably improves the accuracy.

V-a Datasets


The TRAF dataset [chandra2019traphic] consists of road-agents in dense traffic with heterogeneous road-agent captured using front and top-down viewpoints in daylight as well as night setting. These videos correspond to the highway and urban traffic in dense situations from cities in China and India.


Argoverse [argo] contains 3D tracking annotations, 300k extracted road-agent trajectories, and semantic maps collected in urban cities, Pittsburgh and Miami.

For both datasets, we obtained behavior classification labels through crowd-sourcing and used as the ground truth. All annotators were asked to choose from the six labels. We obtain labels for about 600 agents from TRAF and 250 agents from Argoverse. More details on the labeling can be found in the supplementary material.

V-B Evaluation Metrics and Methods

Our data contains a long-tail distribution of behavior labels (refer to the supplementary material for exact distribution). We report a weighted classification accuracy which defined as . Here, for the behavior label, denotes the fraction of the road-agents with that label in the ground truth and denotes the accuracy for that label class. We also report the running time of our eigenvalue computation measured in seconds.

We compare our approach with Dynamic Attributed Network Embedding (DANE) [dane] and Cheung et al. [ernest]. DANE considers two sets of input matrices — the data matrix and the attribute matrix, where the data consists of the IDs of objects and attributes are features describing the objects. We use DANE for behavior classification by using the IDs of road-agents for the data matrix and their trajectories for the attribute matrix. DANE uses an eigenvalue algorithm to compute, and update, the spectrum of the Laplacians for both the data and attribute matrix. Cheung et al. [ernest] do not use an eigenvalue algorithm; instead, they use linear lasso regression based on a set of trajectory features.

V-C Analysis

Classification Accuracies

We present the weighted accuracy for GraphRQI, DANE and Cheung et al. in Table IV. We show an improvement of upto %.


Our approach of classifying the driver behavior is intuitive and interpretable. In Figure 4, GraphRQI classifies road-agent 1 (red bounding box) as threatening and road-agent 2 (yellow bounding box) as conservative. We plot the vector, , where corresponds to the largest eigenvalue of , as a gradient below the figure. The larger entries of correspond to aggressive vehicles and smaller values correspond to conservative vehicles.

Figure 4: GraphRQI Performance: GraphRQI classifies road-agent 1 (red bounding box) as threatening and road-agent 2 (yellow bounding box) as conservative. We plot the vector, , where is the eigenvector corresponding to the largest eigenvalue of , as a gradient below the figure.

V-D Ablation Experiments

Running Time Evaluation

For fair evaluation, we use the same programming platform (Python 3.7) and the same processor for running all the different methods (8 Core Intel Xeon(R) W2123 CPU clocked at 3.60GHz with 32 GB RAM). We empirically validate the theoretical guarantees of Theorem B by replacing our eigenvalue algorithm with several standard eigenvector algorithms, Singular Value Decomposition (SVD), Incremental SVD[incsvd], Restart SVD [zhang2018timers], Truncated SVD, and the state-of-the-art iterative method Conjugate Gradient Descent [cg]. For SVD, and TruncatedSVD, we directly used the library routine implemented in the scipy package, and for RestartSVD, we used the authors’ original implementation. In practice, our method outperforms all these standard methods for Driver Behavior Classification. These results are shown in Table III.

Accuracy Evaluation

GraphRQI algorithm uses MLP for the final driver behavior classification. We replace this classifier with logistic regression, support vector machines (SVM [svm])) and deep neural networks (LSTM [lstm]) and observe the accuracies. These results have been tabulated in Table III.

V-E Driver Behavior-Based Road-Agent Trajectory Prediction

We show that learning driver behavior can benefit road-agent navigation by improved trajectory prediction for each road-agent. Specifically, we demonstrate an application where knowledge of the behavior of a road-agent allows nearby road-agents to generate more efficient or safe paths. At any time-step , given the coordinates of all road-agents and their corresponding behavior labels, we show that for the next seconds, aggressive (impatient, reckless, threatening) drivers may attempt to reach their goals faster through either overtaking, overspeeding, weaving, or tailgating. We also show that conservative (timid, cautious, careful) vehicles may steer away from aggressive road-agents. Our application is currently limited to straight road and four-way intersection networks. When moving through the network, each vehicle’s speed is computed using a car-following model [car-following]. The vehicles are parametrized using their positions, velocities, starting, and estimated short-term goal positions. Each vehicle is assigned a behavioral label through our behavior classification algorithm. Then as per the car-following model, the trajectories of the road agents are predicted. We include a demonstration of the application in the supplementary video.

Figure 5: Trajectory Prediction Application: (top) We see an aggressive vehicle (red) cutting across a pedestrian, not allowing it to cross first. (bottom) Conservative vehicles, on the other hand, stop and wait for the pedestrian to cross first.

Vi Conclusion, Limitations, and Future Work

We present a novel algorithm, GraphRQI, that can classify driver behaviors of road-agents from their trajectories. Our classification of behaviors is based on prior studies in traffic psychology, and our approach is designed for trajectory data from videos or other visual sensors. We compute the traffic matrices from the road-agent positions and use an eigenvalue algorithm to compute the spectrum of the Laplacian matrix. The spectrum comprises of the features used to train a multi-layer perceptron for driver behavior classification. We also present a faster algorithm to compute the eigenvalue and demonstrate the performance of our algorithm on two open-source traffic datasets. We observe improvement in classification over prior methods.

Our approach has some limitations. The accuracy of our approach is governed by the accuracy of the tracking methods used to compute the positions of the road-agent. While the spectrum of the traffic graph can be used to classify many behaviors, it may not be sufficient to classify the behaviors in all traffic scenarios. As part of future work, we also need to evaluate our algorithms on other traffic datasets. One major issue is the lack of sufficient labeled datasets in terms of driver behaviors. We would also like to combine the current approach with other deep learning models in order to improve accuracy and performance.




Appendix A Proof of Theorem IV.1


Given a sequence, , our eigenvalue algorithm for converges cubically with a running time complexity of for each Laplacian at time-step , where , with a storage cost of .


Our eigenvalue algorithm is motivated from the Rayleigh Quotient Iteration algorithm [rqi] (RQI). In RQI, to compute the eigenvector of any given matrix, , corresponding to an approximation to a given eigenvalue, say , the standard update formula for the RQI method is given as , where is initialized as a random vector. The efficiency of the update step depends on the efficiency of computing . In GraphRQI, denoting as the number of non-zero elements of , and , for each of the eigenvectors, the iterative update formula is: . At first glance, it seems as if this update can be performed by applying the Sherman-Morrison (SM) update on the following decomposition:


That is, by expressing as a sum of rank-1 matrices, , where each is a diagonal matrix with and elsewhere. However, SM on in the above equation requires flops ( flops for each SM operation applied times. See Lemma C). Therefore, we look towards a better solution.

Observe that we can decompose , where and is an outer product of rank 2 with is a vector of 1’s and 0’s, where represents the addition of a new neighbor by the road-agent. denotes the laplacian matrix at the previous time-step plus a diagonal update on account of observing new neighbors at the current time-step (This is further explained in lemma IV.2). So,

where is a sum of two rank-1 matrices. Therefore, if we can compute in less than or equal to , we are done. It is easy to verify that for any matrix ,


and thus,

We can now use Lemma IV.2 to show that computing the inverse of the laplacian, can be performed in . The entire update can be summarized as follows,



At each time-step, we add a new column and a row to the laplacian matrix of the previous time-step. Therefore, the space complexity grows linearly with the dimension of the laplacian matrix. Finally, the cubic convergence follows from the classic RQI algorithm [rqi].

Appendix B Proof of Lemma IV.2


The complexity of computing the inverse of a Laplacian at time-step grows as .


Note that since the diagonal elements (the degree of each node (vehicle)) are now different due the entry of new nodes at current time . Let the number of new nodes be . For example, if at the current time-step, 10 cars from the previous time-step observed a new neighbor, then . Since we heuristically set the time-step and the KNN parameters, we constrain each vehicle observing at most 1 new neighbor for each time-step. Therefore,

where is a low-rank matrix with rank , where we treat as a constant due to our earlier assumption. Each is a diagonal matrix with and elsewhere. Hence,


where . Using the Sherman-Morrison formula times on , we get in (See appendix C).

Appendix C Lemma 3: Each Sherman-Morrison operation requires flops.


For a matrix of size , each SM operation requires,

  • 2 matrix-vector multiplications -

  • 1 inner product -

  • 1 outer-product computation -

  • 1 matrix subtraction -

Appendix D Annotation of TRAF and Argoverse Datasets

We study and classify road-agents into six different driving behaviors. We used two datasets for training a neural network and the class-wise distribution of the final labels are shown in Figure 6.

Label Traits
Impatient Overtaking
Threatening Cutting in front of agents
Driving extremely close to agents
Reckless Driving in the wrong direction
Jaywalking through traffic
Not following rules
Careful Maintain neighbors
Uniform Speed
Follow rules
Timid Not moving
Slowing down/coming to stop
Cautious Waiting for vehicles/pedestrians

Studies in traffic psychology show that each of the six driving behaviors are associated with certain driving traits. For example, impatient drivers show different traits (e.g., over-speeding, overtaking) than reckless drivers (e.g., driving in the opposite direction to the traffic) [rohanref6-reckless]. Similarly, timid drivers may drive slower than the speed limit [rohanref7-timid], and careful drivers obey the traffic rules. We now describe the labeling process. First we distinguish between annotators and expert annotators. Annotators correspond to the individuals that labeled the videos via crowd-sourcing. They may or may not have expertise or familiarity with the traffic in the videos. All annotators were asked to observe these traits in the videos and list the most prominent traits observed in each road-agent. The trait table is shown in the table above.

The annotators were not shown the class-wise distribution of traits, nor were they shown annotations marked by any other annotator. Each annotator was presented with a uniform presentation of traits and their task was to simply mark the traits that they observed for each vehicle. Each annotator labeled every video. The final class was assigned by resolving discrepancies for each road-agent. Some vehicles would receive conflicting traits. If the conflicting traits belonged to the same Parent class (aggressive or conservative), then we would simply select the majority trait. In rare scenarios where there is no majority, we would simply take a final call. However, if the conflicting traits belonged to different parent classes, then that road-agent would be re-annotated through crowd-sourcing, and would repeat the steps.

Figure 6: Class-wise distribution of the final labels
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description