Poisson Multi-Bernoulli Mixtures for Sets of Trajectories
For the standard point target model with Poisson birth process, the Poisson Multi-Bernoulli Mixture (\pmbm) is a conjugate multi-target density. The \pmbmfilter for sets of targets has been shown to have state-of-the-art performance and a structure similar to the Multiple Hypothesis Tracker (\mht). In this paper we consider a recently developed formulation of multiple target tracking as a random finite set (\rfs) of trajectories, and present three important and interesting results. First, we show that, for the standard point target model, the \pmbmdensity is conjugate also for sets of trajectories. Second, based on this we develop \pmbmtrackers (trajectory \rfsfilters) that efficiently estimate the set of trajectories. Third, we establish that for the standard point target model the multi-trajectory density is \pmbmfor trajectories in any time window, given measurements in any (possibly non-overlapping) time window. In addition, the \pmbmtrackers are evaluated in a simulation study, and shown to yield state-of-the-art performance.
Multiple Target Tracking (\mtt) can be defined as the processing of noisy sensor measurements to determine 1) the number of targets, and 2) each target’s trajectory, see, e.g., [?]. \mttis a challenging problem due to the partitioning of noisy sensor measurements into potential tracks and false alarms, referred to as data association. Each new measurement received could be the continuation of some previously detected target, the first detection of a thus far undetected target, or a false alarm. In this paper, we consider the standard point target models with Poisson birth, see, e.g., [3, Sec. 10.2], where each target can yield at most one detection per scan, and each detection is from at most one target. For target tracking with multiple detections per target per scan, see, e.g., [?].
In this paper we rely on modelling the \mttproblem using random finite sets (\rfs) of trajectories, recently proposed in [?, ?]. Within this set of trajectories framework, the goal of Bayesian \mttis to compute the posterior density over the set of trajectories. Assuming the standard point target models, we focus specifically on two set of trajectory densities: the density for the set of all trajectories; and the density for the set of current (i.e., remaining) trajectories. We then show that in both cases the conjugate set of trajectories density is of the Poisson Multi-Bernoulli Mixture (\pmbm) form, first introduced for sets of target states in . We derive the \pmbmpredictions and the \pmbmupdate, resulting in two \pmbmset of trajectories filters: we call these tracking algorithms \pmbmtrackers, to easily distinguish them from the \pmbmfilter , which is for sets of target states.
Solutions to the \mttproblem presuppose the ability to solve single target tracking. In single target prediction, filtering and smoothing, linear and Gaussian models are important because they yield closed form solutions, provided by the Kalman filter and the Rauch-Tung-Striebel (rts) smoother, see, e.g., [?]. For notation, let denote the single target state at time , denote a measurement at time . A key result in Bayesian filtering is that for linear and Gaussian models, the posterior density is, see, e.g., [?],
for any and , where denotes the consecutive time steps from to . That is, regardless of if we are doing prediction (), filtering () or smoothing (), the posterior is Gaussian and exactly described by the first two moments and .
Even more generally, if and denote the concatenation of target state and measurements in the time intervals and , it holds that
for any time intervals and , see, e.g., [?]. That is, regardless of what time intervals we consider, the posterior is Gaussian and exactly described by the first two moments.
In this paper, in addition to deriving the \pmbmtrackers, we also show that a result similar to (2) holds also for multi-target tracking with the standard point object models. Specifically, the set of trajectories in some (finite) time interval , is \pmbmdistributed given measurements in any (finite) time interval . Consequently, for prediction, filtering and smoothing, the set of trajectories in a given time interval is \pmbmdistributed. An important special case of this is given by the \pmbmfilter : the set of targets at time step , given measurements up until the same time step, is \pmbmdistributed.
To summarize the contributions in this paper, for the standard point target model we
extend \pmbmconjugacy to sets of trajectories,
present computationally efficient algorithms for computing the \pmbmdensity, and,
similar to (2), we show that for arbitrary time intervals the posterior set of trajectories density is \pmbm.
In [2, Sec. II] computing the set of trajectories density was predicted to be “extremely challenging…even for small problems”. Importantly, the \pmbmtrackers show that there is a closed-form recursion to obtain the density over the set of trajectories, and that this density can be approximated in an efficient way even for problems with many trajectories of varying length, including trajectories that are several hundred time steps long.
The rest of the paper is organised as follows. Related work is discussed in the next section. In Section III we present the two considered problem formulations, and we review the standard point target models. In Section IV we present necessary background theory about random finite sets of trajectories. The prediction and update of the \pmbmdensity for sets of trajectories are presented in Section V. In Section VI we show that, for the point target models, the distribution of the set of trajectories over any time window is \pmbm. Linear and Gaussian implementations of the \pmbmtrackers are presented in Section VII, and Section VIII contains results from a simulation study. The paper is concluded in Section IX.
Ii Related work
Approaches to the \mttproblem include the multi hypothesis tracker , and various tracking filters based on \rfs[3, ?]. \mhtalgorithms maintain a series of global hypotheses for each possible measurement-target correspondence, along with a conditional state distribution for each target under each hypothesis. In the original \mht, each measurement could potentially be the first detection of a new target, and the number of newly detected targets was given a Poisson distribution in order to provide a Bayesian prior. In , the \mhtmodel was made rigorous through random finite sequences, under the assumption that the number of targets present is constant but unknown, and has a Poisson prior. Target state sequences were formed under each global hypothesis, and the Poisson distribution of targets remaining to be detected provided a Bayes prior for events involving newly detected targets. Hypotheses were constructed as being data-to-data, since no a priori data was assumed on target identity. \mhtwas extended further in , to address a time-varying number of targets, incorporating birth of targets which are not immediately detected, necessitating equivalence classes of indistinguishable hypotheses.
In , the conjugate Bayes filter for Poisson birth models was derived using \rfss, obtaining a result somewhat similar to , involving a Poisson distribution representing targets which are hypothesised to exist but have not been associated to any measurements, and a multi-Bernoulli mixture (mbm) representing targets that have been associated at some stage. Adopting the standard point target models, target appearance and disappearance were modelled. The resulting \pmbmfilter has a hypothesis structure similar to \mht, however, track continuity in the form of trajectories was not established.
Like the \pmbmfilter , other early tracking algorithms based on \rfs, such as the \phdfilter [?], did not formally establish track continuity. In previous work, track continuity has been established using labels, see, e.g., [?, ?, 6, ?, ?], or related approaches, see, e.g., [?, ?]. When using labelled \rfs, a label (whose uniqueness is ensured through the model) is incorporated into the target state. With labelled states, track continuity is maintained by connecting estimates from different times that have the same label. The \dglmb[6, ?, ?] filter is conjugate for labelled Bernoulli birth; the similarities and differences between the \pmbmand \dglmbconjugate priors are discussed in, e.g., [?, Sec. V.C] and [?, Sec. IV].
Several simulation studies have shown that, compared to tracking filters built upon labelled \rfs, the \pmbmfilters provide state-of-the-art performance for tracking the set of targets, see, e.g., [?, ?, ?, ?, ?, ?]. The \pmbmfilters have been shown to be versatile, and have been used with data from lidars [?, ?, ?, ?], radars [?, ?], and cameras [?, ?, ?]. They have been successfully applied not only to tracking of moving targets, but also mapping of stationary objects [?], joint tracking and mapping, as well as joint tracking and sensor localisation [?]. It is therefore well-motivated to establish \pmbmconjugacy for sets of trajectories, to derive efficient tracking algorithms, and to prove that the set of trajectories density is \pmbmfor any time interval.
Iii Problem formulations
To clearly differentiate between a target state at a single time and a sequence of target states, we let target denote the state at some time, and we let trajectory denote a sequence of consecutive states. Thus, “set of targets” and “target \rfs” refer to formulations involving \rfssof target states at a single time (e.g., the unlabelled \rfsformulation in ), and “set of trajectories” and “trajectory \rfs” refer to formulations involving a \rfsof trajectories, or sequences of states.
Let denote a target state at time , where represent the base state space, and let denote a measurement at time , where is the measurement state space. We utilise the standard multi-target dynamics model, defined in Assumption III, and the standard point target measurement model, defined in Assumption III.
The multiple target state evolves according to the following time dynamics process:
Targets arrive at each time according to a Poisson Point Process (\ppp) with birth intensity , independent of existing targets.
Targets depart according to independent and identically distributed (iid) Markovian processes; the survival probability in state is .
Target motion follows iid Markovian processes; the single-target transition pdf is .
The multiple target measurement process is as follows:
Each target may give rise to at most one measurement; probability of detection in state is .
Each measurement is the result of at most one target.
False alarm measurements arrive according to a \pppwith intensity , independent of targets and of measurements originating from targets.
Each measurement originating from a target is independent of all other targets and all other measurements, conditioned on its corresponding target; the single target measurement likelihood is .
There are many ways in which a Multiple Target Tracking (\mtt) problem can be formulated, and which one is interesting/relevant depends on the tracking application. In this paper, we begin with the following two Problem Formulations (\probform):
Problem Formulation 1.
The set of all trajectories: the objective is, under Assumptions III and III, to compute the density of the trajectories of all targets that have passed through the surveillance area at some point between the initial time step and the current time step, i.e., both the targets that are present in the surveillance area at the current time, and the targets that have left the surveillance area (but were in the surveillance area at at least one previous time).
Problem Formulation 2.
The following \probformwas considered in [7, ?, ?, ?, ?]:
Problem Formulation 3.
Iv Random finite sets of trajectories
In this section, we first review the trajectory state representation, and then present densities for sets of trajectories.
Iv-a Single trajectory
For two time steps and , , following standard tracking notation the ordered sequence of consecutive time steps is denoted . The (unordered) set of consecutive time steps is denoted We use the trajectory state model proposed in [?, ?], in which the trajectory state is a tuple
is the discrete time step of the trajectory birth, i.e., the time step when the trajectory begins.
is the discrete time step of the trajectory’s most recent state, i.e., the time step when the trajectory ends.
is, given and , the sequence of states
The length of a trajectory is time steps; is finite because and are finite.
The trajectory state space for trajectories in the time interval is [?]
where denotes union of disjoint sets, and denotes Cartesian products of . This is a slight generalisation of the definition in [?], where is used to denote the trajectory state space for trajectories in . The finite lengths of trajectories in are restricted to .
The trajectory state density factorises as follows
where the domain of is for . Integration is performed as follows [?],
Iv-B Sets of trajectories
The set of trajectories in the time interval is denoted as . The domain for is , the set of all finite subsets of . In some applications, e.g., \probform 2, we consider a subset of the trajectories in , namely the ones that were alive at some point in the time interval , where . We denote this set of trajectories as
Let be a real-valued function on a set of trajectories . Integration over sets of trajectories is defined as regular set integration :
The multi-trajectory density is defined analogously to the multi-target density. A trajectory Poisson Point Process (\ppp) is analogous to a target \ppp, and has set density
with intensity , where , if , and if . The trajectory \pppintensity is defined on the trajectory state space , i.e., realisations of the \pppare trajectories with a birth time, a time of the most recent state, and a state sequence.
A trajectory Bernoulli process is analogous to a target Bernoulli process, and has set density
Here, is a trajectory state density (6), and is the Bernoulli probability of existence. Together, and can be used to find the probability that the target trajectory existed at a specific time, or find the probability that the target state was in a certain area at a certain time. Trajectory Multi-Bernoulli (\mb) \rfsand trajectory \mb-Mixture (\mbm) \rfsare both defined analogously to target \mb\rfsand target \mbm\rfs: a trajectory \mbis the union of multiple independent trajectory Bernoulli \rfss; trajectory \mbm\rfsis an \rfswhose density is a mixture of trajectory \mbdensities. Lastly, a \pmbmdensity is the union of a \pppand an \mbm.
V PMBM trackers
In this section we present the modelling for the two problem formulations, and the resulting prediction and update for the \pmbmfilters. For the set of all trajectories (\probform 1), we seek the density for the \rfs, in other words, all trajectories existing at some point in time between the initial time step to the current time step . For the set of current trajectories (\probform 2) we only want the trajectories that are still alive , and we seek the density for the \rfs
As in [7, ?, ?, ?, ?], we hypothesise that the set of trajectories density is a multi-target conjugate prior of the \pmbmform, and we will show that the \pmbmform is preserved through prediction and update. In tracking with standard models, the \ppprepresents trajectories that are hypothesised to exist, but have never been detected, e.g., because they have been occluded or have been located in an area where the sensor(s) have low detection probability. The \mbmrepresents trajectories that have been detected at least once, and each \mbin the mixture corresponds to a unique sequence of data associations for all detected trajectories.
The \pmbmdensity is defined as
where is union of disjoint sets, is a \pppdensity (10) with intensity , are Bernoulli densities (11) with probabilities of existence and trajectory densities . In the \mbmdensity (13b), is a track table with tracks indexed to , and is a possible global data association hypothesis, and for each global hypothesis and each track , indicates which track hypothesis is used in the global hypothesis. For track , there are single trajectory hypotheses. The weight of the global hypothesis is , where is the weight of single trajectory hypothesis from track .
The structure of the trajectory \pmbm(13) is the same as the structure of the target \pmbmfrom . We present “track oriented” (to) \pmbmtrackers, where a track is initiated for each measurement. The set of all measurement indices up to time is denoted , and is the history of measurements that are hypothesised to belong to hypothesis from track . The set of global data association hypotheses can be obtained from , and , see [7, Eq. 35].
A \pmbmdensity (13) is defined by the parameters
In the following subsections we will show how the \pmbmparameters are predicted and updated, in order to track either the set of current trajectories, or the set of all trajectories.
V-a Prediction step
In this section we describe the time evolution of the set of trajectories. A standard \pppbirth model is used, i.e., target birth at time step is modelled by a \ppp, with trajectory birth intensity
Note that it is possible to have alternative birth models, such as \mbbirth, or \mbmbirth, which results in \mbmfilters for sets of trajectories [?]. Performance evaluation of filters based on either \pppbirth or \mbbirth is presented in, e.g., [?, ?]. For those cases, the spatial densities of the Bernoulli birth components would be of the same form as in (15), i.e., for birth at time we have .
The trajectory state dependent probability of survival at time is defined as
where denotes Kronecker’s delta function located at .
Transition model for the set of all trajectories
The Bernoulli \rfstransition density without birth is
where denotes Dirac’s delta function. In this model, the interpretation of the probability of survival is that it governs whether or not the trajectory ends, or if it extends by one more time step. However, importantly, regardless of whether or not the trajectory ends, the trajectory remains in the set of all trajectories.
The prediction step is presented in the theorem below.
Proof outline: Analogous to proof of [7, Thm. 1].
Transition model for the set of current trajectories
The Bernoulli \rfstransition density without birth is
In this model, is used as follows. If a target disappears, or “dies” ( in (19a)), then the entire trajectory will no longer be a member of the set of current trajectories. If the target survives, then the trajectory is extended by one time step. For the set of current trajectories, is therefore used in a way that is typical for tracking a set of targets, see, e.g., .
The resulting prediction step is given in the theorem below.
Analogous to proof of [7, Thm. 1]. ∎
V-B Update step
The measurement model is the same regardless of problem formulation, hence we express it for a general trajectory \rfs. The target measurement model of Assumption III is extended to a trajectory measurement model by defining a Bernoulli measurement density as follows:
The clutter is modelled as a \pppwith intensity .
The measurement model is of the same form as the standard set of targets measurement model, and thus the trajectory measurement update is analogous to the target measurement update in .
For tracks continuing from previous time steps (), a hypothesis is included for each combination of a hypothesis from a previous time and either a missed detection or an update using one of the new measurements, such that the number of hypotheses becomes . For missed detection hypotheses (, :
For hypotheses updating existing tracks (, , , , i.e., the previous hypothesis , updated with measurement ):
Finally, for new tracks, , (i.e., the new track commencing on measurement ),
Analogous to proof of [7, Thm. 2]. ∎
V-C Properties of the resulting trackers
Two \pmbmtrackers result from the theorems:
Note that regardless of which problem formulation is considered, current trajectories or all trajectories, the update, cf. Theorem 3, is the same. Both \pmbmtrackers are to. For each measurement, a potential new track is initiated, see (26). In the update, additional hypotheses are created, as indicated in (24) and (25). In the prediction, the number of tracks and hypotheses remains constant, see (20b) and (20c).
The Bernoulli probabilities of existence have different meanings in the two trackers: for the set of current trajectories problem formulation, is the probability that the trajectory exists at the current time step and has not ended yet; in the set of all trajectories problem formulation, represents the probability that the trajectory existed at any time between and the current time step .
We proceed to discuss the representation of the \pppintensity and the Bernoulli densities in the trackers.
Consider a trajectory mixture density of the form
|where is an index set, and each mixture component is characterised by a weight and a parameter . The parameter consists of a birth time , a most recent time , where , and a state sequence density . For the trajectory density (27a) a pmf for is obtained straighforwadly as|
For the weights we have that if is a density, and if is an intensity function, e.g., a \pppintensity. Note that there is no restriction in (27) that and must be unique, i.e., we may have and/or for , . Densities/intensities of the form (27) facilitate simple representations for the state sequence , conditioned on and .
The target birth \pppintensity is often modelled as an un-normalized distribution mixture, often a Gaussian mixture. It then follows that the trajectory birth \pppintensity , cf. (15), is of the form (27), with the special structure that . From this it follows further that the Poisson intensity , and all Bernoulli densities will be of the form (27).
In other words the parameters of the posterior \pmbmdensity are
|with intensity and state densities of the form (27),|
where and are index sets for the mixture density components, and and are weights such that and .
Time of birth
Consider a data association in which a measurement at time step is associated to a potential new target. Conditioned on the association, for the trajectory end time , we have that . We proceed to focus on the time of birth probability mass function (pmf). The new Bernoulli track density is of the form (26f),
With a Poisson intensity of the mixture form (27),
we get a multi-modal posterior trajectory density , which can be pruned if necessary. The pmf for is
where , and
Note that, as more measurements are associated the trajectory density is updated, meaning that the maximum a posteriori (\map) time of birth may change. An example of this is shown in Section VIII.
Time of most recent state
Consider a posterior Bernoulli density of the form (27), to which a measurement was associated at time , and for the sake of brevity, assume that it has a single mixture component with parameter . Assume, for the sake of brevity, that , and let denote the transitions model’s probability that the trajectory ends. Then it is possible to represent the predicted density at time as a mixture,
This has important implications for the implementation of the \pmbmtrackers for the set of all trajectories: during the prediction step, the hypothesis space for the trajectory density increases, due to the fact that we do not know if the trajectory ended at time , (33b), or continued to time , (33c). However, it is not necessary to explicitly represent both state sequence densities, instead it is sufficient to explicitly represent the state sequence density that continued to time , as well as a pmf .
This is especially important when there are several consecutive misdetections associated: for consecutive misdetections, with a single pmf and a single pdf we can compactly represent different hypotheses for and