Incremental Adversarial Domain Adaptation
for Continually Changing Environments
Continuous appearance shifts such as changes in weather and lighting conditions can impact the performance of deployed machine learning models. Unsupervised domain adaptation aims to address this challenge, though current approaches do not utilise the continuity of the occurring shifts. Many robotic applications exhibit these conditions and thus facilitate the potential to incrementally adapt a learnt model over minor shifts which integrate to massive differences over time. Our work presents an adversarial approach for lifelong, incremental domain adaptation which benefits from unsupervised alignment to a series of sub-domains which successively diverge from the labelled source domain. We demonstrate on a drivable-path segmentation task that our incremental approach can better handle large appearance changes, e.g. day to night, compared with a prior single alignment step approach. Furthermore, by approximating the marginal feature distribution for the source domain with a generative adversarial network, the deployment module can be rendered fully independent of retaining potentially large amounts of the related source training data for only a minor reduction in performance.
Appearance changes based on lighting, seasonal, and weather conditions provide a significant challenge for outdoor robots that rely on machine learning models for perception. While providing high performance in their training domain, visual shifts occurring in the environment can result in significant deviations from the training distribution, severely reducing accuracy during deployment. This can be counteracted by employing additional training methods to render these models invariant to their application domain [1, 2].
For scenarios where labelled data is unavailable in the target domain, the problem can be addressed in the context of unsupervised domain adaptation. Recent state-of-the-art approaches addressing this challenge operate by training deep neural networks in adversarial domain adaptation (ADA) frameworks. An ADA framework is characterised by the optimisation of one or multiple encoders with the objective to confuse a domain discriminator operating on their output [2, 3, 4]. The main intuition behind this framework is that by using the gradients flowing back through the discriminator network, the encoder learns domain invariant embeddings, thus allowing the main supervised task to be robust to changes in the target domain.
Recent successes based on adversarial domain adaptation have achieved state-of-the-art performance on toy datasets [2, 5, 6, 7] as well as real world applications for autonomous driving within changing environmental conditions [8, 4]. However, domains with a significant difference in appearance - such as day and night - continue to present a substantial challenge . We conjecture that the observed change in environmental conditions (e.g. in autonomous driving) is a gradual process which accumulates to produce massive differences over extended periods. This work exploits the incremental changes observed throughout deployment to continuously counteract the domain shift by updating discriminator and encoder incrementally while observing the visual shift (as illustrated in Figure 1).
Our approach is evaluated both on synthetic and real world data. An artificial dataset is created with direct control over the number of sub-domains and the strength of the incremental shifts for illustration and demonstration purposes. The following real world evaluation focuses on a drivable-path segmentation task for autonomous driving based on segments from the Oxford RobotCar Dataset  with different environment illuminations from different times of day.
Adversarial domain adaptation requires access to the unlabelled source data for sampling during the optimisation for domain invariance. The discriminator has to be trained from a balanced distribution over source and target data to prevent overfitting to the target data. Similar to work on synthetic dataset extension , we remove this requirement by training a generative adversarial network (GAN)  to imitate the marginal encoder feature distribution in the source domain. We empirically demonstrate that domain adaptation via aligning target encoder features with GAN generated samples instead of source domain embeddings only results in minor performance reduction. Crucially, this means that the deployment module is fully independent of source domain data, enabling application on mobile platforms with limited compute and memory resources.
The contributions of our work are as follows:
Introduction of a method for incremental unsupervised domain adaptation for platforms acting in continuously changing environments.
Presentation of an additional method to remove ADA’s requirement of retaining extensive amounts of source data by modelling the marginal feature representation of source domain data with a generative model.
Quantitative investigation of the influence of dividing the adaptation task into incremental alignment over smaller shifts based on a synthetic toy example.
Application of the proposed method to the real world task of drivable terrain segmentation and proof of feasibility for online application in the context of runtime evaluation on an NVIDIA GPU.
Ii Related Work
Continuously changing appearance has been a long-standing challenge for robot deployment, as shifts between training and deployment data can seriously impact model performance. A great deal of effort has focused on designing and comparing various feature transformations with the goal of creating representations invariant to environmental change . Other approaches address the problem through retaining multiple experiences  or synthesising images between discrete domains . However, it is unclear how these systems can efficiently scale to a continuous shift in the domain distribution.
In recent years, there has been a steady trend towards applying deep networks for various robotics tasks, where early layers act as a feature encoder with a supervised loss for the desired task on the output of the network. Unfortunately, even such powerful models still suffer from the problem of shifts in domain appearance. This has lead to a number of works which try to address this issue [2, 8, 15, 4].
The possibility to directly optimise complete feature representations via backpropagation for domain invariance  or target-source mappings  has lead to significant success of deep architectures in this field. Long et al.  focus on minimising the Maximum Mean Discrepancy for the feature distributions of multiple layers of the network architecture. Rozantsev et al.  go even further and impose a penalty for deviations in the network parameters across domains. Sun et al.  align second order statistics of layer activations for source and target domains. Hoffman et al.  match the label statistics between the true source and predicted target labels for semantic classification.
Furthermore, adversarial approaches to domain adaptation have been introduced [2, 3, 5], which rely on training a domain discriminator to classify the domains underlying an encoder’s feature distribution. While adversarial training techniques have been shown to be notoriously unstable and difficult to optimise, there has been a pronounced body of work towards improving their stability, including more dominant use of the confusion loss  and more recently the Wasserstein GAN framework .
All the above mentioned works treat the unsupervised domain adaptation problem as a batch transition without exploiting temporal coherence commonly available to robots in continuous deployment. Continuous refinement has however been actively researched in supervised learning for many years (e.g. [19, 20, 21]), yet there has been little work on methods for unsupervised domain adaptation. One notable exception is the work by Hoffman et al. , which addresses the problem with predefined features and focuses on the challenges of aligning to a continuously reshaping target domain. This work seeks to extend the recently developed approach of adversarial domain adaption to a continuously evolving target domain by capitalising on the perpetual observations made by a robot.
Incremental Adversarial Domain Adaptation addresses the problem of degraded model performance due to continuously shifting environmental conditions. This includes changes caused by weather and lighting occurring in outdoor scenarios. Compared to the regular single-step domain adaptation paradigm, we benefit in applications requiring continual deployment through exploitation of the incremental changes that integrate to large domain shifts. Continuously observable lighting or seasonal shifts in outdoor robotics and other applications constitute a prime example for this paradigm.
The approach employs an adversarial domain adaptation framework  aiming to facilitate the learning of an encoder feature distribution which is invariant with respect to the origin domain of its input data. In this way, the method enables the application of a supervised module trained only on source domain data to incoming unsupervised data from the application domain as depicted in Figure 2.
In comparison to existing methods [2, 6, 4] which frame the task of unsupervised domain adaptation as a one step approach between distinct source and target domains, IADA treats the incoming data as a stream of incrementally changing domains and addresses the problem of outdoor appearance change by aligning over minor shifts to reduce the complexity of the domain adaptation task.
Adversarial domain adaptation  generally tends to be hyperparameter search intensive  as it suffers from the fact that optimising a distribution for domain invariance can conflict with the supervised task. Intuitively, by dividing the domain alignment procedure into smaller incremental shifts, we simplify the overall task which can minimise the loss of relevant information.
The training procedure is split into two principal segments: offline supervised optimisation on source domain data and the unsupervised domain adaptation procedure, which potentially can be run online during platform deployment.
Hereinafter, let be the parametrisation of module and the incoming images. Source and target domains are represented with underscore and respectively. The supervised training procedure optimises the supervised module as well as the source domain encoder based on a supervised task (e.g. classification or segmentation as in Section IV).
Both of these modules are frozen during domain adaptation, which enables us to keep source performance unaffected (an approach suggested for regular ADA in ). Only target encoder and discriminator modules are trained via their respective objectives and in Equations 1 and III to align the target and source encoder feature spaces. For simplicity, dependence on model parameters is only represented in the equations which apply to their optimisation procedure.
The target encoder weights are hereby initialised from the trained source encoder parameters and both encoder and discriminator are subsequently incrementally updated to the currently encountered target data.
Our method incrementally updates a target data buffer with unsupervised, incoming data. We sample from this buffer to train our modules with respect to the objectives in Equations 1 and III for domain adaptation.
Iii-a Source Distribution Modelling
IADA is aimed at online deployment on platforms. However, its application is constrained by the requirement of retaining potentially large amounts of source training data. To counteract this requirement, we additionally extend our method with a GAN to generate samples, thus rendering the approach independent of source data during the domain adaptation task. More concretely, we optimise a generator similar to the GAN framework  which maps from n-dimensional, normally distributed noise to approximate the marginal feature distribution in our source domain during the offline training step. However, instead of optimising the generator to imitate source domain data as in the GAN framework we train the module to approximate the encoded source feature distribution (displayed in Figure 3) similar to work on the adversarial autoencoder . The resulting objectives for generator and discriminator are displayed in Equations 3 and III-A.
Subsequently during the domain adaptation procedure, the target encoder is optimised to align to the feature distribution of the GAN, whose parameters are frozen to statically model the source domain. Instead of optimising the discriminator to classify between source and target domain, in this scenario it learns to distinguish synthetically generated source features and actual target features. Target encoder and discriminator are optimised towards the objectives and in Equations 5 and III-A respectively.
In this context, we benefit during training on target data by refining the already well-trained discriminator from the GAN optimisation step. The deployment setup is equivalent to IADA.
Our evaluation is split into two parts: we first investigate a toy scenario with artificially induced domain shift for the purpose of visualisation and clarification, then we demonstrate performance gains in a continuous deployment scenario for drivable path segmentation for autonomous mobility.
The evaluation compares IADA against its one-step counterpart ADA, and furthermore investigates the influence of source domain modelling based on Section III-A. The evaluation metric depends on the supervised task in the respective target domains, classification accuracy for the toy example and mean average precision for the drivable path segmentation task.
Iv-a Toy Example: Incrementally transformed MNIST
To quantify the benefits of IADA as a function of the strength of domain shifts and the number of sub-domains, we first evaluate the approach on an synthetically deformed version of the popular MNIST dataset.
Therefore, we create additional, transformed copies of the original dataset with height-rescaled digits of 0.9 down to 0.5 times the original height which are visualised in Figure 4. These synthetically transformed domains enable us to create a scenario with full control over the underlying domain shift and ensure that the occurring changes can be observed in arbitrary detail.
We employ a Network-in-Network like architecture  with exponential linear activation functions  splitting after the last hidden layer and applying a discriminator with 2 hidden layers with each 512 neurons. The adversarial loss is weighted by a factor of 0.001 for domain adaptation as well as GAN training. All specific model parameters extending standard architectures are the result of hyperparameter search experiments.
Table I shows the target domain classification accuracy of 1-step adaptation methods vs their incremental counterparts which continue optimising the target encoder across domains with incrementally increasing domain shift and digit rescaling. Furthermore, we test the methods in combination with GAN-based Source Domain Modelling (SDM) introduced in Section III-A.
As displayed in Table I, all domain adaptation approaches outperform pure source-optimised models, while incremental domain adaptation provides additional benefits over regular ADA. Finally, the SDM variants of all approaches only result in minor performance reductions.
|target domains||only source||ADA||ADA SDM||IADA||IADA SDM|
To investigate classification accuracy as a function of the number of sub-shifts or sub-domains, the MNIST digits are rescaled even further to 0.3 of the original height and evaluated with varying numbers of equally spread sub-domains with IADA and IADA SDM.
Separating larger shifts into incremental steps as displayed in Table II enables us to address the problem with a curriculum of easier tasks. For the domain adaptation from MNIST to its rescaled copy, the benefits of incremental domain adaptation saturate at around 5 to 10 sub-domains.
Iv-B Free Space Segmentation
An area of active research in autonomous driving is the detection of traversable terrain. We evaluate IADA in this context for drivable path segmentation based on sections of the Oxford RobotCar dataset  using segmentation labels generated based on . The dataset consists of approximately hour-long driving sessions from different days collected over the course of a year. Based on the nature of the dataset, we approximate the scenario of continuous application by picking five datasets to represent different daylight conditions from morning to evening and train on a labelled source dataset based on morning data as seen in Figure 5.
The resulting 5 sub-domains were chosen to incrementally change in lighting conditions and serve as a proxy for the online deployment scenario. Each domain consists of about 2000 images, rescaled for the evaluation to a size of 128 by 320 pixels. Pixelwise segmentation labels for training are available only for the source domain, while the approach utilises labels for the evaluation in all domains.
For all segmentation tasks, we employ an adapted ENet  architecture which presents a compromise of model performance and size. The architecture focuses on strong segmentation accuracy as well as reasonable computational requirements which makes it a strong contender for online deployment on mobile platforms.
For the discriminator, we split the ENet architecture just before the upsampling stages (see ) and employ an additional 4-layer convolutional discriminator.
Comparable to the application on the synthetic domain shift dataset in Section IV-A, IADA outperforms one-step domain adaptation. In this real world scenario, we cannot ensure smoothness over the appearance changes and the turning-on of street lights indeed represents a step change in our environment. It is to be expected that continuous domain shifts would increase the advantages of IADA as displayed with synthetic data in Section IV-A.
|target domains||only source||ADA||ADA SDM||IADA||IADA SDM|
With computation times of approximately 26 minutes for the adaptation to a new domain on an NVIDIA GeForce GTX Titan Xp GPU, we can potentially deploy the system on vehicles to adapt to the currently encountered domain at a rate of about 55 times a day in continuous deployment. The extension with source domain modelling reaches computation times of 29 minutes resulting in nearly 50 updates per day.
The evaluation for drivable path segmentation with our offline datasets still builds on distinct target domains. The approach however can be extended easily to more fluid alignment to the online perceived data domain via the utilisation of sliding window sampling during deployment. Furthermore, it was shown in Section IV-A that the benefits of dividing the target domains further for IADA can saturate when the sub-domains are becoming increasingly similar.
IADA relies on access to the incremental shifts in the appearance of our environment. With limited access or stepwise changes in the perceived environment the approach degrades to regular adversarial domain adaptation. This becomes particularly visible in our segmentation datasets as the turning-on of the streetlights leads to an instantaneous change in the appearance of the environment.
All experimental results noted in our work are based on the confusion loss for domain adaptation . An adaptation of the WGAN framework  for domain adaptation leads to (on average) slightly more stable training and insignificantly better performance. However, we focused on the confusion loss formulation as, due to the additional critic training rounds for the WGAN framework, it leads to significantly lower training duration.
SDM is of particular importance for smaller mobile platforms with limited data storage. In the context of cloud processing or larger platforms with significant data storage volumes, the minor accuracy loss can be prevented when applying the original formulation for IADA in Section III.
Vi Conclusion and Future Work
We present a method for addressing the task of domain adaptation in an incremental fashion, adapting to a continuous stream of changing target domains. Furthermore, we introduce an approach for source domain modelling, training a GAN to approximate the marginal distribution over source features to render the domain adaptation training step independent of retaining large amounts of source data. Both methods are evaluated on synthetically shifted versions of rescaled MNIST digits for illustration purposes and full access to the number of sub-domains as well as the real world task of drivable path segmentation in the context of autonomous driving.
The field of continual training during deployment provides many possible benefits as models can be adapted to the currently encountered environment and learn from data unavailable during offline training. However, the approach also opens up new security risks. The well known problem of perpetrators introducing adversarial samples to the system could lead to not only corruption of the current prediction but prolonged distortion of the model. The area represents a possible direction for further research an counteracting adversarial examples. Further extensions of this work include addressing the additional problem of catastrophic forgetting in lifelong learning scenarios such that previous target domains which were once adapted to continue to provide good performance.
The authors would like to acknowledge the support of the UKâs Engineering and Physical Sciences Research Council (EPSRC) through the Programme Grant EP/M019918/1 and the Doctoral Training Award (DTA) as well as the support of the Hans-Lenze-Foundation. Additionally, the donation from NVIDIA of the Titan Xp GPU used in this work is gratefully acknowledged.
-  Mingsheng Long, Jianmin Wang, and Michael I. Jordan. Deep transfer learning with joint adaptation networks. CoRR, abs/1605.06636, 2016.
-  Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, Victor Lempitsky, Urun Dogan, Marius Kloft, Francesco Orabona, and Tatiana Tommasi. Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research, 17:1–35, 2016.
-  Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, and Mario Marchand. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446, 2014.
-  Markus Wulfmeier, Alex Bewley, and Ingmar Posner. Addressing appearance change in outdoor robotics with adversarial domain adaptation. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, 2017.
-  Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. Domain separation networks. In Advances in Neural Information Processing Systems, pages 343–351, 2016.
-  Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Xingchao Peng, Sergey Levine, Kate Saenko, and Trevor Darrell. Towards adapting deep visuomotor representations from simulated to real environments. arXiv preprint arXiv:1511.07111, 2015.
-  Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. arXiv preprint arXiv:1702.05464, 2017.
-  Judy Hoffman, Dequan Wang, Fisher Yu, and Trevor Darrell. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649, 2016.
-  Will Maddern, Geoff Pascoe, Chris Linegar, and Paul Newman. 1 Year, 1000km: The Oxford RobotCar Dataset. The International Journal of Robotics Research (IJRR), 36(1):3–15, 2017.
-  Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. Learning from simulated and unsupervised images through adversarial training. CoRR, abs/1612.07828, 2016.
-  Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
-  Stephanie Lowry, Niko Sünderhauf, Paul Newman, John J Leonard, David Cox, Peter Corke, and Michael J Milford. Visual place recognition: A survey. IEEE Transactions on Robotics, 32(1):1–19, 2016.
-  Winston Churchill and Paul Newman. Experience-based Navigation for Long-term Localisation. The International Journal of Robotics Research (IJRR), 2013.
-  Peer Neubert, Niko Sunderhauf, and Peter Protzel. Appearance change prediction for long-term navigation across seasons. In Mobile Robots (ECMR), 2013 European Conference on, pages 198–203. IEEE, 2013.
-  Sungeun Hong, Woobin Im, Jongbin Ryu, and Hyun S Yang. Sspp-dan: Deep domain adaptation network for face recognition with single sample per person. arXiv preprint arXiv:1702.04069, 2017.
-  Artem Rozantsev, Mathieu Salzmann, and Pascal Fua. Beyond sharing weights for deep domain adaptation. arXiv preprint arXiv:1603.06432, 2016.
-  Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. CoRR, abs/1612.01939, 2016.
-  Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In NIPS 2016 Workshop on Adversarial Training. In review for ICLR, volume 2016, 2017.
-  Aniruddha Kembhavi, Behjat Siddiquie, Roland Miezianko, Scott McCloskey, and Larry S Davis. Incremental multiple kernel learning for object recognition. In Computer Vision, 2009 IEEE 12th International Conference on, pages 638–645. IEEE, 2009.
-  Vidit Jain and Erik Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 577–584. IEEE, 2011.
-  Alex Bewley, Vitor Guizilini, Fabio Ramos, and Ben Upcroft. Online self-supervised multi-instance segmentation of dynamic objects. In Robotics and Automation (ICRA), 2014 IEEE International Conference on, pages 1296–1303. IEEE, 2014.
-  Judy Hoffman, Trevor Darrell, and Kate Saenko. Continuous manifold based adaptation for evolving visual domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 867–874, 2014.
-  Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv:1511.05644, 2015.
-  Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013.
-  Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015.
-  Dan Barnes, William P. Maddern, and Ingmar Posner. Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy. CoRR, abs/1610.01238, 2016.
-  Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147, 2016.
-  Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.