Auxiliary Tasks in Multi-task Learning

Auxiliary Tasks in Multi-task Learning

Lukas Liebel    Marco Körner

sectioning \RedeclareSectionCommands[ beforeskip=-.5afterskip=.25]section,subsection,subsubsection \RedeclareSectionCommands[ beforeskip=.5afterskip=-1em ]paragraph,subparagraph \newacronymfpsFPSframes per second \newacronymrsuRSUroad scene understanding \newacronymgtaGTA VGrand Theft Auto V \newacronymsideSIDEsingle-image depth estimation \newacronymfcnFCNfully-convolutional network \newacronymcnnCNNconvolutional neural network \newacronymmiouMIoUmean intersection over union \newacronymreluReLUrectified linear unit \newacronymasppASPPatrous spatial pyramid pooling \newacronymresnetResNetresidual network \newacronymrmsctdRMSCTDroot mean squared cyclic time difference \newacronymmseMSEmean squared error \newacronymrmseRMSEroot mean squared error \newacronym[shortplural=ADAS]adasADASadvanced driver assistance system \newglossaryentrycvname=computer vision, description= \newglossaryentryavname=autonomous vehicles, description= \newglossaryentryadname=autonomous driving, description= \newglossaryentrymtname=multi-task, description= \newglossaryentrysiname=synthetic images, description= \newglossaryentryannname=artificial neural network, description= \newglossaryentrycgname=computer graphics, description= \newglossaryentrystdname=synthetic training data, description= \newglossaryentryauxname=auxiliary task, description= \newglossaryentrymainname=main task, description= \newglossaryentrysemsegname=semantic segmentation, description= \newglossaryentryirname=image recognition, description= \newglossaryentryacname=atrous convolution, description= \newglossaryentryconvname=convolutional, description= \newglossaryentrymaxname=max pooling, description= \newglossaryentryfcname=fully connected, description= \newglossaryentrysoftmaxname=softmax, description= \newglossaryentrycename=cross entropy, description= \newglossaryentrybatchname=mini batch, description= \newglossaryentryowname=open world, description= \newglossaryentryodname=object detection, description= \newglossaryentrysctdname=squared cyclic time difference, description= \newglossaryentrylossname=loss function, description= \newglossaryentryicname=image classification, description= \newglossaryentryatname=atomic task, description= \newglossaryentryvbname=vision-based, description= \newglossaryentrye-dname=encoder-decoder, description= \newglossaryentryename=encoder, description= \newglossaryentrydname=decoder, description= \newglossaryentrydatasetname=synMT, description= \glsdisablehyper \setkomafontcaption \setkomafontcaptionlabel\usekomafontcaption


author \publishersComputer Vision Research Group, Chair of Remote Sensing Technology
Technical University of Munich, Germany
{lukas.liebel, marco.koerner}


mt \glsplcnn have shown impressive results for certain combinations of tasks, such as \glsside and \glssemseg. This is achieved by pushing the network towards learning a robust representation that generalizes well to different \glsplat. We extend this concept by adding \glsplaux, which are of minor relevance for the application, to the set of learned tasks. As a kind of additional regularization, they are expected to boost the performance of the ultimately desired \glsplmain. To study the proposed approach, we picked \glsvb \glsrsu as an exemplary application. Since \glsmt learning requires specialized datasets, particularly when using extensive sets of tasks, we provide a multi-modal dataset for \glsmt \glsrsu, called \glsdataset. More than synthetic images, annotated with different labels, were acquired from the video game \glsgta. Our proposed deep \glsmt \glscnn architecture was trained on various combination of tasks using \glsdataset. The experiments confirmed that \glsplaux can indeed boost network performance, both in terms of final results and training time.

Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description