Learned Fine-Tuner for Incongruous Few-Shot Learning

In this paper, we interpret MAML as a general framework for explicit optimization of the fast adaptation objective, and leverage the L2O framework to meta-learn “how to learn with only a few examples” across incongruous few-shot learning tasks (it is obviously applicable to congruous tasks). Specifically, we demonstrate the following:

• (Sec. 2) How L2O can extend MAML to meta-learn across incongruous tasks for fast adaptation where the meta-learned variables are different from the fine-tuned variables (see Figure 1).

• (Sec. 3) How a learned fine-tuner (LFT) is meta-learned for better generalization in few-shot learning and how LFT theoretically differs from L2O.

• (Sec. 4) How LFT applies to solving the problem of universal adversarial attack generation in the novel context of meta-learning across incongruous attack generation tasks (see a summary of superior performance of LFTs in Table 1).

• (Sec. 5) How our proposed scheme has wider meta-learning applicability than MAML and outperforms L2O for few-shot classification and regression tasks.

In MAML, both levels of the optimization operate on the same optimizee (for example, same network parameters), and accordingly, learning tasks $\{T_i{\}}_{i=1}^N$ are restricted to problems which share the same optimizee. However, in the general meta-learning setting, similar tasks could be from related yet incongruous domains corresponding to different objectives $\left\{f_i\right\}$ with optimizee variables of different dimensions that cannot be shared between tasks. For example, adversarial perturbation parameters cannot be shared between images from different data sources; network parameters from different architectures cannot be shared even if they are solving related learning tasks. In such cases, meta-learning the initialization is not applicable. Instead, we propose to meta-learn an optimizer – the fine-tuner – for fast adaptation of the task-specific optimizee ${\theta }_i$ in a few-shot setting even when meta-learning across incongruous tasks.

When fine-tuning the task-specific optimizee variable ${\theta }_i$ by ${\text{{tt RNN}}}_{\varphi }$ (Algorithm 1, Step 6), we can use an invariant RNN architecture to tolerate the task-specific variations in the dimensions of optimizee variables $\{{\theta }_i{\}}_{i=1}^N$. Recall from (3) that ${\text{{tt RNN}}}_{\varphi }$ uses the gradient or gradient estimate $g_i({\theta }_i^{\left(k\right)};D_i^{\text{tr}})$ as an input, which has the same dimension as ${\theta }_i$. At first glance, a single ${\text{{tt RNN}}}_{\varphi }$ seems incapable of handling incongruous $\{T_i{\}}_{i=1}^N$ defined over optimizee variables of different dimensionalities. However, if ${\text{{tt RNN}}}_{\varphi }$ is configured as a coordinate-wise Long Short Term Memory (LSTM) network (proposed by [9]), it is invariant to the dimensionality of optimizee variables $\{{\theta }_i{\}}_{i=1}^N$ since ${\text{{tt RNN}}}_{\varphi }$ is independently applied to each coordinate of ${\theta }_i$ regardless of its dimensionality. In contrast to MAML, the invariant ${\text{{tt RNN}}}_{\varphi }$ expands the application domain of LFT beyond model weights/parameters over congruous tasks to incongruous ones such as designing universal adversarial perturbations across incongruous attack tasks.

The use of L2O in (3) also allows us to update the task-specific optimizee variable ${\theta }_i$ using not only first-order (FO) information (gradients) but also zeroth-order (ZO) information (function values) if the loss function $f_i$ is a black-box objective function. We can estimate the gradient $\nabla f_i({\theta }_i;D_i)$ with finite-differences of function values $g_i({\theta }_i;D_i)$ [17, 15]:

where $\mu >0$ is a small step size (the smoothing parameter), $u_j,j\in \left[n\right]$ are $n$ random directions with entries from $N(0,1)$. This gives us ZO-LFT alongside our original FO-LFT. The function $g_i(\cdot ;\cdot )$ can also be more sophisticated quantities derived from gradients as proposed in [10, 11, 16].

The support for ZO optimization is crucial when explicit gradient expressions are computationally difficult or infeasible. Our experiments show that ZO-LFT can be as effective as state-of-the-art FO-LFT, despite leveraging only the inputs and outputs of the DL model (see Figure 4 in Section 5).

Since Algorithm 1 meta-learns the optimizer variable $\varphi$ rather than the initialization of optimizee variable $\theta$, it requires a different meta-learning gradient ${\nabla }_{\varphi }{\sum }_k{w}_k{f}_i({\theta }_i^{\left(k\right)},D_i^{\text{val}})$. Focusing only on ${\nabla }_{\varphi }f({\theta }^{\left(k\right)};D^{\text{val\%}})$ in (5) and dropping the task index $i$:

where $\bullet$ denotes a matrix product that the chain rule obeys [21]. Statement 1 details the recursive computation of $G^{\left(k\right)}$, which calls for the second-order (or first-order) derivative of $f$ w.r.t. $\theta$ if $g({\theta }^{\left(k\right)};D^{\text{tr}})$ denotes the gradient (or gradient estimate) of $f$ in (3). We refer readers to Supplement A for the details of derivation.

It is clear from (8) and (9) that the second-order derivatives are involved without additional assumptions due to the presence of $\frac{\partial g\left({\theta }^{(k-1)}\right)}{\partial \theta }$ if $g\left({\theta }^{(k-1)}\right)$ is specified by the first-order derivative w.r.t. $\theta$. If it is specified by the ZO gradient estimate, then there will only be first-order derivatives involved in (9) and (8). With the use of the coordinate-wise RNN, the terms $\frac{\partial \Gamma }{\partial g}$, $\frac{\partial \Gamma }{\partial h}$, $\frac{\partial \Pi }{\partial g}$, $\frac{\partial \Pi }{\partial h}$ correspond to diagonal matrices. Note that $G^{\left(0\right)}=0$ and $H^{\left(0\right)}=0$. Unlike MAML, this recursively defined meta-learning gradient w.r.t. $\varphi$ is not as prone to the issue of vanishing gradients for large values of $K$.

A direct solution to problem (12) only ensures the attack power of UAP (${\theta }_i$) against the given data set $D_i$ at the specific task $T_i$. Like any learning problem, the set $D_i$ needs to be large for the UAP to successfully attack unseen examples. In the few-shot setting (small $D_i$), we want to leverage meta-learning to facilitate better generalization for the learned UAP. The attack loss (12) is considered as the task-specific loss $f_i$ in (4) with ${\theta }_i$ as the optimizee. With multiple few-shots tasks $T_i$ & corresponding $(D_i^{\text{tr}},D_i^{\text{val}})$, we meta-learn the fine-tuner ${\text{{tt RNN}}}_{\varphi }$ to generate UAPs for new few-shot UAP tasks. For comparison, we also consider (i) MAML to meta-learn an initial UAP $\theta$ for each task, and (ii) L2O to meta-learn ${\text{{tt RNN}}}_{\varphi }$ for just solving (12). Experiments are conducted over two types of tasks:

We meta-learn LFT, MAML and L2O with 1000 few-shot UAP tasks $\left\{T_i\right\}$. In LFT and MAML, the fine-tuning and meta-update sets $(D_i^{\text{tr}},D_i^{\text{val}})\sim T_i$ are drawn from the training dataset and the test dataset of an image source, respectively. However, both MNIST and CIFAR-10 are used across tasks. In both $D_i^{\text{tr}}$ & $D_i^{\text{val}}$, $2$ image classes with $2$ samples per class are randomly selected. In L2O, $D_i^{\text{tr}}$ is combined with $D_i^{\text{val}}$; there is no meta-validation involved in the meta-learning. We evaluate the performance of the meta-learning schemes over $100$ random unseen few-shot UAP tasks (data for task is generated in a manner described above). Moreover, both LFT and L2O are fine-tuned from random initialization over test tasks; MAML, when applicable, starts fine-tuning with the meta-learned initialization. We refer readers to Supplement C for more details.

In Table 1, we present the superior performance of LFT to tackle attack tasks involving different image sets (such as MNIST & CIFAR10). Specifically, we present the averaged attack success rate (ASR), ${\ell }_1$-norm distortion, and number of fine-tuning steps required to first reach $100\%$ ASR (within $200$ steps) of generated UAP using different meta-learners (LFT, MAML, L2O), where the victim DNN is given by the LeNet-5 model [22]. Compared to MAML, LFT meta-learns the high-level “how” to generate universal attacks in a few-shot setting without being restricted to a single image set (namely, allowing the mismatch of image source between meta-training and testing). Compared to L2O, LFT is impressively effective in generating few-shot attacks on unseen image sets.

In Figure 2, we present the averaged ASR of UAP over test tasks versus the number of fine-tuning steps. We report ASR at every combination of training and evaluation settings, denoted by the pair of datasets. For example, (MNIST, CIFAR-10) implies that meta-learning is performed with MNIST and then used to generate UAP against CIFAR-10 at (meta-)testing. And we use MNIST + CIFAR-10 to represent the union of MNIST and CIFAR-10 tasks for meta-learning (or meta-testing). The results on ${\ell }_1$ distortion strength of UAP are shown in Figure A1 in the supplement. Briefly, we find that LFT yields an UAP generator with fastest adaption, highest ASR, and lowest attack distortion than MAML and L2O; see details as below.

LFT significantly outperforms MAML and L2O when meta-learning and meta-testing with congruous UAP tasks (MNIST, MNIST). As shown in Figure 2 & A1 (and Table 1), the significance lies at three aspects. (i) Fewest fine-tuning steps are required to attack new tasks with $100\%$ ASR; (ii) Highest ASR can be achieved at a given number of fine-tuning steps; (iii) Lowest perturbation strength is needed to achieve the most significant attacking power. We also note that L2O yields better performance (higher ASR and lower distortion) than MAML. This indicates that meta-learning the optimizer ($\varphi$) could offer better generalization than meta-learning the optimizee ($\theta$). In these few-shot UAP tasks, the “how to learn” seems more useful than “where to learn from”.

On the other hand, LFT outperforms L2O in the standard transfer attack settings, corresponding to the scenario (MNIST, CIFAR-10), where the generator of UAP is learnt over MNIST, but tested over CIFAR-10. Note that MAML can not be applied to this scenario, since the meta-learned UAP initialization does not have the same dimension as the test data. Compared to the congruous setting (MNIST, MNIST), the ASR decreases from $100\%$ to $50\%$ for LFT and L2O. However, LFT adapts better to unseen tasks. LFT also outperforms L2O in cases that involve incongruous tasks drawn from MNIST + CIFAR-10. One interesting observation is that the use of hybrid data sources (incongruous tasks) during meta-training enables the learned fine-tuner to generate UAP with faster adaptation on unseen images; compare rows 1 & 2 of Figure 2. In Supplement E, we present additional results, comparison to white-box attacks, and visualization of UAP patterns.

In this experiment, we consider to learn DNN-based image classifiers over 2-way 5-shot learning tasks. These tasks are drawn from two image sources, MNIST & CIFAR-10. We specify the classifier to be trained as a 3-layer multilayer perceptron (MLP) for MNIST data and a convolutional neural network (CNN) with four CONV layers for CIFAR-10 data. Thus, the task-specific optimizee in (4) corresponds to the DNN parameters for a given task. The incongruous tasks are then given by learning DNNs with different architectures (MLP and CNN). We also consider the congruous setting where only one model architecture is adopted across tasks.

In Table 2, we evaluate the classification accuracy of the model parameters acquired from LFT, MAML and L2O over $500$ randomly selected 2-way 5-shot test tasks. Here tasks acquired from different data sources (MNIST or CIFAR-10) correspond to different model architectures (MLP or CNN). Our proposed LFT is able to match MAML for congruous tasks (meta-learning with tasks from (CIFAR-10, CNN) and meta-testing with tasks from same set). MAML is not applicable for the incongruous tasks (at meta-learning or meta-testing). L2O and LFT are applicable, and LFT achieves higher accuracy ($>2-8\%$) throughout. Figure 3 shows that LFT can outperform L2O by up to $20\%$ or more when considering a smaller number of fine-tuning steps. More results on other few-shot tasks are presented in Supplement F.

In the next experiment, we revisit the sine wave regression problem presented in MAML [2, Sec. 5.1]. Here multiple few-shot regression tasks are generated by randomly varying the amplitude and the phase of a sinusoid. The goal is to learn a regression model to gain fast adaption from observed few-shot regression tasks to unobserved regression tasks. In our experiments, we choose a MLP model with 2 hidden layers as the regressor to be learnt. In LFT, the regressor’s parameters are regarded as the optimizee variable, and ${\text{{tt RNN}}}_{\varphi }$ is learnt to obtain a regressor of small prediction error, in terms of mean squared error (MSE), even from a random initialization after a few fine-tuning steps during testing phase. Please refer to Supplement G for more details on the setup.

Figure 4 presents the few-shot test MSE versus the fine-tuning iterations. In this example, we consider both FO and ZO variants of meta-learners based on gradient and gradient estimates, respectively. As we can see, LFT outperforms L2O and MAML, and the use of ZO-LFT could be as effective as FO-LFT. We highlight that MAML uses the learnt meta-initialization, but LFT just fine-tunes from a random initialization. This suggests that learning the optimizer/meta-fine-tuner (namely, ${\text{{tt RNN}}}_{\varphi }$) provides better generalization ability than learning the meta-initialization of optimizee variable.