Few-shot learning is a significant problem, yet existing methods are limited in handling diverse types of tasks due to the absence of a unified architecture and a flexible adaptation mechanism.
Thanks to non-parametric matching, Visual Token Matching (VTM) has recently expanded the scope of few-shot learning tasks up to arbitrary dense prediction tasks.
To extend VTM to the tasks beyond dense prediction (e.g., vision-language tasks), the following two requirements are necessary:
(1) In order to apply non-parametric matching, input and output data have to be aligned.
(2) To handle tasks that highly demands general knowledge, such as language-involved tasks, it is desirable to utilize a pretrained decoder with prior knowledge.
% Our approach
In this work, we propose Latent Token Matching (LTM), which exploits an encoder that learns to align input and output by mapping data into fixed-size latent embeddings.
LTM conditions the latent obtained from the matching to a frozen pretrained decoder, allowing it to exploit prior knowledge effectively.
We train LTM simultaneously from multiple heterogeneous tasks and observe that our model benefits from the joint training.
% Experiments
We demonstrate the effectiveness of LTM on three challenging tasks with non-aligned input and output structure: Interactive Segmentation (IS), Referring Expression Segmentation (RES), and Regional Captioning (RC).


----
Can you rewrite this abstract in your style?  Provide exactly two different responses that are separated with 6 asterisk symbols: ******.
user