Survey: Combining Modular Skills in Multitask Learning

MSR(Microsoft Research)Yoshua Bengio
paper: https://arxiv.org/pdf/2202.13914.pdf
Code: https://github.com/McGill-NLP/polytropon

[] MTL
[] (skill)(sharing)
[] MTL
[]Few-shot Adapatation

MTL

Z

xySZ

End-to-EndGumbel-sigmoid()

Inductive Bias

ZIndian Buffet Process Inductive Bias IBP

ZInductive Bias(Two-speed Learning Rate)

NLPCrossFitFew-shot AdaptationSkilledMTLShared(Private)SharedZ11

DiscreteZSparsityZUsageSparsity