HomeResearch-feedChannelRankingsGCTTHU AI TROpen DataMust Reading
HomeSelf-Supervised Learning
Self-Supervised LearningSelf-supervised learning is essentially a method of unsupervised learning, and we will set up a "Pretext" and construct Pesdeo Labels to train the network model according to some characteristics of data. The self-supervised model can be used as a pre-training model for other learning tasks to provide a better initial training area. Therefore, self-supervised learning can also be regarded as a general visual representation for learning images.
Year#CitationLike
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Zhenzhong Lan,Mingda Chen, Sebastian Goodman,Kevin Gimpel,Piyush Sharma,Radu Soricut
ICLR, (2020)
Conclusion
We have convincing evidence that sentence order prediction is a more consistently-useful learning task that leads to better language representations, we hypothesize that there could be more dimensions not yet captured by the current self-supervised training losses that could crea...
default
Comments
Weibo
Cited by86BibtexViews352Links
0
0
Big Self-Supervised Models are Strong Semi-Supervised Learners
Ting Chen,Simon Kornblith,Kevin Swersky,Mohammad Norouzi,Geoffrey Hinton
Methods
Conclusion
Similar approaches are common in NLP, we demonstrate that this approach can be a surprisingly strong baseline for semi-supervised learning in computer vision, outperforming the state-of-the-art by a large margin
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by21BibtexViews281Links
0
0
Multi-task self-supervised learning for Robust Speech Recognition
Ravanelli Mirco, Zhong Jianyuan, Pascual Santiago, Swietojanski Pawel, Monteiro Joao, Trmal Jan,Bengio Yoshua
ICASSP, pp.6989-6993, (2020)
Conclusion
The proposed problemagnostic speech encoder+ architecture is based on an online speech distortion module, a convolutional encoder coupled with a quasi-recurrent neural network layer, and a set of workers solving self-supervised problems
Must Reading
Comments
Weibo
Cited by7BibtexViews122Links
0
0
Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
Senthil Purushwalkam,Abhinav Gupta
NeurIPS 2020, (2020)
Methods
Conclusion
We demonstrate that these self-supervised representations learn occlusion invariance by employing an aggressive cropping strategy which heavily relies on an object-centric dataset bias
Must ReadingMust Reading
Comments
Weibo
Cited by6BibtexViews123Links
0
0
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
Alexei Baevski,Steffen Schneider,Michael Auli
ICLR, (2020)
Conclusion
We evaluate models on two benchmarks: TIMIT is a 5h dataset with phoneme labels and Wall Street Journal is a 81h dataset for speech recognition
default
Comments
Weibo
Cited by6BibtexViews119Links
0
0
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Alexei Baevski,Henry Zhou,Abdelrahman Mohamed,Michael Auli
NeurIPS 2020, (2020)
Methods
Conclusion
We presented wav2vec 2.0, a framework for self-supervised learning of speech representations which masks latent representations of the raw waveform and solves a contrastive task over quantized speech representations
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by3BibtexViews150Links
0
0
Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification
Guile Wu,Xiatian Zhu,Shaogang Gong
AAAI, pp.12362-12369, (2020)
Methods
Conclusion
We presented a Tracklet Self-Supervised Learning method for unsupervised image and video person re-id
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by3BibtexViews103Links
0
0
Multi-Task Self-Supervised Learning for Disfluency Detection
Shaolei Wang,Wanxiang Che,Qi Liu, Pengda Qin,Ting Liu,William Yang Wang
national conference on artificial intelligence, (2020)
Methods
Conclusion
Experimental results on the commonly used English Switchboard test set show that our approach can achieve competitive performance compared to the previous systems by using less than 1% of the training data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by3BibtexViews149Links
0
0
Self-supervised Learning on Graphs: Deep Insights and New Direction
Wei Jin,Tyler Derr,Haochen Liu, Yiqi Wang,Suhang Wang,Zitao Liu,Jiliang Tang
Methods
Conclusion
We first introduce various basic supervised learning pretext tasks for graphs and present detailed empirical study to understand when and why SSL works for graph neural networks and which strategy can better work with GNNs
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by2BibtexViews177Links
0
0
Distilling Localization for Self-Supervised Representation Learning
Zhao Nanxuan,Wu Zhirong, Lau Rynson W. H.,Lin Stephen
Methods
Conclusion
We identified a strong error pattern among self-supervised models in their failure to localize foreground objects
default
Comments
Weibo
Cited by0BibtexViews100Links
0
0
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Kahn Gregory,Abbeel Pieter,Levine Sergey
Methods
Conclusion
We presented BADGR, an end-to-end learning-based mobile robot navigation system that can be trained entirely with selfsupervised, off-policy data gathered in real-world environments, without any simulation or human supervision, and can improve as it gathers more data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by0BibtexViews156Links
0
0
Digging Into Self-Supervised Monocular Depth Estimation
Clément Godard,Oisin Mac Aodha,Michael Firman,Gabriel J. Brostow
ICCV, pp.3827-3837, (2019)
Methods
Conclusion
We showed how together they give a simple and efficient model for depth estimation, which can be trained with monocular video data, stereo data, or mixed monocular and stereo data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by155BibtexViews113Links
0
0
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Xin Wang,Qiuyuan Huang,Asli Çelikyilmaz,Jianfeng Gao,Dinghan Shen,Yuan-Fang Wang,William Yang Wang,Lei Zhang
CVPR, (2019): 6629-6638
Methods
Conclusion
In this paper we present two novel approaches, Reinforced Cross-Modal Matching and Supervised Imitation Learning, which combine the strength of reinforcement learning and self-supervised imitation learning for the visionlanguage navigation task
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by104BibtexViews168Links
0
0
Revisiting Self-Supervised Visual Representation Learning
Alexander Kolesnikov,Xiaohua Zhai,Lucas Beyer
CVPR, (2019)
Conclusion
As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by95BibtexViews85Links
0
0
Self-Supervised Learning of Pretext-Invariant Representations
Misra Ishan,van der Maaten Laurens
CVPR, pp.6706-6716, (2019)
Methods
Conclusion
We studied Pretext-Invariant Representation Learning for learning representations that are invariant to image transformations applied in self-supervised pretext tasks
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by87BibtexViews35Links
0
0
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
Michelle A. Lee,Yuke Zhu, Krishnan Srinivasan, Parth Shah,Silvio Savarese,Li Fei-Fei,Animesh Garg,Jeannette Bohg
international conference on robotics and automation, (2019)
Conclusion
The primary goal of our experiments is to examine the effectiveness of the multimodal representations in contactrich manipulation tasks
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by80BibtexViews247Links
0
0
Scaling and Benchmarking Self-Supervised Visual Representation Learning
Priya Goyal,Dhruv Mahajan,Abhinav Gupta,Ishan Misra
ICCV, pp.6390-6399, (2019)
Methods
Conclusion
We studied the effect of scaling two selfsupervised approaches along three axes: data size, model capacity and problem complexity
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by77BibtexViews133Links
0
0
Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera
Fangchang Ma, Guilherme Venturelli Cavalheiro,Sertac Karaman
international conference on robotics and automation, (2019)
Methods
Conclusion
This framework requires only sequences of RGB and sparse depth images, and outperforms a number of existing solutions trained with semi-dense annotations
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by74BibtexViews113Links
0
0
S4L: Self-Supervised Semi-Supervised Learning
Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov,Lucas Beyer
International Conference on Computer Vision, (2019): 1476-1485
Methods
Conclusion
We further showed that S4L methods are complementary to existing semisupervision techniques, and Mix Of All Models, our proposed combination of those, leads to state-of-the-art performance
Must ReadingMust Reading
Comments
Weibo
Cited by67BibtexViews95Links
0
0
Keywords
Supervised LearningSelf-supervised LearningComputer VisionSemi-supervised LearningLearning Artificial IntelligencePose EstimationDeep LearningFace RecognitionFeature ExtractionImage Reconstruction
Authors
Sergey Levine
Sergey Levine
Paper 7
Abhinav Gupta
Abhinav Gupta
Paper 5
Xiaohua Zhai
Xiaohua Zhai
Paper 3
William Yang Wang
William Yang Wang
Paper 3
Andrew Zisserman
Andrew Zisserman
Paper 3
Sebastian Thrun
Sebastian Thrun
Paper 3
Dacheng Tao
Dacheng Tao
Paper 3
Michael Auli
Michael Auli
Paper 2
Neil Houlsby
Neil Houlsby
Paper 2
Austin Reiter
Austin Reiter
Paper 2
IJCAI 2020cvpr2020Papers, Authors, Comments