HomeResearch-feedChannelRankingsGCTTHU AI TROpen DataMust Reading
HomeSelf-Supervised Learning
Self-Supervised LearningSelf-supervised learning is essentially a method of unsupervised learning, and we will set up a "Pretext" and construct Pesdeo Labels to train the network model according to some characteristics of data. The self-supervised model can be used as a pre-training model for other learning tasks to provide a better initial training area. Therefore, self-supervised learning can also be regarded as a general visual representation for learning images.
Year#CitationLike
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Zhenzhong Lan,Mingda Chen, Sebastian Goodman,Kevin Gimpel,Piyush Sharma,Radu Soricut
ICLR, (2020)
Conclusion
We have convincing evidence that sentence order prediction is a more consistently-useful learning task that leads to better language representations, we hypothesize that there could be more dimensions not yet captured by the current self-supervised training losses that could crea...
default
Comments
Weibo
Cited by619BibtexViews430Links
0
0
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
Alexei Baevski,Steffen Schneider,Michael Auli
ICLR, (2020)
Conclusion
We evaluate models on two benchmarks: TIMIT is a 5h dataset with phoneme labels and Wall Street Journal is a 81h dataset for speech recognition
default
Comments
Weibo
Cited by51BibtexViews165Links
0
0
Multi-task self-supervised learning for Robust Speech Recognition
Ravanelli Mirco, Zhong Jianyuan, Pascual Santiago, Swietojanski Pawel, Monteiro Joao, Trmal Jan,Bengio Yoshua
ICASSP, pp.6989-6993, (2020)
Conclusion
The proposed problemagnostic speech encoder+ architecture is based on an online speech distortion module, a convolutional encoder coupled with a quasi-recurrent neural network layer, and a set of workers solving self-supervised problems
Must Reading
Comments
Weibo
Cited by26BibtexViews167Links
0
0
Big Self-Supervised Models are Strong Semi-Supervised Learners
Ting Chen,Simon Kornblith,Kevin Swersky,Mohammad Norouzi,Geoffrey Hinton
Methods
Conclusion
Similar approaches are common in NLP, we demonstrate that this approach can be a surprisingly strong baseline for semi-supervised learning in computer vision, outperforming the state-of-the-art by a large margin
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by25BibtexViews364Links
0
0
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Alexei Baevski,Henry Zhou,Abdelrahman Mohamed,Michael Auli
NeurIPS 2020, (2020)
Methods
Conclusion
We presented wav2vec 2.0, a framework for self-supervised learning of speech representations which masks latent representations of the raw waveform and solves a contrastive task over quantized speech representations
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by19BibtexViews368Links
0
0
Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification
Guile Wu,Xiatian Zhu,Shaogang Gong
AAAI, pp.12362-12369, (2020)
Methods
Conclusion
We presented a Tracklet Self-Supervised Learning method for unsupervised image and video person re-id
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by9BibtexViews152Links
0
0
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Kahn Gregory,Abbeel Pieter,Levine Sergey
Methods
Conclusion
We presented BADGR, an end-to-end learning-based mobile robot navigation system that can be trained entirely with selfsupervised, off-policy data gathered in real-world environments, without any simulation or human supervision, and can improve as it gathers more data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by8BibtexViews215Links
0
0
Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
Senthil Purushwalkam,Abhinav Gupta
NeurIPS 2020, (2020)
Methods
Conclusion
We demonstrate that these self-supervised representations learn occlusion invariance by employing an aggressive cropping strategy which heavily relies on an object-centric dataset bias
Must ReadingMust Reading
Comments
Weibo
Cited by6BibtexViews185Links
0
0
Self-supervised Learning on Graphs: Deep Insights and New Direction
Wei Jin,Tyler Derr,Haochen Liu, Yiqi Wang,Suhang Wang,Zitao Liu,Jiliang Tang
Methods
Conclusion
We first introduce various basic supervised learning pretext tasks for graphs and present detailed empirical study to understand when and why SSL works for graph neural networks and which strategy can better work with GNNs
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by5BibtexViews236Links
0
0
Self-supervised Learning: Generative or Contrastive
Liu Xiao,Zhang Fanjin,Hou Zhenyu,Wang Zhaoyu, Mian Li,Zhang Jing,Tang Jie
Conclusion
There exist several comprehensive reviews related to Pre-trained Language Models, Generative Adversarial Networks, Autoencoder and contrastive learning for visual representation
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by5BibtexViews4110Links
0
0
Multi-Task Self-Supervised Learning for Disfluency Detection
Shaolei Wang,Wanxiang Che,Qi Liu, Pengda Qin,Ting Liu,William Yang Wang
national conference on artificial intelligence, (2020)
Methods
Conclusion
Experimental results on the commonly used English Switchboard test set show that our approach can achieve competitive performance compared to the previous systems by using less than 1% of the training data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by5BibtexViews198Links
0
0
Distilling Localization for Self-Supervised Representation Learning
Zhao Nanxuan,Wu Zhirong, Lau Rynson W. H.,Lin Stephen
Methods
Conclusion
We identified a strong error pattern among self-supervised models in their failure to localize foreground objects
default
Comments
Weibo
Cited by3BibtexViews149Links
0
0
Digging Into Self-Supervised Monocular Depth Estimation
Clément Godard,Oisin Mac Aodha,Michael Firman,Gabriel J. Brostow
ICCV, pp.3827-3837, (2019)
Methods
Conclusion
We showed how together they give a simple and efficient model for depth estimation, which can be trained with monocular video data, stereo data, or mixed monocular and stereo data
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by252BibtexViews220Links
0
0
Revisiting Self-Supervised Visual Representation Learning
Alexander Kolesnikov,Xiaohua Zhai,Lucas Beyer
CVPR, (2019)
Conclusion
As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by188BibtexViews129Links
0
0
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
Longlong Jing,Yingli Tian
arXiv: Computer Vision and Pattern Recognition, (2019): 1-1
Methods
Conclusion
This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by148BibtexViews144Links
0
0
S4L: Self-Supervised Semi-Supervised Learning
Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov,Lucas Beyer
International Conference on Computer Vision, (2019): 1476-1485
Methods
Conclusion
We further showed that S4L methods are complementary to existing semisupervision techniques, and Mix Of All Models, our proposed combination of those, leads to state-of-the-art performance
Must ReadingMust Reading
Comments
Weibo
Cited by114BibtexViews149Links
0
0
Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera
Fangchang Ma, Guilherme Venturelli Cavalheiro,Sertac Karaman
international conference on robotics and automation, (2019)
Methods
Conclusion
This framework requires only sequences of RGB and sparse depth images, and outperforms a number of existing solutions trained with semi-dense annotations
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by105BibtexViews161Links
0
0
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Xin Wang,Qiuyuan Huang,Asli Çelikyilmaz,Jianfeng Gao,Dinghan Shen,Yuan-Fang Wang,William Yang Wang,Lei Zhang
CVPR, (2019): 6629-6638
Methods
Conclusion
In this paper we present two novel approaches, Reinforced Cross-Modal Matching and Supervised Imitation Learning, which combine the strength of reinforcement learning and self-supervised imitation learning for the visionlanguage navigation task
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by104BibtexViews224Links
0
0
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath,Dawn Song
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), (2019): 15637-15648
Methods
Conclusion
We found large improvements in robustness to adversarial examples, label corruption, and common input corruptions
Must ReadingMust ReadingMust Reading
Comments
Weibo
Cited by92BibtexViews82Links
0
0
Keywords
Supervised LearningSelf-supervised LearningComputer VisionSemi-supervised LearningLearning Artificial IntelligencePose EstimationDeep LearningFace RecognitionFeature ExtractionImage Reconstruction
Authors
Sergey Levine
Sergey Levine
Paper 7
Abhinav Gupta
Abhinav Gupta
Paper 5
Xiaohua Zhai
Xiaohua Zhai
Paper 3
William Yang Wang
William Yang Wang
Paper 3
Andrew Zisserman
Andrew Zisserman
Paper 3
Sebastian Thrun
Sebastian Thrun
Paper 3
Dacheng Tao
Dacheng Tao
Paper 3
Michael Auli
Michael Auli
Paper 2
Neil Houlsby
Neil Houlsby
Paper 2
Austin Reiter
Austin Reiter
Paper 2
IJCAI 2020cvpr2020Papers, Authors, Comments