V
主页
【VALSE论文速览-114期】An Investigation into Whitening Loss for Self-supervised Learnin
发布人
论文题目:An Investigation into Whitening Loss for Self-supervised Learning 报告嘉宾:翁熙(北京航空航天大学) 作者:翁熙 (北京航空航天大学), 黄雷(北京航空航天大学), 赵磊(北京航空航天大学), Rao Muhammad Anwer (MBZUAI), Salman Khan (MBZUAI), Fahad Shahbaz Khan (MBZUAI) 翁熙,本科毕业于北京航空航天大学人工智能研究院,目前为该学院研一学生,师从黄雷副教授。主要研究方向为自监督学习,以一作身份在NeurIPS 2022发表论文一篇。 个人主页: winci-ai.github.io 报告摘要: A desirable objective in self-supervised learning (SSL) is to avoid feature collapse. Whitening loss guarantees collapse avoidance by minimizing the distance between embeddings of positive pairs under the conditioning that the embeddings from different views are whitened. In this paper, we propose a framework with an informative indicator to analyze whitening loss, which provides a clue to demystify several interesting phenomena as well as a pivoting point connecting to other SSL methods. We reveal that batch whitening (BW) based methods do not impose whitening constraints on the embedding, but they only require the embedding to be full-rank. This full-rank constraint is also sufficient to avoid dimensional collapse. Based on our analysis, we propose channel whitening with random group partition (CW-RGP), which exploits the advantages of BW-based methods in preventing collapse and avoids their disadvantages requiring large batch size. Experimental results on ImageNet classification and COCO object detection reveal that the proposed CW-RGP possesses a promising potential for learning good representations. 参考文献: [1] Weng X, Huang L, Zhao L, et al. An Investigation into Whitening Loss for Self-supervised Learning[C]. Advances in Neural Information Processing Systems, 2022.
打开封面
下载高清视频
观看高清视频
视频下载器
【VALSE论文速览-49期】Towards Face Encryption by Generating Adversarial Identity Masks
【VALSE论文速览-94期】Learning Domain Adaptive Object Detection with Probabilistic……
【VALSE论文速览-30期】基于并行解码的端到端密集视频描述
【VALSE论文速览-21期】Leveraged Weighted Loss for Partial Label Learning
【VALSE论文速览-149期】Robust Test-Time Adaptation in Dynamic Scenarios
【VALSE论文速览-19期】基于Transformer的视觉目标跟踪
【VALSE论文速览-09期】基于多专家一致性建模的医学图像分割校准学习
【VALSE论文速览-107期】Codebook Prior Helps Implicit Neural 3D Representations
【VALSE论文速览-123期】A Closer Look at Few-shot Classification Again
【VALSE论文速览-12期】Point2Skeleton:从点云学习骨架表达
【SAM2+YOLOV8】Meta首个能在图片和视频中实时分割对象的统一开源AI模型!博士精讲论文及源码复现!
【VALSE论文速览-122期】Low-Light Image Enhancement via Structure Modeling and Guidance
【VALSE论文速览-117期】VBLC: Visibility Boosting and Logit-Constraint Learning for ……
【VALSE论文速览-91期】DF-GAN: A Simple & Effective Baseline for Text-to-Image Synthesis
【VALSE论文速览-119期】基于随机归一化层聚合的对抗抵御方法
【VALSE论文速览-74期】Learning What Not to Segment: A New Perspective on Few-Shot……
【VALSE论文速览-98期】基于多面注意力机制的人群计数
【VALSE论文速览-11期】GeoSim: 自动驾驶中高清逼真的相机仿真
【VALSE论文速览-97期】Adversarial Sticker: A Stealthy Attack Method in the Physical……
【VALSE论文速览-81期】Deep Rectangling for Image Stitching: A Learning Baseline
【VALSE论文速览-01期】NeuralRecon: 基于单目视频的实时三维场景重建
【整整200集】不愧是李飞飞,一口把深度学习、计算机视觉、神经网络、图像处理、图像分割、目标检测、物体识别给讲透了,新手小白秒上手!-人工智能/计算机视觉
【VALSE-论文速览89期】β-DARTS: Beta-Decay Regularization for Differentiable ……
20230721 VALSE Student Webinar【开放视觉感知】刘世隆:From Detection to Grounding: A Path……
【三维重建+3D点云】封神之作!计算机博士带你手把手从入门到实战,学三维重建真的不难!人工智能、计算机视觉
【VALSE论文速览-07期】图像风格化鲁棒性的再思考和提升
【VALSE论文速览-53期】BN-NAS: Neural Architecture Search with Batch Normalization
【VALSE论文速览-45期】Neural Body: Implicit Neural Representations with Structured……
【VALSE论文速览-04期】用于3D人体姿态估计的可微的数据增强框
【VALSE论文速览-88期】MSG-Transformer: Exchanging Local Spatial Information by……
【VALSE论文速览-33期】部分标记下的多器官和肿瘤分割算法
【VALSE论文速览-103期】以运动为导向的点云单目标跟踪范式
【VALSE论文速览-57期】DeepMIH: Deep Invertible Network for Multiple Image Hiding
【VALSE论文速览-55期】PyMAF:基于多尺度网格对齐反馈机制的三维人体模型重建
【VALSE论文速览-70期】基于密度和深度分解的自增强非成对图像去雾
论文讲解【英】 | ECCV24: Cross-Domain Few-Shot Object Detection via Enhanced Open-Set..
【VALSE论文速览-36期】高保真任意范围的人脸编辑
【VALSE论文速览-84期】基于双流对比嵌入网络的组合式零样本学习
【VALSE论文速览-109期】基于统一最优运输框架的通用域适应方法
【VALSE论文速览-82期】Causality Inspired Representation Learning for Domain Generalizat