V
主页
VideoCon: Robust Video-Language Alignment via Contrast Captions
发布人
【加群】 一起来刷arxiv,请加vx: pwbot02(请备注:b站arxiv) 【彩蛋】 可以试试/ask + 你的提问和本篇论文进行交流 【论文标题】 VideoCon: Robust Video-Language Alignment via Contrast Captions 【论文简述】 这篇论文介绍了一种新方法,用于提高视频语言对齐模型对视频标题中语义对比变化的鲁棒性。尽管这些模型在海量数据上进行了(预)训练,但它们对视频标题中的语义对比变化并不具备鲁棒性。我们的研究通过识别一系列对比不对齐情况,例如替换实体、动作和翻转事件顺序等,来解决这个问题。为此,我们引入了一个名为VideoCon的视频语言对齐数据集,该数据集通过一个大型语言模型生成合理的对比视频标题,并解释原始标题和对比标题之间的差异。然后,我们通过对VideoCon进行微调,构建了一个生成式视频语言模型,用于评估视频语言蕴含关系并生成解释。我们基于VideoCon构建的对齐模型在人工生成的对比标题上显著优于当前模型,在视频语言对齐任务的AUC值上提高了12个百分点。最后,我们的模型在时间广泛的视频语言任务(如文本到视频检索和视频问答)中展现出了零-shot性能表现,并且在使用新颖视频和人工制作的标题和解释时表现出了卓越性能。我们的代码和数据可在https://github.com/Hritikbansal/videocon上获得。 【引导阅读的问题】 如何提高视频语言对齐模型的鲁棒性? 【论文链接】 https://arxiv.org/pdf/2311.10111
打开封面
下载高清视频
观看高清视频
视频下载器
FlashDecoding++: Faster Large Language Model Inference on GPUs
Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling
Compressing Context to Enhance Inference Efficiency of Large Language Models
BitNet: Scaling 1-bit Transformers for Large Language Models
Video Language Planning
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
Kosmos-G: Generating Images in Context with Multimodal Large Language Models
CLEX: Continuous Length Extrapolation for Large Language Models
Are Large Language Models Post Hoc Explainers?
Large Language Model Cascades with Mixture of Thoughts Representations for Cost-
Toward Joint Language Modeling for Speech Units and Text
Exponentially Faster Language Modelling
Llemma: An Open Language Model For Mathematics
Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
Making Large Language Models Perform Better in Knowledge Graph Completion
Amortizing intractable inference in large language models
基于BIM的建筑结构生成式智能设计
Retrieval meets Long Context Large Language Models
Language Models can be Logical Solvers
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Ubody开源人体三维源数据集-smplx-三维建模-姿态估计
Can Large Language Models be Good Path Planners? A Benchmark and Investigation o
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Interactive Task Planning with Language Models
LayoutPrompter: Awaken the Design Ability of Large Language Models
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning
Large Language Models Cannot Self-Correct Reasoning Yet
LLAMA3.1 中文数据集微调效果如何?
GraphLLM: Boosting Graph Reasoning Ability of Large Language Model
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation
OceanGPT: A Large Language Model for Ocean Science Tasks
AutoMix: Automatically Mixing Language Models
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
Memory Augmented Language Models through Mixture of Word Experts
Creative Robot Tool Use with Large Language Models
FLAP: Fast Language-Audio Pre-training
Video as the New Language for Real-World Decision Making
Ranking LLM-Generated Loop Invariants for Program Verification