V
主页
[Short Review] Axial Attention in Multidimensional Transformers
发布人
Join 'Speech and Language Technologies' Meetup group https://www.meetup.com/speech-and-language-technology-meetup-group/ to see weekly paper reading schedules and discussions. Axial Attention in Multidimensional Transformers Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, Tim Salimans We propose Axial Transformers, a self-attention-based autoregressive model for images and other data organized as high dimensional tensors. Existing autoregressive models either suffer from excessively large computational resource requirements for high dimensional data, or make compromises in terms of distribution expressiveness or ease of implementation in order to decrease resource requirements. Our architecture, by contrast, maintains both full expressiveness over joint distributions over data and ease of implementation with standard deep learning frameworks, while requiring reasonable memory and computation and achieving state-of-the-art results on standard generative modeling benchmarks. Our models are based on axial attention, a simple generalization of self-attention that naturally aligns with the multiple dimensions of the tensors in both the encoding and the decoding settings. Notably the proposed structure of the layers allows for the vast majority of the context to be computed in parallel during decoding without introducing any independence assumptions. This semi-parallel structure goes a long way to making decoding from even a very large Axial Transformer broadly applicable. We demonstrate state-of-the-art results for the Axial Transformer on the ImageNet-32 and ImageNet-64 image benchmarks as well as on the BAIR Robotic Pushing video benchmark. We open source the implementation of Axial Transformers.
打开封面
下载高清视频
观看高清视频
视频下载器
[Long Review] Cascaded Diffusion Models for High Fidelity Image Generation
[Short Review]Conformer Convolution-augmented Transformer for Speech Recognition
[Long Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using
语音文本技术论文阅读 Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recogni
十分钟看懂谷歌铁布衫BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised ...
[Long Review] CLAS: Deep context: end-to-end contextual speech recognition
[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs
[Short Review] Xception: Deep Learning with Depthwise Separable Convolution
十分钟看懂微软大力金刚掌WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack
十分钟看懂谷歌易筋经BERT
[Long Review] Transfer Learning from Speaker Verification to Multispeaker TTS
十分钟看懂谷歌金钟罩Transformer以及语音的LAS模型
十分钟看懂脸书虎爪绝户手 - 虎BERT - HuBERT: Self-Supervised Speech Representation Learning
[Long Review] Deduplicating Training Data Makes Language Models Better
[Short Review] Cascaded Diffusion Models for High Fidelity Image Generation
语音NLP论文阅读 Token-level Sequence Labeling for SLU using Compositional E2E Models
语音文本技术论文阅读 UniSpeech-SAT - Universal Speech Representation Learning with Speaker
[Long Review] Towards Zero-Label Language Learning
[Short Review] Transfer Learning from Speaker Verification to Multispeaker TTS
详解微软零样本语音合成VALL-E
[Long Review] Conformer: Convolution-augmented Transformer for Speech Recogniti
语音文本技术论文阅读 OpenAI最新的Whisper ASR也会像GPT-3一样火起来吗？
[Long Review] GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
[Long Review]Kullback-Leibler Divergence: Listen, Attend, Spell and Adapt ASR
十分钟看懂谷歌W2v-BERT: Combining Contrastive Learning and Masked Language Modeling
[Short Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using
[Long Review]Switch Transformers: Scaling to Trillion Parameter Models with
语音文本技术论文阅读 Improving Speech Recognition Accuracy of Local POI Using Geographical
30分钟吃透Transformer架构！pytorch从0实现！ | 代码逐行讲解 | 源码开放 | 高效入门
CV论文阅读OPENAI CLIP(1/3):Learning Transferable Visual Models From Natural Language
语音文本技术论文阅读 SNRi Target Training for Joint Speech Enhancement and Recognition
语音文本技术论文阅读 One-Edit-Distance Network (OEDN) in Mispronunciation Detection & ASR
语音文本技术论文阅读 RefineGAN - Universally Generating Waveform Better than Ground ...
【2024最新完整版】不愧是李宏毅教授！一口气学完机器学习、深度学习、强化学习、NLP、生成式AI等课程！一套全解决！
强推！这可能是唯一能将AI Agent讲清楚的课程了，公认最适合新手入门Agent大模型实战系列，3小时全流程解读分析，简直比刷剧还爽！人工智能|大模型
很少有人把【NLP自然语言处理】说的这么通俗易懂了！NLP中最重要的核心内容全整理好啦！这么好的课程还没人看？我不更了！！
性能翻倍！LSTM+Transformer王炸创新，荣登Nature，精度高达95.56%！！整理11种融合创新方案！机器学习|深度学习|计算机视觉
还得看吴恩达！一口气讲透CNN、RNN、GAN、LSTM、YOLO、transformer等六大深度学习神经网路算法！真的不要太爽~（AI人工智能丨机器学习）
这绝对是全B站最系统（没有之一）的人工智能基础教学！内含机器学习、深度学习、强化学习、NLP、等多个方向解析，零基础必看！
语音文本技术论文阅读 XLS-R: Self-supervised Cross-lingual Speech Representation Learning a
