Browsing NTNU Open by Author "Cheng, Lei"
Now showing items 1-3 of 3
-
Classification of long sequential data using circular dilated convolutional neural networks
Cheng, Lei; Khalitov, Ruslan; Yu, Tong; Zhang, Jing; Yang, Zhirong (Peer reviewed; Journal article, 2023)Classification of long sequential data is an important Machine Learning task and appears in many application scenarios. Recurrent Neural Networks, Transformers, and Convolutional Neural Networks are three major techniques ... -
Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attention
Yu, Tong; Khalitov, Ruslan; Cheng, Lei; Yang, Zhirong (Journal article, 2022)Self-Attention is a widely used building block in neural modeling to mix long-range data elements. Most self-attention neural networks employ pairwise dot-products to specify the attention coefficients. However, these ... -
Sparse factorization of square matrices with application to neural attention modeling
Khalitov, Ruslan; Yu, Tong; Cheng, Lei; Yang, Zhirong (Peer reviewed; Journal article, 2022)Square matrices appear in many machine learning problems and models. Optimization over a large square matrix is expensive in memory and in time. Therefore an economic approximation is needed. Conventional approximation ...