Transformer是穀歌發表的論文《Attention Is All You Need》提出一種完全基於Attention的翻譯架構

VIP內容

Transformers 在自然語言處理(NLP)任務中是普遍存在的,但由於計算量大,很難部署到硬件上。為了在資源受限的硬件平台上實現低延遲推理,我們提出使用神經架構搜索設計硬件感知轉換器(HAT)。我們首先構造了一個具有任意編碼-解碼器關注和異構層的大設計空間。然後我們訓練一個超級Transformers,它能覆蓋設計空間中的所有候選Transformers ,並有效地產生許多具有重量共享的次級Transformers。最後,我們執行帶有硬件延遲約束的進化搜索,以找到專用於在目標硬件上快速運行的專用子轉換器。對四種機器翻譯任務的大量實驗表明,HAT可以發現不同硬件(CPU、GPU、IoT設備)的有效模型。在Raspberry Pi-4上運行WMT’14翻譯任務時,HAT可以實現3×加速,3.7×比基準Transformer小;2.7×加速,比進化後的Transformer小3.6倍,搜索成本低12,041倍,沒有性能損失。

成為VIP會員查看完整內容
0
4
0

最新論文

Intellectual Property (IP) theft is a serious concern for the integrated circuit (IC) industry. To address this concern, logic locking countermeasure transforms a logic circuit to a different one to obfuscate its inner details. The transformation caused by obfuscation is reversed only upon application of the programmed secret key, thus preserving the circuit's original function. This technique is known to be vulnerable to Satisfiability (SAT)-based attacks. But in order to succeed, SAT-based attacks implicitly assume a perfectly reverse-engineered circuit, which is difficult to achieve in practice due to reverse engineering (RE) errors caused by automated circuit extraction. In this paper, we analyze the effects of random circuit RE-errors on the success of SAT-based attacks. Empirical evaluation on ISCAS, MCNC benchmarks as well as a fully-fledged RISC-V CPU reveals that the attack success degrades exponentially with increase in the number of random RE-errors. Therefore, the adversaries either have to equip RE-tools with near perfection or propose better SAT-based attacks that can work with RE-imperfections.

0
0
0
下載
預覽
Top