T-Fixup is an initialization method for Transformers that aims to remove the need for layer normalization and warmup. The initialization procedure is as follows:
Paper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Reading Comprehension | 1 | 14.29% |
Semantic Parsing | 1 | 14.29% |
Text-To-SQL | 1 | 14.29% |
Decoder | 1 | 14.29% |
Language Modelling | 1 | 14.29% |
Machine Translation | 1 | 14.29% |
Translation | 1 | 14.29% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |