Video

Unknown · 0:00

This StatQuest episode explains how the GELU, SiLU (Sigmoid Linear Unit), and SwiGLU activation functions were derived from a probabilistic reinterpretation of dropout, and why they tend to overfit less than plain ReL...

Read the full summary on tuber