DiscoverLinear DigressionsAttention in Neural Nets
Attention in Neural Nets

Attention in Neural Nets

Update: 2019-06-173
Share

Description

There’s been a lot of interest lately in the attention mechanism in neural nets—it’s got a colloquial name (who’s not familiar with the idea of “attention”?) but it’s more like a technical trick that’s been pivotal to some recent advances in computer vision and especially word embeddings. It’s an interesting example of trying out human-cognitive-ish ideas (like focusing consideration more on some inputs than others) in neural nets, and one of the more high-profile recent successes in playing around with neural net architectures for fun and profit.
Comments 
loading
00:00
00:00
1.0x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Attention in Neural Nets

Attention in Neural Nets

hello@lineardigressions.com (Ben Jaffe and Katie Malone)