Wow! "Attention is All You Need" (i.e., Transformers) was inspired by the Alien's communication style in the movie Arrival.

Arrival (2016) followed by Attention is All You Need (2017) imdb.com/title/tt254316…

@IntuitMachine You have to share where this is from

@IntuitMachine Can someone ELI5 the analogy to me?

@IntuitMachine So cool. A couple fun points. Stephen Wolfram and his son consulted on the language used in the movie. And the movie is based on a Ted Chiang short story, “Story of Your Life.”

@IntuitMachine like this?

@IntuitMachine where is this from

@IntuitMachine Is this wrong? Attention is from Bahdanau et al who was inspired by his own (human) strategy of doing translation, attending over words of source sentence to sequentially emit words in the other. Transformer then deleted the recurrence and stacked Attention with MLP in a ResNet.

@IntuitMachine I don’t think it was the source of the idea - @kyosu came up with it based on work on attention for query similarity. But Heptapod B was a way for me to visualize his idea and describe to people.

@IntuitMachine "Please encode this 32 dimensional 5 trillion bit alien sentence into a single atomic symbol. Do NOT use markdown"

@IntuitMachine whoa where is this from?

@IntuitMachine Not being obtuse but I don't see how any of this has any connection to transformers.

@IntuitMachine the original source of the story replied to fact check it: it was a useful metaphor, but not the inspiration for the idea

@IntuitMachine Have you read the short story that inspired the movie? It's great x.com/timourxyz/stat…

@IntuitMachine That’s really interesting, I loved that movie thanks for sharing that!

@IntuitMachine Cool