Oren Melamud

New paper in Science today on playing the classic negotiation game "Diplomacy" at a human level, by connecting language models with strategic reasoning! Our agent engages in intense and lengthy dialogues to persuade other players to follow its plans. This was really hard! 1/5

Not from Twitter, but I've also been part of large-scale layoffs, so I'm looking for a new machine learning / data science position, esp. in NLP Lots of experience in both academic & industry settings I live in France so I'm after jobs that can be done from here please retweet

Apparently DALL-E 2 couldn’t pick the appropriate word-sense for “bass”, and just settled on using both senses. I find it surprising #dalle2

Stunning transcript proving that GPT-3 may be secretly a squirrel. GPT-3 wrote the text in green, completly unedited!

Have an idea for DALL·E? Reply with a caption. I'll generate 20 or so!

I wrote a blog post, trying to give some insights into the incredible recent advances in NLP with language models, and then share some practical tips, based on my industrial experience with the latest and (literally) greatest at @getanyword. anyword.com/blog/understan…

Great overview on the latest and greatest in LM fine-tuning. I'm especially excited about the parameter-efficient methods. This seems to be super useful adapterhub.ml. I also liked the work on Prefix-Tuning that was missing from the summary arxiv.org/pdf/2101.00190….

DALL-E — our new neural network for generating images from text: openai.com/blog/dall-e/

@GabiStanovsky @CseHuji מזל טוב!

I'm happy and excited to share that I'll join the CS faculty at the Hebrew University this fall! @CseHuji

Our much-anticipated BERTology primer is out on arxiv: arxiv.org/abs/2002.12327. Why and how does BERT work? What does it learn, and where is it stored? We review 40+ recent papers in search of answers and give our view on future directions (with #OlgaKovaleva & @annargrs).

How to train a model with 10^11 parameters without running out of GPU memory? Use DeepSpeed from Microsoft Research! It's PyTorch compatible. It partitions the network onto multiple processors automatically and efficiently.

I analyzed compensation & level details of 19k tech workers to find answers to: 1. How long does it take for SWEs to reach a certain level? 2. Compensations across jobs/levels? 3. Do women get paid less than men in tech? 4. Is there a deadline for SWEs? huyenchip.com/2020/01/18/tec…

10 ML & NLP Research Highlights of 2019 New blog post on ten ML and NLP research directions that I found exciting and impactful in 2019. ruder.io/research-highl…

We've created a powerful product that uses AI to enhance social media posts. We're hosting a focus group for social media managers on November 22nd in NYC. We'll ask you a bit about your workflows and give you exclusive first access. Sign up: keywee.typeform.com/to/SKlCSp

I'm hiring an Applied Scientist with experience in #nlp and #ml for my team in @GoKeywee (can be either NYC or TLV). If you want to make a difference in a small dynamic company, let me know! #open" target="_blank" rel="nofollow noopener">keywee.co/careers-new-yo…

Very excited and proud to officially announce the opening of AI2 Israel! allenai.org/ai2-israel/ many thanks to @etzioni and the entire team for making this happen.

Work with @ekshoonyame on automatically generating synthetic clinical notes to overcome privacy considerations that limit our ability to publicly share this kind of data.

New blog post: "A Recipe for Training Neural Networks" karpathy.github.io/2019/04/25/rec… a collection of attempted advice for training neural nets with a focus on how to structure that process over time

A new bigger, better language model by @OpenAI: - Scaled-up version of their Transformer (10x params) - Trained on 10x more curated data (40 GB of Reddit out links w/ >2 karma) - SOTA on many LM-like tasks - Discuss potential for malicious use blog.openai.com/better-languag…