Tobias Domhan

@dioxuslabs Nice! What's the status of supporting text input like input fields or text areas?

Dioxus 0.7 will be natively rendered 🤯 We spent a year building GPU-based HTML and CSS renderer. ⚒️ It already supports hot-reloading, accessibility(!), wgpu, mobile platforms, and many widgets. macOS apps are self-contained and <6mb.

@jb55 good to know it's doable. Maybe worth giving egui another try.

@jb55 Are you able to use text inputs with the virtual keyboard?

@emjotde Ah I was only aware of ruskie and in a place that used to have ruskie they suddenly only had ukrainskie with the same filling (potatoes and cheese)

@tdomhan hm... I think those are just two different kinds :)

Back in Poland for a few days and I am wondering if "Russian Mustard" (musztarda rosyjska) has been cancelled everywhere or am I imagining things? [It's not actually from Russia, it's just a name for a popular type of mustard which now seems to just not exist anymore?]

Interested in an internship with us working on multilingual LLMs? Apply here: amazon.jobs/en/jobs/248175…

@dariods_ @logseq Same. It looks interesting, but has not been useful to me.

@logseq recently updated their graph view to include forces. I'm still not convinced that the graph is very useful. It's nice to look at, especially when you get started, but I still haven't found practical, repeatable use cases. Keen to hear different opinions on this...

What does it mean for MT evaluation if the reference translations are also machine-translated? Our finding: trained metrics learn to cope. arxiv.org/abs/2312.00536 WMT23 paper from my @AmazonScience internship, with @tdomhan, @sonytrenous, @RicoSennrich and @EvaHasler.

@Bsunter With Safari plus Adguard I'm getting 98%. I also realized the browser pop-up window on top of Twitter doesn't run extensions and therefore ads dont get blocked.

Safari + Adblock pro extension: 20% blocked

Mobile Browser Ad Blocking Benchmarks (iOS) I tested: Safari (+ Adblock pro extension) Firefox Arc Orion Brave This site lets you see how effective your ad blocker is. It loads 150 different types of ads and trackers and tells you how many were blocked d3ward.github.io/toolz/adblock.…

@Bsunter bone conduction headphones are also pretty nice to hear both surroundings plus something else (up to some limit of outside noise)

Intrigued by the idea of a personal sound bubble. I've seen a couple wearable parametric (directional) speaker concepts now. Not a fan of wearing headphones.

@TobiasKatsch @ykilcher @_akhaliq @HochreiterSepp @orvieto_antonio @_albertgu Looks interesting. How well does it fare compared to RetNet, say in Table 1?

Excited to announce GateLoop! arxiv.org/abs/2311.01927. In this paper, we generalize linear recurrent models (S4, S5, LRU, RetNet) by employing data-controlled state transitions. GateLoop outperforms the state-of-the-art architectures for natural language modeling.

@camis397 😢

@jxmnop Are there no larger vision models because they don't improve anymore over smaller models or because no one has trained them yet?

An amazing mystery of machine learning right now is that state-of-the-art vision models are ~2B parameters (8 gigabytes) while our best text models are ~200B parameters (800 gb) why could this be? philosophically, are images inherently less complicated than text? (no right?)

Interested in parallel data mining for Machine Translation? Consider participating in the WMT23 shared task for parallel data curation. Deadline: September 1st, 2023. www2.statmt.org/wmt23/data-tas…

@bchesky Search for specific attributes like places with a sauna would be great.

What else can we improve about Airbnb? We will prioritize your top suggestions

@DFinsterwalder @rasbt @terrible_coder One reason not to do this is to expose the model to its own predictions. Nowadays with an RL the model is actually later exposed to its predictions though (where when sampling we can no longer parallelize computation of course).

@rasbt @terrible_coder @tdomhan I don't really see why anyone would not train a decoder transformers like this. You don't need to know if the output of the third row a, b, c would be d or not if your training data is a,b,c,d,e,f,g anyway.

Are transformers truly more easy to parallelize than recurrent neural networks? Yes and no. For encoder-style models like BERT, this is of course true. But in recent months, transformer has become synonymous to GPT (a decoder-style model). Sure, we can parallelize each individual step in GPT, but due to its autoregressive nature, both training and inference is pretty sequential due to masked multi-head attention. Specifically. masked multi-head attention requires information from previous tokens in the sequence. This means that each token in the sequence must be generated sequentially, leading to a sequential nature of both training and inference.

@rasbt @terrible_coder Yeah, both for the decoder of seq2seq models and GPT style models this is usually how training is done. The same can be applied for the user supplied prompt where computation only becomes sequential for the newly generated part.

@terrible_coder @tdomhan Ah I see. But would that impact qualitative performance? Do people commonly train decoders like that? I thought it's literally one word at a time inspired from language translation tasks (like in the original transformer)

@barrelltech @logseq @James_Senva These look great. Can't wait for them to be in Logseq!

@logseq @James_Senva There will be plenty of options to change already! Color, borders, stripes, hover effect, column order 👀 If you don't like the winning theme, the other 2 are doable via plugins. Feel free to reach out if you want help setting one up ^^

Voting time! Please help shape Logseq 🙏 Which table do you like best? (see tweets below) (Focus on the shape; colors will be customizable)

@marian_nmt What's happens if you write that this function doesn't exist? I also have gotten functions that don't exist or arguments to functions that don't exist. When stating this is the case sometimes you get something useful. Other times not so much 🤷‍♂️

and very sure about it:

Ha, after days of working with chatGPT for explorative coding (read: I don't know what I am doing) a first major blunder (there is no curandGenerateBeta function):