Benjamin Schwarz

Come say hi 👋 tomorrow afternoon at my @EuroGeosciences talk! #EGU23

The analogy between GPTs of today to the CPUs of early days of computing are interesting. GPT is a funny kind of programmable text computer. Have to think through it more 🤔 but e.g.: ## Memory GPT-4 RAM is ~log2(50K vocab size)*(32K context length)/(8 bits/byte) ~= 64kB, roughly a Commodore64. Just as then, optimizing this precious resource is critical. GPT registers are the residual stream. There are d_model of them, e.g. GPT-3 has ~12K registers. VLIW architecture vibes. ## CPU The LOAD instruction is the Attention mechanism, except it can address by both location and/or content. The STORE instruction is forced every n_layer number of clock cycles. The ALU are the MLPs + LayerNorms. Awkwardly, as their params are not shared across layers, the ALU changes at each clock cycle. Optionally the MLPs may also be interpreted as supporting a kind of fixed knowledge database lookup. The programs always takes the form [[LOAD, ALU]*N, STORE]*M, where N is n_layer and M is num_tokens. ## Architecture GPT feels closer to a fixed-function than stored-program computer because the number of parameters is so large. In contrast, the description length of a CPU is very low and all the action is in the memory configuration. Another way to look at it is that GPT is a much more bloated/complex computer. Which is fine because it is not engineered but optimized and the upshot is that the programs can be shorter.

@Fl0rianWagner @RWTH @dgg_news Unfortunately extremely busy then, but will try to stop by for a day or so!

@seismicbeam @RWTH Thank you Benni! See you in Bremen for @dgg_news ?

Some personal news: I have been appointed Full Professor of Geophysical Imaging and Monitoring at @RWTH #Aachen University starting today.

The time commitment of scientific peer-review should not be underestimated, but I find the process incredibly rewarding. I am very honored to have been selected an "Outstanding Reviewer 2022" by the GJI editorial board - thank you @RAS_Journals !

Forschungsreise ins Mittelmeer: Ein Vulkan zerstörte vor 3600 Jahren die minoische Kultur auf Kreta. Wie hat er sich seitdem entwickelt? Das untersuchen die Geophysiker @JonasPreine und Christian Hübscher der UHH mit Bohrkernen. Alle Infos zur Expedition: uni-hamburg.de/newsroom/im-fo…

I‘m just now seeing this — a big thank you to everyone involved! 🎉 Hope to see you all again at this year‘s #EGU23!

Kipppunkte der Erde 🌍 aber auch Kipppunkte der Hoffnung 🔆. Mein neuer Beitrag! spiegel.de/wissenschaft/n…

@seismo_koel Congrats Paula, that’s amazing! ❤️

Signing off for awhile, not because of Christmas holidays or moving platforms, but because this little wonder joined us and is keeping me busy ever since. Merry Christmas from us four!

3-year postdoc in induced seismicity and seismic imaging at Oxford. Application deadline 28th January, 2023. my.corehr.com/pls/uoxrecruit…

Postdoc opportunities in Data Science at @Brown_DSI. Those interested in joining my research group to work on SciML (sites.brown.edu/bergen-lab) should apply here: apply.interfolio.com/115615. Review begins Jan 1, open until filled. Please share!

Looking for a PhD in seismology based in Oxford @OxUniEarthSci? Interested in deep Earth structure, mantle upwellings, phase transitions, or urban seismology and monitoring seismicity? Projects are listed here: earth.ox.ac.uk/teaching/gradu…, but also keen to discuss your ideas! (1/n)

Today's wonderful #GoogleDoodle celebrates American geologist and oceanographic cartographer Marie Tharp and the discovery of the Mid-Atlantic Ridge Rift Valley google.com/doodles/celebr…

#PhDposition(s) 👨‍🎓👩‍🎓 in Geophysics at @anuearthscience of @ourANU -pls. contact me for expressions of interest by 09/12/22. Exciting research in global/planetary/Antarctic/blue earth observational seismology + computational geophysics flavor. @AGUSeismology @SEDI_AGU @EGU_Seismo

Double job alert! @ChrisDaviesGeo and I have 2 NERC-funded post-doc research positions to fill. Come spend 3 years working with us on "Modelling of Earth’s Core and Magnetic Field" Details at jobs.leeds.ac.uk/Vacancy.aspx?r… Please get in touch if any questions

#phd #jobalert The Swiss Seismological Service at ETH Zurich is looking to fill a PhD position in Volcano Seismology: Joint seismic tomography and stress change modeling at Kīlauea Volcano, Hawaiʻi Check it out: jobs.ethz.ch/job/view/JOPG_…

Interested in applying for a PhD at @OxUniEarthSci? Curious about planet formation, Earth habitability or tectonics? Mad about magnets? I'm advertising 4 projects (all available via NERC DTP or STFC departmental funding). For details read on... 1/6

“It’s a really bleak moment, not only because of the reports showing that emissions are still rising (...), but we also have so much scientific evidence that we are very, very close to irreversible changes." PIK Director @jrockstrom in @guardianeco. theguardian.com/environment/20…

@jeffreypauperez @GFZ_Potsdam Cool, congrats! :)

Happy to share that I'm starting my independent research as a Discovery Fellow in #OrganicGeochemistry @GFZ_Potsdam. I will be studying organic carbon dynamics in redox-impacted Arctic soils over the next 3 years.

The #julialang #sciml packages now have tons of customized error messages to make it easier for newcomers to get into the ecosystem. If there are any that we have missed, please open an issue!