Darryl Barnhart

Catch @DarrylBarnhart speak @NVIDIAGTC today about our exciting work @LatentSpaceAI on scaling LLM with retrieval! 📚 There’ll be an open QA in 30 mins: reg.rainfocus.com/flow/nvidia/gt…

@wightmanr Reminds me a lot of the equalized learning rate ideas from Tero Kerras et al in the StyleGan lineage. Also by using std norm init, the variance scaling in optimizers like ADAM is better defined. The grad has the same expected var as the weights (one).

This paper is worth a read. Interesting tidbits on ResNets, EfficientNets, RegNets, (batch) normalization, augmentation, regularization, bottlenecks. `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - arxiv.org/abs/2101.08692

We’ll be presenting our paper “Low Distortion Block-Resampling with Spatially Stochastic Networks” today at 9pm PST🧩🏰🎨 Joint work w/ @martinarjovsky @darrylbarnhart @iantbd @LatentSpaceAI! ➡️ paper: arxiv.org/abs/2006.05394 ➡️ poster: neurips.gather.town/app/pl8sKCby9F… #NeurIPS2020

Excited to share our @nvidiaai work on reconstructing high-quality 3D models from monocular photographs and making StyleGAN a controllable neural renderer. @ChenWenzheng @HuanLing6 @JunGao33210520 Yinan Zhang, Antonio Torralba @FidlerSanja paper: arxiv.org/abs/2010.09125

Check out Performer, a generalized attention framework based on the Transformer architecture, which implements the novel FAVOR+ algorithm to provide linearly scalable, low-variance and unbiased estimation of attention mechanisms. Read more at goo.gle/3mhiCu4

@SebAaltonen I'd be curious to trade notes on this, as I implemented something very similar about a year ago. Rust + vulkan using atlased mipmaps. I also had a raster pipeline to generate SDF tiles for real time streaming of analytic surfaces.

It's just a single mip level big atlas containing pow2 tiles (cubes).

Just started writing Vulkan mipmapped volume texture upload. Seems like "fun"... Plan: 1. Map a region in host visible buffer (capable of storing all mips). 2. Copy data in buffer. 3. Issue vkCmdCopyBufferToImage with N VkBufferImageCopy entities, each with a different offset.

Purposefully overfit neural networks are an efficient surface representation for solid 3D shapes In arxiv.org/abs/2009.09808 with Thomas Davies, @DerekRenderling, we make a few observations:

v1.6: native mixed-precision support from NVIDIA (~2x perf improvement), distributed perf improvements, new profiling tool for memory consumption, Microsoft commits to developing and maintaining Windows PyTorch. Release Notes: github.com/pytorch/pytorc… Blog:pytorch.org/blog/pytorch-1…

See how ML teams, like @LatentSpaceAI, working on cutting edge deep learning research use W&B Reports to collaborate remotely, debug models iteratively and to showcase results as mini research papers. #machinelearning #deeplearning #100daysofmlcode app.wandb.ai/latentspace/pu…

@ProgrammerLin I've done something similar but with sparse textures and lods as well. Let's you do 65k render distance with similar performance numbers. But pretty complicated to implement! Your materials are way nicer too.

Today we say bye to #rtx and begin trialing a new voxel engine: its own ray tracing backend, infinite world generation, up to 8K render distance, 4K world heights, full multiplayer support & zero-cost acceleration structure updates. 1080p ~80 fps 1spp GI here. #screenshotsaturday

Promising improvements coming in cuDNN v8. Excited to see how they pan out, especially with #PyTorch.

@wightmanr We're using your repo to do finetuning with some of these models. Noticed that using the Adam optimizer instead of SGD+Momentum (default) yields a much higher test accuracy on our datasets (e.g. 95% vs 89%). Is this expected?

@wightmanr Awesome! Thanks!

@wightmanr Hey Ross, do you have more complete training hyperparameters somewhere? e.g. for the skresnext family github.com/rwightman/pyto…

@holdenkarau Congrats!

I now have my physical green card in my possession :) No more carying a fedex envelope filled with immigration papers in my back pack for me.

We're at #NeurIPS2019 — Come chat with us about scaling generative models for real-time, applied ML use cases (hint: a game! 👾👾👾) Reach out to @latentcodes and @iantbd who will be onsite!

ML Best Practices: Test Driven Development at Latent Space. buff.ly/2zZteYp 🖊by @wandb #DataScience #MachineLearning #TDSPick 🚀

After 7 years, Manifold Garden has finally launched! It is available now on @AppleArcade & @EpicGames. Trailer: youtube.com/watch?v=jZ_yxZ… Apple Arcade (iOS, tvOS, macOS) - apple.co/-Manifold Epic Games Store (Windows) - epicgames.com/store/en-US/pr… Thank you for your support.

Haven’t made the switch to @PyTorchLightnin yet? check out @LatentSpaceAI’s reason for switching. Latent space is building the first fully AI-rendered 3D game engine. link.medium.com/tMLJQwftH0 @iantbd @PyTorch #NeurIPS2019 #ICLR2020 #icml #cvpr #AAAI2020

Our #siggraphasia paper "Non-linear sphere tracing for rendering deformed signed distance fields" is online now! Sphere trace deformed SDFs without inverting your deformation function! cs.dartmouth.edu/~wjarosz/publi… @wkjarosz @_AlecJacobson @DerekRenderling