I MADE THE BEST 2D SPRITE AI PIPELINE EVER! After testing so much, doing tons of math, trying various different methods I finally got truly the cleanest sprite result I could ask for! You should bookmark this post! ✅ I think this is the happiest 'aha' moment I've had since starting this journey. Day 48 of making a custom webGL game engine using @cursor_ai 🔽🔽🔽 I've been trying to solve sprites since I started this project. I came close a couple of times but it was always something - either dynamic lights wouldn't work, or the genAI results would make them grainy, or I would have to have incorrect alpha etc. After so much struggling, wanting to give up, thinking it's impossible - it was finally done! Perfect matte interpolation of black & white backgrounds resulting in a clean outline of the sprite, half transparent effects keeping their full quality. But this was not enough for me. I wanted more. I wanted those dynamic lights for the most badass anime effects that I'll be creating for the next many weeks! And somehow, I managed to come up with the perfect balance that is nice to the eye with almost no visible imperfections or artifacts when using any kind of dynamic lights. See for yourself! 🔽🔽🔽 How does this magic happen? 1⃣ First, the animation is created using @Kling_ai. I found it the best to keep the quality anime 2D style I wanted, but I guess others may work as well. 2⃣ Then, the frames are carefully extracted at lossless quality and processed via automation through Nano Banana pass to create an opposite background (white->black or black->white). 3⃣ Once we have a proper spritesheet, I then used MiDaS to compute the depth maps, and then algorithmically create PBR maps for lighting. The remaining depth maps can also be used for various effects, although haven't explored it much yet. 4⃣ Finally, all elements are cut into single frames and overlapped with their respective normal maps. If done right, they're pretty much ready for runtime. I like my heroes to be 512 to truly stand out like fighting game characters, but all of this works with far smaller or bigger sprites. Mind you, it's important to note that Nano Banana has a max output of 4k. Therefore you don't want to make your atlases larger than 4k - which is still quite a lot since it allows you 8x8 512 frames. 💡💡💡 I believe the output quality is beyond anything currently available, and I'll be looking into packaging the whole thing into a web app so people can use it. If you find this useful, please repost it! I really wanted to share this method to the world. Thank you for reading! Look forward to day 49 update!
