Generative A-Eye #16 - 14th Oct,2024
A (more or less) daily newsletter featuring brief summaries of the latest papers related to AI-based human image synthesis, or to research related to this topic.
Just a note that since I’m moving house in this period, there are definitely going to be some interruptions and catch-ups in this newsletter.
SceneCraft: Layout-Guided 3D Scene Generation
Whenever a new games-related or vid-2-vid diffusion method comes out, I’m always looking to see if the video examples backtrack and can repeat the same rendition of a view as the method did earlier (one recent example hit on Hacker News recently, but proved to have no such ‘memory’ or overarching scene understanding).
One example from the project page of this new initiative hints that the system can perform this functionality, but it seems doubtful. LDM rendering systems need a CGI-style prior that allows them to backtrack and remember what happened before, and that’s a hard prospect at the moment.
‘[A] novel method for generating detailed indoor scenes that adhere to textual descriptions and spatial layout preferences provided by users. Central to our method is a rendering-based technique, which converts 3D semantic layouts into multi-view 2D proxy maps. Furthermore, we design a semantic and depth conditioned diffusion model to generate multi-view images, which are used to learn a neural radiance field (NeRF) as the final scene representation’
http://export.arxiv.org/abs/2410.09049
https://orangesodahub.github.io/SceneCraft/
Context-Aware Full Body Anonymization using Text-to-Image Diffusion Models
I’m not entirely sure how big the market for full-body anonymization is really likely to be, but it’s interesting to see how generic and idealized most of the replacements are in this new system.
'[A] workflow for full body person anonymization utilizing Stable Diffusion as a generative backend'
http://export.arxiv.org/abs/2410.08551
My domain expertise is in AI image synthesis, and I’m the former science content head at Metaphysic.ai. I’m an occasional machine learning practitioner, and an educator. I’m also a native Brit, currently resident in Bucharest.
If you want to see more extensive examples of my writing on research, as well as some epic features (many of which hit big at Hacker News and garnered significant traffic), check out my portfolio website at https://martinanderson.ai.