Project Rodin is an investigation into the ability of state-of-the-art generative ML models to synthesize polygonal meshes that could be used in a video game. The project’s name comes from Auguste Rodin, a famous French sculptor. Through this project I wish to explore if it is possible to create a pipeline that uses one of these ML models to automatically generate 3D models and then converts them into a format ready for use in a game engine such as Unity.
My work is being done as part of EECS 499 independent study research at the University of Michigan. I will be using this website to post weekly updates on the status of the project, so please check it out!
Today I presented Project Rodin at the Fall 2019 U-M Eningeering Design Expo and had the opportunity to share my findings with the general public.
As the project’s timeline is wrapping up, I did some final experiments and then finished by finally running the VAE model I have been working in conjuction with the transfer learning method I demonstrated previously.
I continued various experiments to try and determine why my current VAE implementation has not been giving good reconstructions yet.
This week I took big steps to modify the application so it can actually sample novel meshes. This chiefly involved the conversion of the AtlasNet autoencoder architecture into a variational autoencoder (VAE).
After the results I obtained last week, this week I focused on expanding my dataset to try and get better results. Before I did this however I ran a few experiments to get a better idea of the situation.
This week I continued my work I started last week with the AtlasNet generative autoencoder network. While I have not yet achieved satisfactory results, I did make some important understandings this week that helped me understand the network better.
This week signalled a more complete shift in my efforts as I fully transitioned from working with the Soltani et al. depth map network to AtlasNet. However, initial work with AtlasNet shows more potential that I encountered with the previous model.
This week I attempted to dive into experimentation with the AtlasNet network I went back to last week. I prepared data from the swords dataset I created before following the advice I got from the author previously, but after finally getting the network ready it appears that further changes have to be made to the network for me to actually use my own data.
This week I began to investigate alternative options to the multi-view depth map model of Soltani et al. However, I still continued some limited experimentation with the model, which did allow me to gain some more insight into what might have been going wrong last week. This side effort was also motivated by the fact that alternative models I found either do not seem promising enough, or do not provide adequate enough documentation for me to parse through them.
This week I continued on the work I completed last week with generating shapes from a Pokémon dataset using Soltani et al.'s multi-view encoder-decoder model. One of the major findings from last week was that my dataset was too diverse in its shapes, which resulted in the generation of many uninteresting blob-like shapes that did not express any distinctive features. Thus this week I created a new dataset that was more targeted to attempt to solve the problem.
This week I made my first forays into running generative networks for synthesizing 3D models for video games. My goal was to take an existing model and try training it with a dataset of 3D game assets to see what sorts of outputs it would produce.
To understand who would benefit most from my research, I investigated various stakeholders working in the game industry. As mentioned in a previous post, from a product perspective my research would mostly affect 3D modeling software and repositories, but the people it would mostly affect work in game studios or other creative enterprises. Here are some of the archetypes I found.
My research comes is at interesting intersection in that product-wise it would impact 3D modeling software and those in that industry who make it, but a much larger number of affected people would be those using said software in the video game development industry (and theoretically in other media industries such as animated films and marketing). Thus in my market research I investigated both sectors but for products focused on 3D modeling software and model repositories.
I searched for pre-existing academic research in my area of interest in creating a generative ML model for creating 3D models for video games. Here is a selection of the most relevant articles I found with links and annotations.