Project Work: Week 4

A new track with AtlasNet

This week I attempted to dive into experimentation with the AtlasNet network I went back to last week. I prepared data from the swords dataset I created before following the advice I got from the author previously, but after finally getting the network ready it appears that further changes have to be made to the network for me to actually use my own data.

Messaging with model authors last week

Coming from last week, I really wanted to try an alternative network to the Soltani et al. depth map one since I still have not obtained good results from that network. Last Wednesday I contacted the author of that model with my situation asking for advice, but did not receive a response all week. On the other hand, I did receive a response from the AtlasNet author, who told me that to train the AtlasNet autoencoder I would only need point clouds. This was a good sign because when I initially approached the network I was confused by several available data types in addition to point clouds such as normalized clouds and renderings of each model.

Working with AtlasNet

Thus I worked to take the dataset I prepared before of 30 sword meshes and convert these into .PLY point clouds so that they could be fed into the AtlasNet network. I used MeshLab for this task, which allowed me to easily import each mesh and perform Monte Carlo sampling on it to obtain the point cloud. I sampled with 12,000 points each to get good coverage of each mesh.

Sword point cloud

With these point clouds, I attempted to insert them as a new class into AtlasNet’s data folder, which only has out-of-the-box support for ShapeNet classes. Trying to get the network to train, I received errors informing me that my computer’s CUDA version was too old: v7.5 while this required v8.0+. Thus I had to take a detour through the laborious process of uninstalling and then reinstalling an updated version of CUDA on my machine. After some time though I was able to get it done and run the training script.

Unfortunately, when I did this I received further errors about not being able to load the data properly. In going through the code for loading data I discovered that even though the AtlasNet author told me I would only need point clouds to train the autoencoder, the data loader script expects all data types anyway and was throwing and error that the renderings were not present. I wasn’t quite sure what to do here so I made another post on the project GitHub page. The author actually responded to my post quite rapidly, but he told me: “If you want to train on your own data, it’s probably easier to write your own dataloader.” This was disappointing to hear since (at least currently) the data loader code seems a little too dense and linked with other parts of the paper, including little hacks the author had to include to get things to work. In order to even run the network at all, it seems like I would have to essentially learn about and modify a major part of the model code. As far as time goes this seems risky to me since I will not even be able to see if the results work until I put in all that effort.

Back to Soltani et al.

With this roadblock, I was driven to go back to the Soltani et al. depth map network and dabble with it more. In particular, I had discovered a “silhouette” mode before that takes binary black/white maps of the images instead of depth maps as input. Since before I had an issue with the swords dataset of portions of the images vanishing because the objects were not spherical enough, I thought this mode might alleviate this problem.

In trying to run the network though, I received an error that apparently was due to me having switched from CUDA v7.5 to v10.0. It seems that v10.0 is too new for a certain PyTorch function that was trying to execute, and thus an error was thrown. I found helpful posts online with steps to solve the problem, and tried to follow them to fix the issue. However one of the last steps still is not working for me as of now, and I have not been able to proceed further on this. Given that I am so close though this will be the first issue I tackle in the next week.

Next steps

Based upon my interaction with AtlasNet, I can try looking around some more but I think I may not be continuing with that network. I still like the Soltani et al. network because it at least provides enough connection points and documentation for me to use my own data and train without having to rewrite a significant amount of code. However, so far the results have not been particularly impressive. Thus, my plan for the next week is to make a concerted effort to try and get the Soltani network to work again, especially with the “silhouette” mode I have not fully explored.

Given that I need to find a working model soon though, I also plan to search out for any other alternative models I can try so that I do not continue to rely on this single network if it will not work for me in the end.