DesignMyRoom AI - Google AI Hackathon 2024 entry

Hello everyone,

I recently submitted my entry for the Google AI Hackathon on devpost, and used Panda3D in my submission!

DesignMyRoom takes in a user input, and decorates a room based on that input using Google Gemini and Kenny’s Incredible Furniture Kit.




The most challenging part was definitely the prompt engineering. It was really difficult to consistently get accurate and efficient results, and combined with how little Gemini knew about the exact details of the assets it used, it often got rotations incorrect. Also Gemini kept loading in files that simply don’t exist despite listing out the files numerously.

And like every other LLM, Gemini does mess up in comprehending the exact positional instructions of each prompt, and placement of objects can be random and inconsistent. However I think I’ve managed it well compared to when I first realized its flaws for the first time.

Source code
Devpost submission

I will greatly welcome any feedback!

3 Likes

Thats pretty cool! I bet this kind of thing would work better if it could be more tightly integrated into a model, like how the new gtp4o can process voice, vision, and text natively, a native 3D model would probably take this kind of thing to the next level.

Then we can take auto-gen scenery/procedural worlds to the next level with AI!