Google AI Researchers Propose An Artificial Intelligence-Based Method For Learning Perpetual View Generation of Natural Scenes Solely From Single-View Photos

Our earth is beautiful, with majestic mountains, breathtaking seascapes, and peaceful forests. Flying through the intricately detailed three-dimensional landscape, picture yourself taking in this splendor as a bird can. Can computers learn to create visual experiences like this? However, current techniques that combine new perspectives from photographs usually allow only a small amount of camera movement. Most previous research can only extrapolate the content of the scene within the range of view constraints associated with subtle head movements.

In the latest research by Google Research, Cornell Tech, and UC Berkeley, they presented techniques to learn to create unrestricted flythrough videos of natural conditions starting with a single view, where this capacity is learned through a collection of single images, without the need. for camera poses or even multiple views of each scene. This method can take a single image and build a long camera trajectory of hundreds of new views with realistic and varied content during testing, even without ever watching a video during training. This method contrasts with the most recent supervised view generation techniques, which demand multi-view movies and show better synthesis performance and quality.

Also Read :  MX908 Device Enables Correctional Facility Personnel to Quickly Identify Illegal Drugs in Incoming Mail

The basic concept is that they gradually learn to generate flythroughs. Using a single-image depth prediction technique, they first calculate the depth map from the initial view, such as the first image in the image below. After rendering the image to the new camera perspective, as shown in the middle, they use that depth map to create a new image and depth map from that perspective.

This intermediate image, however, has a hole where they can see beyond things into areas that are not visible in the original image, which is a problem. In addition, the fog is because the pixels from the previous frame are being stretched to reveal large objects even though they are now closer to them.

Also Read :  12 Best Black Friday Deals on Google Hardware (2022): Pixel 7, Pixel Watch, Nest Cam

They developed a neural image refinement network to solve this problem, which is an incomplete, low-quality intermediate image and produces a complete, high-quality image with its associated depth map. This synthesized image can then be used as a new starting point to repeat these stages. As the camera progresses deeper into the area, the system automatically learns to build additional scenes, such as mountains, islands, and oceans. This process can be repeated as desired as it refines the image and maps its depth.

Using the ACID dataset, they trained this render-refine-repeat synthesis technique. They then apply this technique to generate multiple fresh perspectives entering the scene along the same camera trajectory as the ground truth video and compare the provided frames to the corresponding ground truth video frames to extract the training signal.

Also Read :  How AI plays its role in medical innovation

With such capabilities, new types of materials for video games and virtual reality experiences can be created, such as the opportunity to relax while floating in an infinite natural setting.

This Article is written as a research summary article by Marktechpost Staff based on the research paper 'InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images'. All Credit For This Research Goes To Researchers on This Project. Check out the paper, code and project.
Please Don't Forget To Join Our ML Subreddit


Rishabh Jain, is a consulting intern at MarktechPost. He is currently pursuing B.tech in computer science from IIIT, Hyderabad. He is a Machine Learning enthusiast and has interest in Statistical Methods in artificial intelligence and Data analytics. He is passionate about developing better algorithms for AI.


Source

Leave a Reply

Your email address will not be published.

Related Articles

Back to top button