Providing a virtual environment that matches the actual world, the recent widespread rise of 3D applications, including metaverse, VR/AR, video games, and physical simulators, has improved human lifestyle and increased productive efficiency. These programs are based on triangle meshes, which stand in for the intricate geometry of actual environments. Most current 3D applications rely on triangular meshes, which are collections of vertices and triangle facets, as a basic tool for object modeling. Reckless in its ability to streamline and accelerate rendering and ray tracing, it is also useful in sensor simulation, dense mapping and surveying, rigid-body dynamics, collision detection, and more. The current mesh, however, is mostly the output of talented 3D modelers using CAD software, which hinders the ability to mass-produce large-scene meshing. So, a prominent topic in the 3D reconstruction community is developing an efficient mesh approach capable of real-time scene reconstruction, especially for big scenes.
One of the most difficult challenges in computer, robotics, and 3D vision is the real-time mesh reconstruction of big scenes from sensor measurements. This involves re-creating scene surfaces with triangle facets near each other and linked by edges. Constructing the geometric framework with great precision is essential to this difficult challenge, as is reconstructing the triangular facet on real-world surfaces.
To accomplish the goal of real-time mesh reconstruction and simultaneous localization, a recent study by The University of Hong Kong and the Southern University of Science and Technology presents a SLAM framework called ImMesh. ImMesh is a meticulously developed system that relies on four interdependent modules that work together to provide precise and efficient results. ImMesh uses a LiDAR sensor to accomplish both mesh reconstruction and localization at the same time. ImMesh contains a novel mesh reconstruction algorithm built upon their earlier work, VoxelMap. More specifically, the proposed meshing module uses voxels to partition the three-dimensional space and enables quick identification of voxels containing points from new scans. The next step in efficient meshing is to reduce dimension, which turns the voxel-wise 3D meshing problem into a 2D one. The last stage uses the voxel-wise mesh pull, commit, and push procedures to incrementally recreate the triangle facets. The team asserts that this is the initial published effort to recreate large-scale scene triangular meshes online using a conventional CPU.
The researchers thoroughly tested ImMesh’s runtime performance and meshing accuracy using synthetic and real-world data, comparing their results to known baselines to see how well it worked. They started by showing live video demos of the mesh being rapidly rebuilt throughout data collection to ensure overall performance. After that, they validated the system’s real-time capability by thoroughly testing ImMesh using four public datasets acquired by four separate LiDAR sensors in distinct scenarios. Finally, they compared ImMesh’s meshing performance in Experiment 3 to preexisting meshing baselines to establish a benchmark. According to the results, ImMesh maintains the best runtime performance out of all the approaches while achieving high meshing accuracy.
They also demonstrate how to use ImMesh for LiDAR point cloud reinforcement; this method produces reinforced points in a regular pattern, which are denser and have a larger field of view (FoV) than the raw LiDAR scans. In Application 2, they accomplished the goal of scene texture reconstruction without loss by combining their works with R3LIVE++ and ImMesh.
The team highlights that their work isn’t very scalable regarding spatial resolution, which is a big drawback. Due to the fixed vertex density, ImMesh tends to reconstruct the mesh inefficiently with numerous small facets when dealing with big, flat surfaces. The proposed system does not yet have a loop correction mechanism, which is the second limitation. This means that there is a chance of gradual drift due to cumulative localization mistakes in revisited areas. If revisiting the problem happens, the reconstructed results may not be consistent. Adding this recent work on loop identification using LiDAR point clouds will help the researchers overcome this issue in this work. By utilizing this loop detection approach, it would be possible to identify loops in real-time and implement loop corrections to lessen the drift’s impact and enhance the reliability of the reconstructed outcomes.
Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter..
Dhanshree Shenwai is a Computer Science Engineer and has a good experience in FinTech companies covering Financial, Cards & Payments and Banking domain with keen interest in applications of AI. She is enthusiastic about exploring new technologies and advancements in today’s evolving world making everyone’s life easy.
