Robotics, driver-less cars, and forestry are just some areas where 3D point clouds have grown indispensable in recent years. Point clouds provide several advantages over traditional methods of perceiving the environment such as cameras which make them uniquely suitable for a host of industrial applications. In this blog article, we will explore 3D point cloud scans in detail and understand why they are so important.
What is a 3D Point cloud?
A point cloud is a three-dimensional representation of an object composed of many individual data points, each of which is represented in space by a set of x, y, and z coordinates. Together, these points provide an approximation of the objects in the environment.
Each data point in a point cloud has at-least 3 spatial coordinates - X, Y, Z. In addition to these coordinates, points can also carry additional attributes such as color (RGB values), intensity, and occasionally other metrics like temperature.
How is a Point cloud produced?
3D Point clouds are usually captured using advanced 3D laser scanning techniques which collect numerous data points from the surfaces of the scanned items. Some of the most widely used methods are:
- LIDAR: A LIDAR (Light Detection and Ranging) uses laser pulses to measure distances accurately. Lidar systems generate a 3D area map by evaluating the time of flight data to find the exact distance between the sensor and the object. They are widely used in applications like topographic mapping, forestry, and autonomous vehicles, as they provide an accurate representation of large outside environments.
- RADAR: A Radar (Radio Detection and Ranging) produces point clouds by emitting radio waves and measuring the time it takes for the signals to bounce back after hitting an object. Each reflection point provides a data point with X, Y, and Z coordinates, representing the position of the object in 3D space; this data is represented in the form of a point cloud. Radars are widely used in ship navigation and ADAS (Autonomous driving and assistance) systems to provide collision detection capabilities.
- Photogrammetry: Photogrammetry employs multiple photographs of an object or scene taken from different angles to reconstruct 3D models. Using photogrammetry to scan a location requires a complex setup of cameras to make sure the target object is covered from all the required angles. While this method is not suitable for use cases where accuracy is paramount, it produces rich colored point clouds that provide a detailed view of the target object. It is widely used in fields such as cultural heritage preservation and construction.
- Structured light scanning: In this method, the device projects a pattern of light onto an object and measures how the pattern deforms. This technique is often used in manufacturing and quality control.
- Time of flight cameras: Time of flight cameras measure the time it takes for emitted light to return to the sensor. They are commonly used in robotics, gaming, and various other applications.
What is a point cloud scan ?
A point cloud scan refers to the actual process of capturing 3D data points. During a point cloud scan, technologies like Lidar or photogrammetry, as described above, are used to gather the data. A point cloud on the other hand is the result of a point cloud scan, which is a collection of 3D data points representing the scanned object or environment.
Note that depending on the use-case, multiple point clouds may be produced by the capturing device. For e.g. in autonomous driving, Lidars produce several lidar point clouds of the environment per second. This is necessary in dynamic environments as you want to detect any change as soon as it occurs. Each such point cloud can sometimes also be referred to as a point cloud scan.
Commonly used point cloud file formats
There are a variety of point cloud file formats in use today. Some of the most commonly used formats are:
- PCD format
PCD is the most widely used file format for point clouds captured with a Lidar. It comes in two flavors - ASCII and binary, and has a simple header and body structure. The header contains information about the contents of the body, which contains the points themselves. - PLY format
The PLY format can be used to represent point clouds as well as meshes (we talk about meshes later in this article). Like PCD, PLY files can be in ASCII as well as binary, each with their own advantages. PLY files are commonly used to represent object scans such as those in the medical industry. - LAS format
LAS format is widely used for saving aerial point clouds captured by lidars mounted on drones or airplanes. These provide a bird's eye view of large geographical areas and are extensively used in forest management and surveying. LAS format is more complicated that the two formats described above, as it can contain additional information like classification, RGB data etc.
You can learn more about the different point cloud file formats and how to process themin this article.
Point cloud processing software
Unlike images created by cameras, point clouds need more specialized software to visualize and process them. There are a couple of free tools you can use to visualize point clouds.
Cloud Compare
Our personal favorite! CloudCompare is an open-source 3D point cloud processing software capable of handling large point clouds. It offers a variety of tools for visualizing, editing, and analyzing point clouds. It can handle point clouds in all of the formats described in the section above. In addition to handling point clouds, Cloud Compare can also process 3D meshes.
MeshLab
MeshLab is an open-source tool primarily used for processing and editing 3D triangular meshes. However, It also supports point cloud visualization, making it a versatile tool for 3D data.
Point cloud annotation tool
To perform point cloud classification or train Machine Learning algorithms for various use-cases, annotating point clouds with meaningful information about what the points in the cloud represent is crucial.
Annotation of 3D point clouds is particularly challenging because of the difficulty of identifying and marking objects in a 3D scene. The most critical piece when annotating complex 3D point clouds is the annotation tool. Our own 3D point cloud annotation tool has been built to support simultaneous annotation of point cloud as well as any supporting camera images. This allows you to create single objects that can be tracked across sensors as well as across different frames. In addition, we support dense, colorized point clouds with RGB data. You can learn more about our 3D point cloud annotation tool here.
What advantages do point clouds offer?
While different approaches to acquiring and representing 3D data have different limitations, point clouds have several advantages over traditional scene capturing devices like cameras.
- Point clouds are accurate and can capture even the finest of details. While camera images provide a scaled down 2D representation of the world, point clouds provide accurate x,y,z co-ordinates of each identified point on the surrounding objects. This makes them suitable for use-cases that need extreme precision, such as surveying and mapping.
- Unlike cameras, 3D point clouds provide a truly 3D view of the surroundings. This means you can see "behind" an object, which is impossible for images produced by a camera.
- Devices producing point clouds usually work in all weather conditions - fog, snow, rain etc. Cameras can not see through fog, and rain can cloud the lens of a camera, providing an obscured view of the surroundings. This makes them ideal to be used with vehicles, as it provides unmatched safety and redundancy.
Difference between depth maps, meshes and point clouds
In addition to point clouds, depth maps and meshes are other different ways of representing 3D information. Each has its own structure, advantages, and use cases.
Depth Maps
A depth map is a 2D image where each pixel value represents the distance from the sensor to the surface of an object in the scene. Essentially, it’s a grayscale image where pixel intensity corresponds to depth. It is easy to process as it can simply be treated as a 2D array, and can be integrated with other image processing techniques. Depth maps are widely used in robotics and AR/VR. Compared to point clouds however, depth maps contain less accurate information and they cover a smaller area.
3D Meshes
A 3D mesh is composed of vertices, edges, and faces (usually triangles or quadrilaterals) that define the shape of a 3D object. This creates a continuous surface by connecting points in space. Meshes are a more suitable method to represent individual objects, as they provide a clear surface with the ability to add texture and lighting effects. They are widely used in gaming and simulation.
Challenges of using point clouds
While point clouds have several benefits over traditional sensors like cameras, they also present certain difficulties.
- It is common for point clouds to accumulate millions of data points, making them challenging to process and interpret. To manage and manipulate the data, specialized software and hardware are required.
- Different methods of capturing point cloud data result in varying quality and accuracy. Lighting and camera settings can impact the accuracy of a photogrammetry survey, whereas the accuracy of a LIDAR scan can be impacted by environmental factors and the surface type being scanned.
- Devices used to capture point clouds are usually expensive, bulky and require a lot of energy to operate. This can render them unsuitable for small and medium-sized business. However, Lidars continue to improve at a rapid pace, and are expected to get better in these aspects over time with better technology.
Industry applications of point clouds
Traditionally, point clouds have mostly been used in surveying and mapping. In recent years, with the rapid rise of the self-driving industry, point clouds are being adopted in other industries as well.
- Civil engineering - 3D Point clouds can be used to make precise 3D models of structures, roads, and other infrastructure for surveying and mapping.
- Agriculture - 3D Point clouds can be used to make comprehensive maps of crop fields and monitor crop health in agriculture. It can also be used to measure the height and density of trees in forest areas.
- Mining - Point clouds can be used to autonomously map mines deep below the earth's surface. This has the potential to save a lot of lives, as it can describe the entire layout of the mine before a human attempts to go in. Since there might not be any light available in the mines, cameras cannot be used in this setting.
- Drone navigation - Point clouds are a popular choice for navigation sensors for drones, as they provide a more accurate and obstacle free view of the surrounding in 3 dimensions.
Conclusion
To summarize, a 3D point cloud is a powerful representation of spatial data, capturing accurate coordinates of surfaces in three dimensions. The act of capturing a point cloud using a 3D scanning device is called a point cloud scan. Leveraging 3D scanning technologies like Lidar and Photogrammetry, point clouds offer detailed and accurate models of real-world environments. Their versatility makes them invaluable across various industries, from construction and manufacturing to autonomous vehicles and virtual reality. Understanding and utilizing point clouds can greatly enhance your ability to analyze, visualize, and interact with complex 3D data.