3X faster 3D LIDAR annotation with Magic Select

3D pointclouds are notoriously hard to annotate. Part of the problem is that unlike 2D images, you don’t see all your data in a single view. Instead, you have to navigate and isolate small parts of a scene that are of interest to you.

But an even bigger issue is that it is simply impossible to place a cuboid at the correct position by using your mouse to click on a particular point. This is because a 3D scene is a 2D projection of the real 3D data. When you point your cursor at a particular position, there are a large number of points in 3D space, where you might want to actually place your cuboid. Whatever tool you are using has to guess what that intended position is, because it is impossible for us to convey the crucial depth value to the platform.

The usual flow of annotating an object is:

1. Click somewhere within the object to place the initial cuboid. This cuboid is sized using some default values.
   
2. Move the cuboid to place it roughly in the correct position. Depending on how you are viewing the pointcloud, the initial cuboid could be off in any of the 3 dimensions.
   
3. Use the top, right, front views to finetune the size of the cuboid.

Magic Select

As you can tell, this is pretty time consuming. This is why we developed Magic Select - a new method of drawing cuboids that creates the right annotation in a single attempt. Here is how it works:

1. Draw a rough boundary around the object of interest.
   
2. The tool finds the points that could be within the boundary, and draws the tightest cuboid that contains those points.
   
3. If the object contains extra points, use the top, right, and front views to fine-tune the size of the cuboid.

If you draw the boundary carefully or if the object is relatively isolated - that is - it does not have many objects close to it, you won’t even need to adjust the cuboid after the initial draw. Which is an incredible time-saver. But what about the other objects? What about objects that are close together? In cases where the Magic Select method does not work well, you can continue to adjust the cuboid just like you would on a traditional annotation platform. So you never lose annotation time, you only gain it !

Comparing Magic Select with CVAT

However, talking is cheap. Claims of “x times better” are abundant and don’t really hold meaning without any proof to back them up. So we decided to compare Magic Select with a popular open source labeling platform - CVAT.

Experiment setup:

1. Dataset - LiDAR samples from the NuScenes dataset.
2. Files - 10
3. Annotation type - Cuboids identifying Vehicles, pedestrians and others.

Results

For the 10 sample files, it took CVAT a total of 35 minutes to annotate a total of 60 cars and 28 cars and 60 pedestrians with cuboids. For the same set of files, it took Magic Select 14 minutes to annotate the same number of objects. This represents an improvement of x2.5 times !

You can checkout a short video of a comparison between the two on youtube.

Some observations

1. It’s easy to label cuboids with Magic select. However, it takes a little practice to draw the right outline so that the correct cuboid can be drawn in just 1 go, without need for any adjustments. We saw our labeling team get progressively better at labeling cuboids as they drew more and more. Towards file 6-7, our labelers were drawing perfect cuboids for most of the objects. We have detailed other such tips at the bottom of this blog post.
   
2. For CVAT, we found that the larger the object, the more time it took to label it. For example, it took CVAT much more time to label cars, than it took to label pedestrians. This is because for larger objects, more adjustments need to be made.

Tips for using Magic Select

1. Try to isolate the object, so that you don't accidentally select points from other objects.
   
2. One simple way to do this is to draw the boundary from the top view - that is while viewing the object from the top.
   
3. If the top view does not offer a good view, try to find a view from which as few objects are visible in front of , or behind the object of interest. When you draw a boundary, Magic select chooses all visible points within the boundary. So if other objects are visible, the selection will be inaccurate.
   
4. If you are unable to find such a view, draw a boundary within the object, rather than outside it. This is a good way to draw a cuboid that is at-least positioned at the right place. From there, you can adjust the size of the cuboid through a variety of different methods.

Also checkout this youtube video where we detail the process of labeling cuboids on the Mindkosh annotation platform

Upcoming improvements

We are constantly working on a host of features that will make the labeling process more efficient and less time-consuming. However, one thing we are super excited about, that will make the Magic Select tool even more powerful is the ability to separate ground and non-ground points.

With this feature, users will be able to see ground and non ground points separately. Even being able to hide the ground completely, so they can easily see the objects of interest in an uncluttered environment. This will make Magic Select truly magical! With the ground points removed. Most objects will be separated from each other, meaning you can draw an outline roughly, and the tool will be able to capture the object perfectly. Currently, when you select an object, oftentimes, there are ground points included with the object as well, which means that the cuboid is slightly bigger than it should be. This won’t be a problem anymore and you will get perfect cuboids everytime. The feature is currently in development and will soon be launched for you to try.

With Magic Select and a host of other features, our hope is that the labeling process for annotation of 3D pointclouds will become more efficient and less time-consuming. Data is the fuel powering the AI revolution. If we can make the acquisition of that fuel even a little bit more efficient, we believe it will have a huge impact on both the speed and the quality of AI systems.

You can try our annotation platform for free here.

If you are a student or a researcher from an academic institution and wish to use the platform, please reach out to us at support@mindkosh.com