In 2013, I and my team launched Cappasity, a 3D scanning software solution company, putting to work five years of experience in 3D technologies that we acquired through game development. 3D reconstruction has been our focus for a long time, and in 2013 we were experimenting extensively with existing products, gathering momentum to produce an original solution. As you can guess, we accomplished this task successfully. The timeline of our journey is presented below, including our story with Intel® RealSense™ technology.
Before starting the company, we had been engaged in 3D reconstruction research to discover the optimum way of creating 3D models. We investigated many of the solutions on the market at that time and found them lacking for several reasons. Some offered low-resolution textures while our content required high-resolution textures. Other solutions offered very good texture quality but required complicated calibration and were quite expensive. So we decided to develop our own instant 3D scanning software.
First, we came up with a list of requirements for a content production system that we thought would be ideal:
- Stationary systems – calibrate cameras once and shoot
- Instant photos for immediate results
- Use of DSLR cameras to achieve photorealistic textures
- 3D sensor agnostic – as technologies evolve, new sensors can be adopted
- Full implementation of client-server technology – each client is a sensor-equipped PC or Android* tablet
- Unlimited number of devices – captured objects can be of various sizes, so any limitations would lead to difficulties
- Sending a model to a 3D printer
- A WebGL plug-in for browser view and SDK for mobile app integration
In the autumn of 2014, after a year of work, we had the opportunity to share our preliminary results with potential investors. Early in 2015, we showcased a system built with Intel RealSense technology at CES 2015.
Image may be NSFW.
Clik here to view.
How we got there
Creating networking code with a proprietary data transfer protocol justified itself from the start. We used PrimeSense* sensors, but putting two or more sensors on a PC made the hardware unstable. At that time we did not know that Intel, Google and other market leaders were working on new 3D sensing devices. However, we opted for a scalable architecture as a matter of course, which enabled our system to easily support any new hardware.
The most time-consuming phase was writing the calibration code. Existing calibration toolkits were not flawless due to lack of insight into the structure and functioning of PrimeSense sensors, which is why we could not calibrate all the necessary settings. Discarding the factory calibration settings of PrimeSense led us to writing a proprietary algorithm based on IR data. Meanwhile, we carried out extensive research of sensor functionality and the algorithms for creating and texturing meshes. This stage involved a lot of changing and redesigning, but in the end we achieved uniform operation of all sensors in our system. As soon as we did, we applied for a patent and currently have a U.S. Patent Pending.
After PrimeSense was acquired by Apple, we decided to shift our attention to alternative solutions. With the foundation already laid, our software became operational with Intel RealSense technology in just 2 weeks after we received the 3D cameras from Intel.
Shifting to Intel RealSense technology brought back the calibration difficulties. Applying necessary colors to point clouds and texturing the meshes obtained from the cloud required knowing how RGB and depth cameras are positioned against each other. With Intel RealSense technology we were planning to use the same proprietary manual algorithm that worked with PrimeSense. However, we were confronted with a sensor limitation—data was only acquired after being processed with filters, and working with RAW data demanded a different approach. Fortunately, the Intel RealSense SDK included the tools to convert coordinates from one space to another, i.e., from Depth Space to Color Space, from Color to Depth Space, from Camera to Color, etc. To avoid wasting time on developing alternative calibration methods, we decided to use this functionality, which proved to work amazingly well. If we compare colored point clouds obtained with PrimeSense and Intel RealSense technology, the latter provides higher quality coloring.
Image may be NSFW.
Clik here to view.
So how does the resulting process work? Sensors and cameras encircle the object to be scanned and are then calibrated and positioned against each other at the right distance and angles. We use DSLR cameras for texturing in case we need to get high-fidelity textures. However, a built-in RGB camera is enough for 3D printing purposes.
Image may be NSFW.
Clik here to view.
Textures are directly projected to and blended with the 3D model, which provides rendering accuracy.
Image may be NSFW.
Clik here to view.
The 3D model is created from point clouds that are formed with an optimal number of sensors. The speed of data capture is 5 to 10 seconds, depending on the number of sensors. And this is all we need to obtain a complete object model!
You can view a 3D model example at Sketchfab.
Our first software products are Cappasity Human 3D Scan and Cappasity Easy 3D Scan. The former is based on instant capturing technology as described above, and the latter was presented at GDC 2015 in March this year. It lets you 3D scan objects using the Intel RealSense 3D camera.
Cappasity Easy 3D Scan
Image may be NSFW.
Clik here to view.
We are really excited that Ultrabook™ devices with the Intel RealSense 3D camera are now available to the public. Once we learned that vendors like Asus, Acer, and Lenovo were going to launch new devices with built-in Intel RealSense 3D cameras, we decided to focus on development of Easy 3D Scan software to have it on the market as soon as possible..
Our scalable 3D scanning solution is affordable for mass audience. For instance, if you have an Ultrabook PC with an Intel RealSense 3D Camera, you can now scan objects in 3D by moving the device around the object or you can use a turntable.
Image may be NSFW.
Clik here to view.
For enhanced 3D scans, we suggest using DSLR cameras, which will produce high-fidelity textures.
Image may be NSFW.
Clik here to view.
Using our software, regular users can now create 3D selfies for free and share them in social networks or order 3D selfie souvenirs. Users can also 3D scan objects and order 3D prints of their 3D models through our software. There is a new trend to create 3D scans of your family and friends. Now you don't have to go anywhere to 3D scan your loved ones—you just need an Ultrabook device with Intel RealSense technology.
Professional users, like CG artists or 3D designers, can use 3D scanning for 3D modeling purposes. A 3D camera lets you create content for mobile applications or games. Our solution offers several optimizing profiles, so your content is ready to be integrated into your application without complicated post processing.
Say you wanted to start a 3D printing company. You may scan objects and humans manually with just one device and Cappasity Easy 3D Scan software. Later you may want to buy more devices with Intel RealSense 3D cameras and do instantaneous 3D scanning with Cappasity Human 3D Scan. This will let you create 3D models really fast, as all devices are connected into the one scanning system with our software, and you scan objects from all sides at the same time. In this case it is less probable that your model, e.g. a human, moves during the scanning process, as the capturing time takes less than a minute.
For instance, e-commerce companies can use our software to create true 3D views of their products. Thanks to our algorithm that projects images onto meshes (polygon model) users get high-fidelity textures.
We published our first samples of 3D models this February and we are constantly updating our portfolio with new models. Please visit our Easy 3D Scan web page to see how the software works.
Why did we choose Intel RealSense technology?
Our decision was based on a number of reasons:
- About 15 models of laptops already support Intel RealSense technology
- Tablets that support Intel RealSense technology are expected to arrive shortly
- Using Intel 3D cameras sets you on a path to B2C sales—an additional way of monetizing our products
- Intel provides a high-quality SDK
Last, but not least, are the high standards of technical, marketing, and business support Intel offers. Our team has partnered with Intel since 2007, and we cannot recall a single time when we did not receive a prompt response from our colleagues at Intel.
Intel RealSense technology easily beats the competition among solutions for 3D scanning at a distance of 1 meter or less.
We can say with confidence that modern technologies are a path to immeasurable opportunities for discovering the depth of the surrounding reality!
Please follow the updates on our progress on Facebook and Twitter.
About the Author
Konstantin Popov has a 10+ years as a CEO of a video game and software company. He brought many complex software projects from a design document to final product. In 2013 Konstantin founded Cappasity Inc. where he focuses on 3D scanning/reconstruction technology and mobile engine development.
Изображение значка:
Clik here to view.
