By MATT LINDSEY
POTSDAM -- Have you ever wanted a 3D printed miniature model of yourself or family? By early next year, that may be possible thanks to a Hannawa Falls couple who will offer the service for public use.
Husband and wife Sean Banerjee and Natasha Kholgade Banerjee, both assistant professors at Clarkson, will soon offer 3D printed miniature models of people, captured using a set up of five Kinect version 2 sensors at their lab, the Terascale All-sensing Research Studio (TARS), located at Clarkson University.
“These sensors consist of a color camera and a depth camera packaged into a single device,” Sean Banerjee said. “We physically do 3D printed miniatures in porcelain, plastic, metallic plastic, stainless steel, and aluminum. Gold and silver are available on request.”
This summer, the Banerjees set up the Kinect capture environment at their lab, called the Terascale All-sensing Research Studio (TARS) at Clarkson to observe human behavior.
As a by-product of their prior and current work, they provide 3D printouts of people posing in the Kinect studio.
“The main purpose of our multi-Kinect system is to perform scientific research on understanding human behavior in everyday environments for learning-based recommendations on group dynamics in disciplines such as engineering, business, education, and rehabilitation sciences,” he said. “However, when we saw our students get excited about having their own 3D printed miniatures, we decided to have an open studio where the public can get 3D prints of themselves.”
How Is This Done?
A person steps into the environment where a command is issued to start the capture process. The person does a pose of interest, and the system captures the person, and stores information about their 3D structure.
“We use the information to build a virtual 3D model and print it,” he said. “Unlike existing 3D selfie approaches that use infrared scanners such as the Structure Sensor where the person has to hold their a pose for a considerably long time, our approach requires the person to hold their pose for a second or two, enabling people to make exaggerated poses with minimal effort.”
People can get prints from 2 inches to 10 inches.
A 3D capture is a virtual three-dimensional representation of a person on the computer, created after the person’s image is captured by the sensors. It is also commonly referred to as a 3D model.
A 3D print is a physical tangible object, created using a 3D printer. The 3D model is fed to the 3D printer, and a 3D print is created as a result.
It is possible to create permanent memories such as parents holding their child or married couples holding their hands or kissing.
“Our system can have multi-person captures with people interacting with each other and people interacting with props,” he said. “We captured our student Phillip Tibberts who is a computer science major and avid theater actor holding a skull with a pose from Hamlet.”
Understanding the Technology
The sensors are off the shelf, at $150 a piece. They are the same sensors that are used to interpret people’s gestures for playing games with the Xbox One gaming system. The sensors contain a color camera and a depth camera.
The color camera captures color information about the person. The depth camera, also called a time-of-flight camera, shoots beams of infrared light into the scene and waits for the beams to bounce off objects in the scene and return to the camera.
“Based on the time of arrival of each beam of infrared light and the speed of light in air, the depth camera computes distance to different parts of the scene,” he said. “The principle of working of the depth camera is the same as sonar, except that it uses light waves instead of sound waves.”
A single sensor can capture a part of a person -- for instance, the front of the person -- however, to get a full-range capture of the person, they need to have sensors observing the person from multiple viewpoints.
“Our system connects the information from all five Kinects to create a single 3D model the contains the complete 360 degrees information about the person,” Sean Banerjee said.
Buying 3D Models
Prints are done through an external vendor. The typical prices they charge for a 3 inch miniature in various materials prior to shipping are as follows: porcelain, $25; plastic, $35; metallic plastic, $60; stainless steel, $220 and aluminum, $400.
“We have recently ordered a porcelain print of ourselves captured using our studio,” Sean Banerjee said. “We have done 3D prints of the two of us sitting having a conversation with each other, as well as prints of our students in a variety of poses.”
Shipping ranges from $5 to $10, and depends on the material and location. 3D printed models through the online service are available within approximately three weeks.
“We have actually never captured pets. I think it would be quite cool to try,” he said. “But, we should ask the university to make sure we don’t violate any rules.”
The studio is located in the Center for Advanced Materials Processing (CAMP) building at Clarkson.
Two open houses for the studio will be held every semester and additional ones every summer.
However, they intend to set up the service so that people can schedule a visit outside of the open houses as well. A formal opening is planned in January of 2017.
Dozens Contribute to Work
The work of several students has contributed to the Kinect capture system, and to related projects in 3D model reconstruction and data-driven understanding of human behavior.
Those students include graduate students Marc Bishop, Yijun Jiang, and Lintao Guo, and undergraduate students David Russell, Matthew Inkawhich, Phillip Tibberts, Tim Dunn, Damon Gwinn, Eric Sognefest, David Skufca, Gus Naughton, Abigail Matthews, Lee Taylor, Benjamin Lowit, Scott Straw, Trinity Dickinson, and Milton Griffin.
Prior collaborators at Carnegie Mellon University, including Tim Godisart, Tomas Simon, Taiki Shimba, and Yaser Sheikh, have contributed to the work.
As a postdoctoral researcher in 2014-2015 at Carnegie Mellon University working under the guidance of Dr. Yaser Sheikh (now head of Oculus Research in Pittsburgh), Mr. Banerjee led the design and build of 11 Kinect version 2 studios called the Kinoptic Studio.
The studio was part of a larger 520-camera studio called the Panoptic Studio, which has been featured on Discovery Channel. Natasha Banerjee’s prior work has been in allowing everyday users to manipulate objects in 3D in a single photograph, which won the 2014 Popular Science Best of What’s New Award, and was featured in the New York Times.
Natasha Banerjee is an assistant professor in the department of computer science. She is of Indian origin, but spent her childhood in Lagos, Nigeria. She obtained her B.S. and M.S. degrees from Rochester Institute of Technology in 2009, and her Ph.D. from Carnegie Mellon University in 2015.
Sean Banerjee is an assistant professor in the Department of Electrical and Computer Engineering. He was born in Dublin, Ireland, and grew up in Kolkata, India. He obtained his B.S., M.S., and Ph.D. degrees from West Virginia University at Morgantown, W.v. in 2004, 2006, and 2014 respectively. He was a postdoctoral associate at Carnegie Mellon University from 2014 to 2015.
The couple joined Clarkson in August 2015.
In their free time, they do woodworking, and have built nearly all of their furniture at their house in Hannawa Falls.