Scratch the surface of the 3D data acquisition industry and you soon realise it's not just one surface, rather it's a mesh of related industries as dense as a point cloud. The sheer diversity of new 3D products and applications on show earlier this month at SPAR Europe illustrates this well - all part of the same sector, yet each targeting a specific niche, from laser scanning in the harsh conditions of a gassy mine to providing an immersive 3D environment for geologists to plan fieldwork, the variety is almost endless.

Thus, when I caught the Tuesday afternoon session on New 3D Technologies and Applications, it ran the gamut from the photogrammetric 'gigatextures' of xRez Studio to Mantis Vision's handheld scanners, via the point cloud CAD add-ons of Lithuania's InfoEra.

 

InfoEra's Undet point cloud processing software.

As co-founder Egidijus Zilinskas explained, the latter company has developed point cloud processing software called Undet, that works with AutoCAD to increase 2D drafting and 3D modelling productivity at an entry-level price.

A significantly higher price tag (the figure of 60,000 euros was mentioned) is attached to the handheld scanners developed by Israel's Mantis Vision 3D Technology. VP Shabtay Negry spoke about the company's F5 3D handheld camera at SPAR Europe 2010, describing it as a breakthrough technology designed for field use in dynamic scenes by non-experts, whose active triangulation provokes sub mm accuracy at a range of less than 1 m. Use and acceptance of the product is growing he said: "We have customers in Sweden, Netherlands, Japan and Australia," noted Negry. Further developments are also in the pipeline: "We are working on bringing to market a wide angle field-of-view product," said Negry. This could be used for mapping large environments, such as crime scene objects and specified areas. "In six to seven months we will have a short range system with embedded color," he added.

Multiple camera synchronisation has been successfully demonstrated. Mantis Vision is also working on increasing scanning range, from 4.5 m at present to "maybe 10-15 m," said Negry. Another aim is "capturing static and dynamic data to process together." 

According to Negry, the uniqueness of Mantis Vision's active triangulation is the flexibility of the coding, which means you can extend the number of characters and therefore the resolution. "You can have 50,000 coordinates on a single plane - we would like to go up to 500,000 … We are going to increase the resolution by a factor of five to 10 more," he added.

That would make an already impressive piece of technology even more so, but when it comes to high resolution 3D, the photogrammetric gigapixel textures of Greg Downing and his company xRez Studio really take the cake (fittingly the company name is short for 'extreme resolution'). Downing, who has a background in visual effects for feature films started xRez with a partner about six years. What the company has come to specialize in are so-called 'gigatextures': gigapixel photographs overlaid onto 3D digital terrain models. "We do a lot of work in natural science and cultural heritage," explained Downing. He then proceeded to tell the fascinating story of how xRez got asked to produce a gigapixel image of Yosemite National Park in the US to help the park's geologist work out where are the most dangerous places for rockfalls. Unfortunately, the park had no budget for this, so xRez came up with the idea of getting their friends to help: "The only way we could do something this ambitious was to crowdsource it," said Downing. So, with support from Microsoft Live Labs and Google and the help of 20 photographic teams with Gigapans (shooting 10,000 images/hr), they were able to photograph the entire park from 20 locations and generate a final image approximately 300,000 pixels wide.

xRez was also able to use a 1 m resolution aerial lidar model (captured thanks to the one daily hang glider flight allowed by the park authorities) and overlay the gigapixel photo onto the 3D digital terrain model. As well as providing a beautiful, easily searchable visual record of the park, the gigapixel image enabled the park's geologist to assess the possibility of rockfalls near Curry Village, leading to the relocation away from danger of 300 campsites. "The park's search and rescue team is also excited by the possibilities of the gigapixel photo," added Downing. (You can see the complete 45 gigapixel online and read more about the Yosemite Extreme Panoramic Imaging Project here).

Here's another example of the firm's work, this time with Glacier Works, a non-profit organization that uses art, science and adventure to raise public awareness and encourage mitigation of the consequences of climate change:

Khumbu Glacier Approach from xRez Studio on Vimeo.

I guess what this cornucopia of applications and services shows is that it's not about the point cloud so much as it's about the point. (Or, in other words, what are you using the data for?). As Dr. Ivo Milev reminded everyone during the Mobile Scanning for Railways session: "A point cloud is a measurement, not a product." Amen to that.

One of the key trends that emerged from SPAR Europe this year was the importance of the visualization aspect in selling 3D services to sometimes skeptical audiences. "Using 3D is a very good way to show what you are doing," reckoned Dr James Jobling-Purser of 3Dmsi, a firm that specialises in laser scanning of mine environments. A three-dimensional fly-through is not only nice to look at, its visual accuracy also helps get messages across faster to customers and potential customers who may otherwise stick with familiar solutions. As Jobling-Purser succinctly put it: "The guys controlling the development of a project have very little time to discuss using new technology." 

 

Dr. James Jobling-Purser says sometimes a fly through is valuable sales tool.

Simon Barnes, CEO of In Good Measure, agreed that 3D images help to persuade, but only when "you go back and give them sections and volumes they know already that they can check against." 

Paul DiGiaccobe, the Associate Vice President and Director of Strategic Technology for Incubation, HNTB (USA), made a similar point in his presentation on mobile surveying, arguing that "having the lidar data available to the client can be very effective." One interesting example of this was his firm's involvement in a project to reconstruct the Interstate-405 highway in Los Angeles, where the data gathered by a mobile mapping of the construction site during the build phase was actually used by the contractor to successfully see off lawsuits by showing the precise position of road signs at the time of construction. 

For Norbert Benecke of DMT, visualization tools such as Leica Publisher and Autodesk TruView are "very important for planning meetings." Morten Helleman of COWI agreed that, "TruView is a great tool for handing over data."

It's not only the professionals who can benefit from the visualization possibilities: "3D is brilliant for communicating with non-scientists and the general public," reckons Julian Trick of the British Geological Survey. (See my next report from SPAR Europe for more about the BGS's GeoVisionary 3D mapping software.) This is one reason why, in addition to generating working conservation drawings, the Centre for Digital Design at Glasgow School of Art aims to use laser scan data for multimedia, explained the centre's director, Douglas Pritchard. One example of this can be seen in the 3D animations that are now part of the interpretation centre at Rosslyn Chapel. Another will be the 2000-3000 characters who will be scanned and visualized in order to recreate the Battle of Bannockburn at a new museum opening in Scotland in 2014. 

For 3D service providers it seems looking good has never been so important. 

 

Firms were asked to present information that would help in removing a piece of equipment from the boiler room of the World Forum.

As a relative newbie to the industry, one of the most eagerly awaited parts of SPAR Europe for me was the live demonstrations taking place at a variety of locations in and around the World Forum. This was the first chance I had had to see how the experts conduct scan work in the field and it was highly educational. I was able to check out three of the demos as they took place (Faro at the Murder Scene in the exhibit space, Leica in the Boiler Room and Riegl at the Roadway Accident Scene – the original scene replaced by an old Jeep Wagoneer, after a bit of miscommunication about how shot-up cars might be received in a UN zone…).

Something that really stood out for me was the speed of the scanning process in all three cases - with the latest technology, such as the Faro Focus 3D, Leica HDS7000 and Riegl VMX-250 mobile laser scanning system – scan time has been significantly reduced, a development that is definitely helping to drive the growth of the industry.

Unfortunately, the demands of editorial coverage and session moderation meant I wasn't able to see the results of any of those three live demos (look for coverage in coming weeks from Sam Pfeifle). However, I was able to catch the results from the scans of the world-renowned Gemeentemuseum, conducted by Faro’s Wolfgang Hesse, Erich Buttner of Z+F, Andrew Evans of Topcon and Steven Ramsey of Lecia.

 

Investigators are relying more often on scans to help them notice clues they may have overlooked in real time.

All four companies achieved excellent results, but the most interesting thing from my point of view was the different ways they went about the task, for which each team was allotted half an hour. Hesse, for instance, made four scans without targets because of the need for speed. Despite not using target spheres, he was able to achieve an accuracy of between 6 mm and 1.8 cm, with a standard deviation of 1.25 cm using a FARO Focus 3D. “It would be less than 1 cm if I took a little longer, and 4 mm overall using spheres,” he said.

By contrast, Z+F's Buttner took scans from eight positions (high resolution and normal quality, including one scan with color information), whilst Andrew Evans from Topcon scanned from three positions (15 minutes per scan, using up slightly more time than was allotted) and then used tie points to marry the scans. Leica’s Steven Ramsey only scanned from one point as attendance was light for his demonstration, but made up for this by taking scans at a variety of resolutions. “The Leica C10 has the ability to do multiple scans from the same location at different resolutions for different needs – that flexibility is very important,” he said. According to Ramsey, “a complete architectural scan of the front of the building would take around 2 hours, then 10 minutes processing in the office and it is ready to draw.” Impressive stuff.

 

Riegl waits to cross the driveway to the accident scene.

 

Topcon displays the results of their scan of the Gemeentemuseum.

To kick off SPAR Europe 2011, I spent a fascinating morning as part of a tour group at the newly inaugurated CSI Lab, part of the Netherlands Forensic Institute (NFI) on the outskirts of The Hague. The lab is a state-of-the-art forensic test and training facility developed by the NFI in collaboration with a range of knowledge institutes and high-tech companies (for a full list click here). 

The aim of the game for CSI Lab is to be able to record and store a crime scene in digital form and in three dimensions so that it may be examined 'virtually'. As our host, Jurrien Bijhold from the NFI, explained, the institute was already augmenting its vast array of pathology and forensics techniques with 3D modelling and image analysis for scenario testing (e.g. how did the crime happen?), body height measurements and facial image comparison. Now, with the CSI Lab project, it is taking things one step further. A real crime scene has been recreated within the lab. This is monitored by a plethora of high-tech recording and sensing technologies, including a spectral camera for detecting traces of blood or proteins and systems for recording the heart rate and movements of forensics specialists as they carry out their tasks, providing valuable information that can be used to improve their work. 

 

The CSI Lab takes interrogation of a crime scene to the next level.

In addition (and this is where it gets really cool), the physical recreation of the crime scene is augmented by a virtual simulation that users can interact with in 3D. Project partners e-semble and TNO (both Dutch firms) are developing a crime scene rapid 3D scenario tool (codenamed "PD3D") that will make it possible to stage forensic training and briefing scenarios virtually and without time delays using point clouds. The instructor will also be able to influence the virtual reality scenario based on the performance and actions of the person receiving the training. 

E-semble project manager Steven van Campen, who gave us a live demonstration of this 'serious gaming' tool and of the progress to date on PD3D, explained that his firm is developing "3D models for operational support, witness interrogation and court scenario visualization."  

TNO, in addition to its work on PD3D, has developed a "3D CSI Tool" to combine scans from different angles to create one reference frame. The tool links a Kinect to a tablet device and TNO's own software and it is designed to enable multiple forensics investigators to access the same information and even (using a type of augmented reality) annotate and add traces for colleagues to view and respond. It is hoped that the 3D CSI Tool will eventually be suitable for field work. 

Also in development at the CSI Lab is a pair of 3D glasses that can be used to mark traces in the virtual crime scene. As the NFI's Andro Vos noted in a press release to announce the official opening of CSI Lab, in future the goal will be to develop new forensic innovations, combine existing innovations in a convenient and open way and validate the techniques so that they may be applied more broadly. 

After the demonstrations, we heard from two of the forensics experts that, if all goes to plan, could well be using techniques developed at the CSI Lab for crime detection (or training) in the future: Paul Lucas and Alex de Bruijn from the Forensic Visualization Unit of the Dutch Police. Messrs Lucas and de Bruijn gave an insightful presentation on the history and current status of forensic visualization within the Netherlands. 

The first use of 3D at a crime scene in the country was in 2003. Today, the technology is used in a wide range of settings, including potential homicide cases, major road and rail accidents, calamities (e.g. plane crashes) and police internal investigations. As Mr. Lucas explained, the visualization unit is using 3D data to generate panoramic HDR images, 3D point clouds, 3D models, 3D crime scene scenarios and high definition 3D maps. Hardware in use includes Nikon, Panoscan and Spheron cameras, stationary Leica scanners, handheld Mantis scanners and Fugro's FLI-MAP and DRIVE-MAP airborne and mobile mapping solutions. That's all backed up with a host of software, including Leica Cyclone, Alice Labs, Maya/3DSMAx, AutoCad, Cubic Convertor and Cubic Connector, Geomagic, Mantis software, Quick Time, Autodesk Stitcher and Photoshop CSS.

Mr Lucas and Mr De Bruijn showed how they have used 3D for forensic analysis in cases ranging from a murder to a death caused by a tram. The latter was particularly mystifying at first. "I am convinced we would never have had the solution of this case if we didn't do it in 3D," said Mr. Lucas. He also noted that witness statements tend to be more accurate/credible when 3D scans of crime scenes take place. However, the Forensic Visualization Unit tries to avoid using 3D animation so as not to influence witness recollection through a photorealistic representation of the crime scene. "We have only used animation twice," noted Mr. Lucas. 

As yet, there are only small teams of police officers working in forensic visualization within the various regions of the Netherlands, but the demand for their expertise is increasing and new services are being developed. One such instance involved using CT scan data and 3D printing to give an accurate picture of a bullet's trajectory in a case where the bullet was lodged in a person's skull and it was too dangerous to remove it. 

"A laser scanner scans without prejudice," concluded Mr. De Bruijn. "A surveyor knows what he wants to measure; in forensics, we don't know what we want to measure, therefore a laser scanner is very useful."

Below are some photos of the tour to give you a flavor of the facility:

This week sees the launch in Japan of Sony's Tron-inspired 3D headset the HMZ-T1. Now, as well as looking hella cool, with its separate high-definition OLED display panels (doing away with 'crosstalk' between left and right images) and "Virtualphones" surround sound, early reviews suggest it also packs enough technological punch to become a much sought-after new way of watching movies and playing video games.

A 3D headset that makes you look like Geordi from Star Trek certainly has the wow factor, but, at the end of the day, it is just another playback technology. What I think is much more exciting is the possibility of using consumer technology to capture 3D data and create new applications in entertainment, shopping, security and even medicine. That is what third-party developers are doing with the Microsoft Kinect controller-free interface for Xbox 360, using the Kinect for Window SDK beta programming toolkit, creating everything from a virtual piano (just tap your fingers on an empty desk to play) to a virtual way of trying on dresses in a store, to 3D videoconferencing and video surveillance tools, to applications for manipulating an MRI to guide cancer surgery. 

Solutions Architect and Technical Manager, Microsoft Research Connections EMEA (UK), Kenji Takeda, will be highlighting a whole heap of 3D scanning and Natural User Interface applications for Microsoft Kinect at SPAR Europe this week. 

One of the most exciting ones I have seen is demonstrated by this new music video from the Canadian duo, New Look. 

Created by Tim & Joe, the video for "Nap on the Bow" takes place in a wonderful underwater 3D world made entirely from data captured using Kinect. To see exactly how they made the video, click here. 

Of course, 3D data capture isn't only useful for building virtual aquatic worlds, it can also be used for as-built measurement of industrial equipment and installations below the waves in real life. Marc Daeffler and Arnauld Dumont's presentation on Underwater Photogrammetry at SPAR Europe promises to be highly instructive in this regard, particularly concerning developments in image and data processing and underwater camera calibration. One not to be missed.