News in Brief

Hologram Particle Screen for McEnroe vs. McEnroe

Subscription service ESPN+ has streamed what they claim to be the world’s first tennis match featuring a real person versus a virtual player. In the match streamed on 7 May, John McEnroe faced off against his younger self – represented by a virtual player powered by artificial intelligence (AI).

The 45-minute film showcased how the match was created using a combination of AI and machine learning, plus five virtual avatars of John McEnroe from pivotal points of his career. The avatar game system was projected on a hologram particle screen and was a simulation of gameplay with a system of ball launchers and ball return robots.

When (the real) McEnroe sends the ball over the net, the avatar responds to the direction of the real ball. As the avatar swings, a new ball is fired from the ball cannon and then flies through a smokescreen at a precise point in space to make it appear from the avatar’s racket position. From what is shown in the trailer, the hologram particle screen makes the virtual John McEnroe look a bit blurry and distorted.

The production company, Unit 9, analysed hours of footage from John’s matches throughout his entire career and recorded hundreds of shots, strokes and movements. In total, they recorded 308 shots with over 259 loops and blends to really capture his footwork and well-known shot-making and volleying skills.

NVIDIA and Stanford Collaborate on Holographic Glasses

Graphics processing unit (GPU) designer, NVIDIA, has released a total of 16 research papers which will be on display at SIGGRAPH 2022, taking place 8-11 August in Vancouver, Canada, and online.

The papers and projects highlight the latest academic collaborations in graphics and AI research to enable advancements in content creation, virtual reality (VR), real-time rendering and 3D simulation.

One of the projects to be presented focuses on ultra-thin holographic glasses for virtual reality. Most virtual reality users access 3D digital worlds by putting on bulky head-mounted displays, but NVIDIA states that its researchers are working on lightweight alternatives that resemble standard eyeglasses.

A collaboration between NVIDIA and Stanford researchers has packed the technology needed for 3D holographic images into a wearable display just a couple millimetres thick, according to the company (see HN February 2021). The 2.5mm display is less than half the size of other thin VR displays, known as pancake lenses, which use a technique called folded optics.

NVIDIA stated that researchers accomplished this feat by approaching display quality and display size as a computational problem and co-designing the optics with an AI-powered algorithm.

While prior VR displays require distance between a magnifying eyepiece and a display panel to create a hologram, this new design uses a spatial light modulator, a tool that can create holograms right in front of the user’s eyes, without needing this gap. Additional components – a pupil-replicating waveguide and geometric phase lens – help to further reduce the device’s bulkiness.

NVIDIA added that this is one of two VR collaborations with Stanford that will be shown at the conference, with another paper proposing a new computer- generated holography framework that improves image quality while optimizing bandwidth usage.

A third paper in this field of display and perception research, co-authored with New York University and Princeton University scientists, measures how rendering quality affects the speed at which users react to on-screen information.

Hologram Doctors Visited Space Station

NASA recently disclosed that in October 2021, flight surgeon Dr Josef Schmid and other team members became the first humans holoported from Earth to space, CNET reports ¹. Schmid described holoportation as ‘a type of capture technology that allows high-quality 3D models of people to be reconstructed, compressed, and transmitted live anywhere in real time’. The team members were able to see and speak to International Space Station (ISS) astronauts.

Schmid was joined by Fernando De La Pena Llaca, CEO of software provider Aexa Aerospace. NASA says that while the technology has been around since 2016, this is the first time it has been used in ‘such an extreme and remote environment such as space’.

NASA expects that ‘we’ll use this for our private medical conferences, private psychiatric conferences, private family conferences and to bring VIPs onto the space station to visit with astronauts.’ NASA added that the technology could be used in future missions to Mars— and on Earth, in extreme environments like Antarctica.

The technology behind the telepresence can be described as high-quality 3D models of the holoporters being developed, digitally compressed, transmitted and reconstructed in the spaceborne lab – all in real time.

Meanwhile, a mixed reality display aboard the ISS, namely Microsoft’s HoloLens, allowed both the holoporters and astronauts to see, hear and interact with one another as though they were in the same physical space. Astronaut Thomas Pesquet, for instance, had a two-way conversation with Schmid and De La Pena in the middle of the ISS despite being miles away from the holoporters.

NASA is also looking at combining the technology with augmented reality, which would let holoporters move around the space station, and haptics, which can simulate touch through vibrations or motors.

The purpose of this technology is to help with extra-terrestrial telemedicine for astronauts on ISS building projects and ultimately to benefit future deep space exploration, although this may present some technical obstacles. Earth-to-space communications typically experience up to 20-minute delays when talking to systems in outer space, NASA says. Holoportation, however, is meant to be ‘live’, so holoporters could overcome this inherent delay by maintaining their telepresence on-board for real-time communication as demonstrated by the recent prototype delivered to the ISS.

‘It doesn’t matter that the space station is traveling 17,500 miles per hour and in constant motion in orbit 250 miles above Earth,’ Schmid said. ‘The astronaut can come back three minutes or three weeks later, and with the system running, we will be there in that spot, ‘living’ on the space station.’ NASA is also optimistic that this could have direct applications on Earth, such as for researchers working in extreme environments or military operation specialists.

‘Imagine you can bring the best instructor or the actual designer of a particularly complex technology right beside you wherever you might be working on it,’ Schmid said. ‘You can work on the device together, much like two of the best surgeons working during an operation. This would put everyone at rest knowing the best team is working together on a critical piece of hardware’.

1 - www.cnet.com/science/space/nasa-holoported-a-doctor-onto-the-international-space-station/.