XR Developer News - January 2024
Hi all, I'm trying something new. I couldn't really find any one spot keeping track of all things XR development, so I thought I'd give it a go myself. Hope it proves useful to others!
In this edition a lot of Apple Vision Pro news, Augments and Meta Spark, Xreal Air 2 Ultra, a bunch of software/framework/SDK updates, a few interesting links and upcoming events. To steal a line from a very solid XR podcast: let's dive right in!
Apple Vision Pro and visionOS
This was clearly the story of the month, as pre-orders started and the launch on February 2nd approaches. As all XR and tech news channels have been flooded with Vision Pro news, I'll skip most of it, and highlight a few developer-specific points here:
- Apple released details on the process for submitting visionOS apps to the App Store. Amidst all the details the thing that stood out immediately was the sentence "Follow these style guidelines when writing about Apple Vision Pro and visionOS, including when marketing your app outside of the App Store. Refer to your app as a spatial computing app. Don’t describe your app experience as augmented reality (AR), virtual reality (VR), extended reality (XR), or mixed reality (MR).". It obviously remains to be seen how strictly this will be enforced, especially in marketing outside of the App Store.
- Apple also released a new Q&A document with a range of specific technical details on visionOS development.
- UploadVR has a detailed comparison of the specifications to Quest 3. Useful to get a bit of a feeling for how much more performance budget apps and games could use.
David Heaney
- Unity's announced that its visionOS support (through PolySpatial) is now in early access based on visionOS 1.0, Xcode 15.2 and Unity 2022.3.18f1. It released v1.0.3 of the relevant packages (changelogs here, here and here).
IsaacsUnity
- There's a chance Apple may roll out Apple Vision Pro internationally before WWDC in Q2.
郭明錤 (Ming-Chi Kuo)
- To me, looking at the early Apple Vision Pro impressions by various publications, although pre-orders seem to have taken off very fast, it really feels like a developer kit. Reportedly it's too heavy for extended use, which will strongly limit the type of use cases Apple presents in its marketing materials like watching movies or doing serious office work in floating 2D windows. When designing apps or games for this iteration of Apple Vision, I would focus for now on shorter <30 minute experiences. Still, it feels like the start of something very promising to grow over the next few years!
Augments and Meta Spark Studio
Although no official confirmation has been given by Meta yet, pretty conclusive evidence is emerging that Meta Spark will be the technical foundation for the Augments feature Meta previewed for Quest at Connect 2023 in October.
UploadVR got its hands on two accidentally leaked demo videos, one of which showcases the developer setup in Spark Studio. The other focuses on the end user experience. Definitely worth having a look at both videos in the article.
David Heaney
I think it makes sense for Spark to be the basis for this feature, for two reasons:
- Augments have to be pretty lightweight for several to run in parallel in the same space at the same time. Building this on top of Unity or Unreal would probably quickly result in apps that are too resource intensive. Spark experiences have strict limits on them, because they have to load really quickly in Instagram, etc, so the whole toolset is tailored around that performance limitation, making it a natural fit.
- It is in Meta's interest to reduce its reliance on Unity and Unreal, and instead channel developers to a developer platform it owns and controls. Unity for instance has been going through significant upheaval in recent months, so Meta growing its own alternative would make sense from a strategic point of view.
I suppose another direction that they could have chosen might have been WebAR tech, using three.js, A-Frame, etc, which would have been a nice push for that WebAR tech stack. However, I think Meta Spark Studio is a bit more friendly to people with less development experience though, so it makes sense Meta would prefer that approach.
Until Meta publicly launches Augments, I suspect your best bet to get early access would be getting access to the Meta Spark Partner Network.
In other Meta news:
- Meta released v176 of Meta Spark Studio. See changelog and Community Roundup.
- The Meta Quest 2 has been reduced in price permanently to $250. Good in the sense that it should grow the market for games and apps faster, bad in the sense that it means having to support a significant part of the user base with a pretty old XR2 chipset for a long while to come.
Xreal Air 2 Ultra
As part of the CES madness at the beginning of January, Xreal announced a new set of smart glasses called the Xreal Air 2 Ultra. In contrast to many other devices use the AR smart glasses terminology, these appear to be the real thing, with the closest comparison being to Magic Leap 2 and HoloLens 2, although with a slimmer form factor. I see many reporters struggle with how to classify the device, some putting in the same bucket as Apple Vision Pro (incorrect), others with VR headsets (incorrect) and yet others with Meta's Ray Ban smart glasses (also incorrect).
Reaching this form factor does however entail various major compromises, such as a very small 52 inch diagonal field of view and being tethered to a smartphone (or yet to be announced compute puck similar to Magic Leap 2). Recognizing the limits this will place on consumer adoption, Xreal is specifically marketing these as a developer device, intended to facilitate software development in preparation for future iterations, priced at $700.
Cool toy, but why would I buy the device as a developer when there is no installed base for it, and none expected in the foreseeable future? Better to wait until a future iteration which is good enough for Xreal to actually put on sale to the general market. Unless of course as a hobby playground!
Software updates
- TikTok Effect House was updated to v3.7.0. Highlight include cloth effects, depth estimation and various fonts.
- Unity announced improvements to its glTF 3d file format support. Good to see this standard becoming more commonly used! Interesting to see how glTF and USD (which Unity also improved its support for recently) will each grow over the coming years.
- Meanwhile let's hope none of the XR initiatives at Unity are hurt by the recent massive layoffs there.
- Niantic released v3.2 of the Lightship ARDK. Check out the highlights on their YouTube channel, including enhancements to world meshing and performance improvements on iOS. The full release notes are in the developer documentation.
- Magic Leap released v1.5.0 of its SDK. Major improvement in the Unity SDK is support for OpenXR, making developing cross platform software for the headset easier. Its Unreal SDK has been promoted to general availability. Full release notes are available in the developer documentation.
- For those interested in developing for Magic Leap, Dilmer Valecillos recent posted a video about developing with the hand tracking features on Magic Leap 2.
- 8th Wall released v24.1 of its tooling, now supporting A Frame v1.5.0 (which was released in November). With A Frame being on of the primary frameworks on which 8th Wall experiences are built, it's good to see them keeping up to date with it.
- Snapdragon Spaces posted a blog highlighting the support for QR code tracking it added to its recent v0.19 release.
Paula Monteiro
Other XR tidbits
- ShapesXR is a really interesting design tool which I think is invaluable for designing headset-based XR experiences. Really worth adding to your toolbelt, if you haven't yet. They released a new series of articles about designing for XR. Not convinced? Have a look at this recent video by Dilmer Valecillos showcasing how you can use it.
Albert MauriExperience & Product Designer at ShapesXR
- With generative AI being at peak hype, I found this interview by Gabriele Romagnoli with Meshy's Ethan Hu to be very insightful. It's very realistic about where generative AI for 3D workflows are now, the limitations it currently still faces and the long term potential. Definitely worth watching.
- In that same neck of the woods, Luma AI announced a significant new funding round and the promotion of its Genie model for generative 3D to v1.0. It's good to play around with it a bit, both to see the potential and the obvious limitations. Impressive and sobering at the same time.
Luma AI
- Jason McDowall, host of the always interesting 'The AR Show' podcast, had an in depth interview with Michael Hoffman (of HoloLens and MRTK fame). My recommendation: subscribe and support Jason through Patreon, because his interviews are always top notch.
Jason McDowall
- Two interesting articles about the trials and tribulations of VR development: one about Quality Assurance / testing of VR games/apps, the other about developing for PlayStation VR2. And an interesting opinion piece predicting hand tracking will become the dominant interaction paradigm in headset-based XR.
Upcoming XR events
- January 25-29, 2024 - MIT Reality Hack. One of the most prominent XR hackathons of the year, with an impressive list of sponsors, is going on this weekend, as this newsletter is published. Should be interesting to see the results that come out of it.
- January 30, 2024 - The team at Niantic's 8th Wall is looking back at 2023 and ahead to 2024 in an online event.
- February/March, 2024 - Exact date TBD - VisionDevCamp, an Apple Vision Pro / visionOS hackathon planned to take place in the weeks after the device launch.
- March 18-21, 2024 - NVIDIA GTC, primarily an AI conference, but there are a handful of potentially interesting XR talks in the schedule.
- April 10-12, 2024 - Laval Virtual, a long-running XR conference in Laval, France. Early bird rates end on January 31st.
- June 18-20, 2024 - AWE USA - The mother of all XR conferences. If you've got an interesting topic, the deadline to apply as a speaker is February 28th. Super early bird rates end on March 14th.
A bit about this newsletter
My plan is to put together an overview of XR developer news like this every month, really because I couldn't find anything like it, and I suspect it could be a useful resource.
I'm hosting this on ghost.io as a free newsletter. If it's a valuable service to you as a reader though, and you'd like to support me, you can subscribe as a Supporter at €5 per month and I would really appreciate it!
Tiny bit about me: Pieter Siekerman, specialized in XR since 2016, before that mobile apps and TV apps since iPhone & Android appeared in 2007/2008. I greatly enjoy working with front-end technology and am truly excited about XR as the next frontier in that space. So much cool XR stuff still to come these next few years!
Feel free to reach out to me on LinkedIn, for instance if I missed anything which definitely should be in this monthly round up next time!
