By Jason Snell

June 12, 2024 2:10 PM PT

visionOS 2 offers small, but promising, steps forward

I’m just back from Cupertino, and there’s an awful lot to think about. But before all that, I thought I’d cover what absolutely everyone is talking about: visionOS 2.

More seriously, I installed the visionOS 2 developer beta this morning—this is the entire reason I have a Vision Pro!—and I’ve got a few quick thoughts before plowing on to developer sessions, thoughts about Apple Intelligence, and the rest.

Spatial photos. The most impressive single feature I’ve seen in visionOS 2 so far is the ability to create spatial photos from out of your old, mundane 2D photos. Load pretty much any photo in the Photos app and tap the Spatial icon in the top left corner of the image, and a fun sound effect plays as an animation sweeps across your photo, representing the system’s machine learning-driven software scanning your photo and building an artificial depth map to provide the illusion of depth.

You know, very few movies are shot in stereo anymore. It’s more expensive and cumbersome to shoot, and conversion to 3-D after the fact is good enough. Upon viewing Apple’s spatial-converted photos in visionOS 2, I had similar thoughts: I don’t see why we would ever need to shoot stereo images if machine learning is this good at faking it.

Seriously, whether it was a photo taken last week or 50 years ago, Apple’s algorithm does a staggeringly good job at building a depth map. You don’t need embedded LiDAR or other depth information—the algorithm does it, and it does in incredibly well. Pictures of my kids taken when they were little are suddenly given more depth (literally and figuratively). Pictures of me as a kid, even. A complex shot with a tower made of LEGO bricks in the foreground was scanned and mapped perfectly.

It takes about 15 seconds to spatialize a photo, so it’s unlikely that Apple will ever ask the Vision Pro to churn in the background through every single photo in your library, but I’ve yet to see a photo that failed to become more interesting after being converted.

New gestures. Apple has added some new gestures to visionOS, which key off of you holding out your palm and looking at it. When you do that, a floating icon appears next to your hand indicating that you can tap your finger and thumb together to open the Home View. (There’s also a new Close button at the top of the Home View so you can close it again.) It’s a nice idea because as much as I’ve internalized reaching up and tapping the Digital Crown to bring up the Home View, the act of doing so is also jarring given that all other Vision Pro interactions are gestures happening in the air in front of me.

I’m actually more excited about the second gesture that keys off the first. After you look at your stretched palm (have you ever really looked at your hand?) you can flip your hand over to reveal a floating bubble that displays the time, battery percentage, and volume. Given how frustrating it was to quickly check the time in visionOS, this is a great new feature that I anticipate using a lot.

If you bring your fingers together while looking at the bubble, you can slide right or left to adjust the device volume quickly. And if you tap while looking at the bubble, Control Center opens. This, to me, is a much better way to access Control Center—though it’s certainly less discoverable than the little firefly that frequently hovers at the top of your vision.

Customize Home View. You can now move apps around in the Home View, and it works pretty much how you’d think: while looking at an app, bring your finger and thumb together to enter Jiggle Mode and then pinch the app to drag it around, even across pages. It worked exactly as I expected it to, and I’m happy to finally be able to put some of my favorite apps on page one.

Breakthrough keyboards. Typing on a keyboard in an Environment was previously very weird, because while your hands were visible, your keyboard itself was not. Apple has upgraded this in visionOS 2 so that it recognizes either the Apple Magic Keyboard or the keyboard on a MacBook, and that’s great.

But at least in the first developer beta, I found that it was a bit finicky—it needed the keyboard to be positioned just-so for it to appear, and even then it sometimes felt like the environment was eating away at the edges of the keyboard. I’m not even a hunt-and-peck typist, but being able to actually orient on the keyboard by seeing it is still valuable.

Also, I’m disappointed that Apple has limited this feature to its own keyboards. I realize it would be harder to generate a model that recognizes more generic keyboards, but most keyboards really do have some pretty obvious characteristics in common, don’t they? Alternately, maybe Apple should consider a feature like the one Meta offers, which lets you work in an environment but with a specific cut-out—like a tabletop—set to pass through.

Life’s a beach. Famously, visionOS shipped with two Environments marked as “Coming Soon”—one featuring a blurry beach image and another with fog-shrouded trees. The trees remain a mystery—Twin Peaks environment?—but the beach has arrived in the form of Bora Bora. As someone who loves beaches, it is spectacular. You can hear the sound of the waves, the palm trees gently blow in the breeze, and you can even see the color of the beach change as a thin cloud passes over the sun. I think I am going to be spending a lot of time in Bora Bora (the visionOS 2 environment, alas).

I didn’t get a chance to test out some other features due later this year, including support for a much larger display in Mac Virtual Display mode and multi-view support in the Apple TV app. In a Spatial Persona call with my podcast co-host Myke Hurley, the updated personas looked good (most notably hand gestures) but Myke’s mustache still prevented his persona’s mouth from moving. There’s more work to be done there.

Still, it’s exciting to get some major new features out of visionOS 2, despite the fact that visionOS 1.0 only shipped a few months ago.

By Jason Snell

June 7, 2024 2:35 PM PT

Siri’s biggest gains might be found at the edges

We’re on the verge, maybe, of something big. Apple broke ground with its voice-driven Siri assistant back in 2011, but over the years, Siri’s trajectory has been meandering and frustrating. It’s driven deeply into popular culture—”Hey Siri” is practically a catchphrase—but it’s still painfully inconsistent at best and a failure at worst.

The rise of AI chatbots, which (despite their tendency to hallucinate) manage to project the illusion of intelligent conversation, has really put Siri up against it. This week, the Wall Street Journal reported that Apple had a faster, more conversational, and more accurate Siri waiting in the wings at least three years ago—but decided to go with a more modest upgrade code-named “Siri X.”

If the idea that someone inside Apple felt that Siri didn’t need to be more conversational and accurate makes you angry or confused or both, I’m right there with you. We can only hope that Apple has reconsidered its complacency over the last 18 months, since Apple executives reportedly realized that the rise of AI chatbots was destined to change user expectations.

I hope that the people who lost the arguments for a better Siri back then may have been winning those arguments for the past 18 months. And it’s made me optimistic—here in the shadow of WWDC, where everything’s possible, and nothing’s decided—that Apple might really be on the road to giving Siri the biggest upgrade it’s ever received.

That’s a big deal—and not just for the obvious reason, which is that I’d like to use Siri for something other than a tea timer or to add something to my shopping list. That’s because with a good, reliable, conversational Siri, a lot of Apple’s products can benefit.

When I’m running or walking my dog, I generally use just AirPods and my Apple Watch. Theoretically, I’m covered by Siri, but I generally avoid talking to it because it’s unreliable. I’ve also turned off all of the features that summarize messages I’m receiving because I find them awkward and unhelpful.

But imagine what a good Siri experience could unlock for users of AirPods, Apple Watch, or both. Humane built an entire product—the Ai Pin—out of a voice assistant that you carry around with you. But with a better Siri interface, Apple’s already most of the way there. The better the voice interface, the more useful your device can be when you need to interact with nothing but your voice.

Similarly, an upgraded Siri could be a huge step toward improving the CarPlay experience. As is the case with most contexts in which Siri is available, I’ve tried Siri in the car and learned my lesson—namely, to never use it again. When I’ve got to keep my hands on the wheel and my eyes on the road, being able to have a proper voice assistant that can summarize what’s going on, understand context, and follow up when I ask for it could be incredibly valuable. But Siri being unreliable has degraded that experience.

If there’s one place where the Humane Ai pin and those Meta Ray-Ban sunglasses have an edge over Apple’s stuff, it’s that they both offer a forward-facing camera to interpret the world around you. With an upgraded and competent Siri, it might make more sense for Apple to sell a pair of glasses or a clip-on camera pin as another accessory to help improve the data being input into Apple’s AI systems.

And then there’s the kitchen, where a device that combines a HomePod with a screen and camera might end up being quite welcome—I’ve used Google and Amazon devices in my kitchen for years—but it really needs to be rock-solid when it comes to voice commands. The Siri on my current HomePods doesn’t measure up.

Of course there are numerous ways that a better Siri and more AI-focused features can improve Apple’s core platforms. (I’d love a reason to use Siri on my Mac, for example, but I’ve never really found one.) But it strikes me that Apple’s biggest opportunity might actually be at the edges of our lives, where traditional interfaces are harder or even impossible to use. A useful Siri instantly makes the Apple Watch, AirPods, CarPlay, and HomePod better—and could facilitate even more interesting products to come.

But first thing’s first: Siri needs to get vastly better. I hope that we get the first sign of that on Monday.