Here are all of the AI features coming to the Samsung Galaxy S24

With the growing popularity of AI tools like ChatGPT, it was only a matter of time before tech giants would begin incorporating the tech into smartphones. Google kicked off the trend late last year, adding copious AI photography features to the Pixel 8 series. Now, only a few months later, Samsung has announced a new suite of AI features alongside the Galaxy S24 trio of smartphones. Dubbed Galaxy AI, it replicates some of the features we’ve seen on rival smartphones while also introducing a host of new and exciting ones.

So what does Galaxy AI bring to the S24 series and will any of these new features trickle down to existing Samsung smartphones? Let’s break it down.

When Qualcomm announced the Snapdragon 8 Gen 3 in late 2023, it showcased the chip running generative AI models completely on-device. In particular, we saw it run text-to-image models like Stable Diffusion, an object eraser feature for videos, and the ability to expand images beyond existing borders.

Fast forward a few months and Samsung has collaborated with Qualcomm to bring those capabilities to its smartphones under the Galaxy AI umbrella. The South Korean giant also worked with Google on certain AI features coming to the S24 series, but more on that later.

Smartphone chipsets have supported on-device AI for a few generations now, thanks to built-in neural processing units or NPUs. However, even the latest SoCs won’t be able to run cutting-edge large language models like the one powering ChatGPT. So for the best results, some of Galaxy AI’s best features will rely on Samsung’s servers in the cloud for processing.

We’ve seen Google employ a similar strategy with its Pixel 8 series, where many features like Magic Editor don’t work without an internet connection. Having said that, Galaxy AI will allow you to choose between cloud-based and on-device models. You’ll get lower-quality results from the latter, and some advanced features might not be available at all, but it’s a small win for the privacy-conscious among us.

Galaxy AI features: What’s new?

Samsung has crammed several features under the Galaxy AI umbrella, so we’ve grouped them into two distinct categories. We’ll start with photography, which is arguably the centerpiece of the S24 series and Galaxy AI’s raw potential.

From the outset, Samsung says it turned to AI to improve the image quality of the Galaxy S24 series’ cameras. Dubbed ProVisual Engine, the software suite incorporates AI during the capture process and also while editing shots after the fact.

Thanks to Galaxy AI, the S24 series produces better shots in low-light conditions. Digital zoom quality has improved, too, and the S24 Ultra gets an additional boost thanks to its new 50MP sensor with 5x optical zoom. According to Samsung, the phone will rely exclusively on hardware to deliver shots at 3x and 5x zoom. However, 2x and 10x zoom shots will combine the phone’s optics with Galaxy AI-driven enhancements for “optical-equivalent image quality”. We’ll test these claims in the real world, though, so stay tuned for our full review.

Galaxy AI plays an equally prominent role after you’ve captured your shot. Like Google’s Magic Editor on the Pixel 8 series, activating Samsung’s AI editing tool via the button pictured below will let you erase and move objects within photos. This Generative Edit feature will seem familiar if you’ve ever used an AI image generator like Midjourney. It can also fill in or expand an image beyond its original size, a practice commonly referred to as “outpainting” in AI circles.

Finally, Instant Slow-mo will let you transform a video shot in standard mode into a slow-motion clip. The feature will use AI to generate additional frames as slow-mo typically requires you to shoot at 120, 240, 480, or even 960 fps. This technique of adding new frames is already known as motion interpolation and is commonly used in televisions to make motion look more fluid. The non-AI approach is known to exhibit visual artifacts, so Galaxy AI has some heavy lifting to do here.

Samsung says the Gallery app will automatically add a watermark to any images modified using Generative Edit. This will signal to others that an AI edit was performed on the image.

Language, voice, and more

Galaxy AI includes a range of language-based features that work across both written text and voice. Here’s a quick rundown:

• Live Translate: Samsung’s built-in Dialer app is now capable of translating phone calls in real-time. The feature uses on-device AI to generate voice and text translations of live phone calls both ways. In fact, the person you’re calling doesn’t even need a Samsung phone for Live Translate to work. However, you’ll need an internet connection to use this feature.
• Interpreter: Similar to the Interpreter mode on Google’s Pixel Fold, this feature can transcribe and translate conversations in a split-screen view. And since it relies on an offline language model, it doesn’t require a network connection.

• Chat Assist: The Samsung Keyboard app is getting a number of AI features, ranging from translation to changing the tone of your messages. The latter will rewrite or edit your messages to suit different conversations. You’ll be able to choose from options like professional, casual, polite, and witty.
• Summarization and Note Assist: Galaxy AI will offer an option to summarize documents and websites as well as notes in the Samsung Notes app. The latter is part of a broader AI feature called Note Assist that will also offer pre-made templates and auto-generated covers.

• Transcript Assist: Samsung’s Voice Recorder app will now use AI to generate transcriptions, summaries, and translations of files and phone calls up to three hours in length. It seems pretty similar to the Pixel-exclusive Google Recorder app.
• Android Auto: This is another feature that will make its way to non-Samsung devices in the near future. An upcoming Android Auto update will allow your phone to summarize incoming messages and suggest relevant actions. For example, the AI can suggest replying to an incoming message with your estimated arrival time.

• Circle to Search: To activate the feature, you’ll need to press and hold the home button (or navigation bar if you’re using gestures) and circle or tap any part of your screen. It will then perform a search related to the content you highlighted and surface relevant results. While Circle to Search launches alongside the Galaxy S24 family, Samsung’s collaboration with Google means that it’s also coming to the Pixel 8 series.

Yes, Galaxy AI will make its way to existing Samsung flagship phones and tablets with the One UI 6.1 update. This includes the Galaxy S23 series and the Tab S9 family as well as foldables like the Galaxy Z Flip 5 and Z Fold 5. The exact timeline for the update can vary depending on your model, region, and even carrier, but you should expect it within the first half of 2024.

We’re a little less clear on which Galaxy AI features will stay exclusive to the S24 series. The company hasn’t kept any software features exclusive to the more expensive Plus or Ultra models, so we’re hopeful that the majority come to last year’s phones too. That said, some on-device features may require the Snapdragon 8 Gen 3 or Exynos 2400 in the latest Samsung flagships.

Still, it’s a positive sign that Samsung has agreed to bring over new AI features to its older phones. We’ve seen Google do the opposite with the Pixel 8 series, after all. It also bodes well for future releases as you can take solace in the fact that you’ll likely get some of next year’s features if you buy a Galaxy S24.