What is Galaxy AI? Samsung’s generative AI plans explained

No matter where in commercial technology you turn, you’ll probably find mentions of large language models and generative AI tools splattered all over the place. From ChatGPT to Google Bard, every big tech company wants to get a piece of the generative AI pie.

Samsung has started on a potentially radical entrée of its own, the consequences of which will affect millions of Galaxy device buyers worldwide soon enough. We explain what Galaxy AI is, where you’ll find it on the Galaxy S24 lineup, what it might become, and what the way forward might be.

What is Galaxy AI and where does it come from?

Samsung first teased Galaxy AI in November 2023 as a user-facing set of on-device and cloud-based services that will improve how you use your phone or tablet. It officially came into life with the launch of the Galaxy S24 series of Android phones the following January. In short, Galaxy AI leverages your phone’s processing power instead of massive central servers to deliver some of the same features as the web-based generative AI services you’ve come to know, but without the need to be connected to the internet.

— Wonjoon Choi, EVP and Head of R&D, Mobile eXperience Business.

The company’s initiative actually follows upon Google’s own dip of its toes with a couple of on-device AI features it worked into its Pixel 8 Pro. Android is effectively being reworked to allow every device on Android 14 and above to tap into a tiny version of its own model (Gemini Nano) when an app calls for it.

In the case of Galaxy AI, however, the Korean tech giant is using its own large language model and is being more ambitious about where it plugs into — at least in the beginning. The goal here is to harness whatever power is on-site and reduce the user-side cost of requesting AI transactions by running applications locally.

The company has developed three sub-models or tools with Gauss:

• Gauss Code is in active use among the company’s developers. Along with a derivative assistant tool named code.i, Gauss Code is used to generate test cases and annotate code for in-house software.
• Gauss Language is specifically tuned for a ChatGPT or Google Bard-type application that can summarize or translate existing texts and generate new ones for replying to messages.
• Gauss Image would behave something like the DALL-E, Midjourney, or Adobe Firefly engines with the ability to generate and edit images based on user-given prompts.

What features can you use with Samsung Galaxy AI?

With the Galaxy S24 series, no matter whether it runs on Samsung’s Exynos 2400 or Qualcomm’s Snapdragon 8 Gen 3 chipset, you’ll be able to try out the following features from right within certain application experiences on your device — some of which can run without an internet connection. Here are the features you’ll be seeing:

• Live Translate offers real-time two-way translation during conversations using the on-board phone app or the Interpreter interface if you’re chatting in-person.
• Chat Assist is the text version of Live Translate and is designed to help users converse in up to 13 languages, working in many communication and social media apps. You’ll be able to adjust the level of formality and fix your grammar and spelling with this feature.
• Note Assist is definitely more generative in nature, helping Samsung Notes app users summarize documents and create templates.
• Android Auto has been tuned to summarize incoming messages. You’ll be able to issue simple, short voice commands to send information-rich replies such as ETA.
• Transcript Assist generates an instant transcript for any voice memo — complete with speaker tags — and comes with a translator interface as well.
• Circle to Search is being touted as an industry first. By long-pressing the home button, users can then circle, highlight, or tap on photos or text (including words baked into photos and videos) they see and instant feed that into a Google search query, right within the same view.
• Edit Suggestion gets into some of the camera features of Galaxy AI and does exactly what the name suggests.
• Generative Edit can be used on existing photos to fill in parts that you’d like to replace or generate new material to expand the view.
• Instant Slow-mo generates extra frames when you want to create a slow motion moment in a video that you didn’t shoot in slow motion.

Of course, as generative AI features have the potential to gin up controversy, there are some caveats to be had:

• Certain features require that you log into a Samsung account to use.
• The accuracy of translation features is not guaranteed.
• When Galaxy AI is used to edit photos and video, the media will be watermarked, and its metadata will be tagged.

What’s the big deal with Samsung Galaxy AI?

There has seemingly always been a back and forth fight between major tech competitors over one trend or another. This time around, it looks like everyone’s going all out on generative AI features. Apple seems to be the other elephant in the room that is slow-rolling its own AI hype. Execution matters in determining the success or failure of a particular approach. We can recall the digital assistant craze last decade when Samsung decided to put up Bixby against the likes of Google Assistant, Siri, and even Amazon’s Alexa.

While there’s been some continued use in the company’s insular home market of South Korea, it has been a non-factor elsewhere. Bixby’s knowledge and intuition weren’t that great in Western markets to begin with. The fact that Samsung dedicated a hardware button on the Galaxy S8 and S9 to calling Bixby (which initially could not be reprogrammed to do something else) also did not ingratiate the assistant to users.

Samsung and its competitors seem keen on diversifying the surfaces where they apply AI features to avoid bundling everything into a central client that can either be easily ignored or pervasively present. That might benefit their generative AI pushes in the long run while keeping the focus on the core services they’re enhancing.