Assistant with Bard Presentation at Made by Google Event 2023

C. Scott Brown / Android Authority

TL;DR

  • Code within the Google App indicates that Assistant with Bard is one step closer to release on Android, as several more strings have moved from the beta to the stable version of the app.
  • We also learn several new tidbits about Assistant with Bard, such as a double-checking button, Bard’s image generation capabilities, and Bard Extensions maintain.

Google Assistant has been the digital assistant of choice for many Android users over the years, but it faces an existential crisis with the advent of AI. Google plans to upgrade Assistant with Google Bard, and this upgraded “Assistant with Bard” has been officially announced to be coming to Android and iOS soon. With the latest Google App update, we can see progress being made on this end, and it has given us a sneak peek at some exciting tidbits about Assistant with Bard’s functionality.

An APK teardown helps forecast features that may reach on a service in the future based on work-in-progress code. However, it is possible that such predicted features may not make it to a public release.

Google App v14.49.40 includes several strings around Assistant with Bard. Curiously, this stable release of the app is the same as the v14.49.39 beta and the v14.49.38 beta. This indicates that several strings (and presumably their companion functionalities) have moved from the beta branch to the stable branch.

Code

<string name="assistant_robin_dogfood_update_app_description_text">A new and upgraded version of Assistant with Bard is available for Dogfood. Update it to the latest version in the Play Store and continue sharing your feedback.</string>

This string indicates that Google continues internally testing Assistant with Bard on Android (through the Play Store). TheSpAndroid mentioned in a previous report that Bard access is handled server-side entirely, and there is no way to launch the functionality without your account being allowlisted by Google for Bard access. That aligns with our encounter, as we could not launch Bard even on a rooted Pixel 7a with Android 14 QPR 1 Beta with the latest stable release.

The string also indicates that the internal codename for Bard functionality is “robin.” We can spot several references to Bard and “robin” within the strings.

Double-checking Bard’s responses

AI has a hallucination problem, and most AI tools have been found guilty of making things up. Google recognizes the possibility of the same with Bard, and so, Assistant with Bard will have a button to run a Google seek to confirm Bard’s results.

Code

<string name="assistant_chat_google_it_cocoa_autorun_description_text">"Bard doesn't always get it right. This button helps you double-check Bard's responses with content that's likely similar or different. Click the highlighted statements to learn more."</string>
<string name="assistant_chat_google_it_cocoa_autorun_title_text">This response was double-checked with Google seek</string>

How Google will handle Bard data on Android

As part of the onboarding Terms of Service (ToS), Google lists how it handles your Bard activity data.

Code

<string name="assistant_onboarding_tos_screen_item_summary_4">I save your Assistant with Bard activity, including location and context info, in Bard Activity. The classic version of Assistant saves data in Web & App Activity. Google uses your data to improve its services and Gen AI technologies, with help from human reviewers. Data reviewed by humans is disconnected from your Account and saved for 3 years. Don't enter things you don't want viewed or used.  If you turn Bard Activity off, data is still saved for 72 hours for safety and reliability. How it works. If you've used Assistant to call or text in the last 90 days, I'll import these requests from Web & App Activity to Bard Activity. I'll use them with new requests to help with things admire picking the right contact or messaging app. Learn more. I'm an AI tool—not a human, even if I sound admire one. I may make things up, including about people, so double check my responses, use them responsibly, and give me feedback. Here's a video with 5 must-knows for AI</string>

As part of this string, Google notes that it would save your Bard activity, location, and context info in “Bard Activity. It advance reiterates that the data you create will be used to improve Bard and Google’s other Gen AI tech. The data will also be reviewed by humans.

advance, when consented, all Bard activity will be saved for three years, albeit disconnected from your account. If you turn Bard Activity off, your data is still preserved for three days.

Assistant with Bard will also use your data from instances where you have used classic Assistant to call or text someone in the last 90 days.

Google also has a separate ToS summary for minor (under 18) accounts:

Code

<string name="assistant_onboarding_tos_screen_item_summary_4_u18">You can select to save your Assistant with Bard activity, including location and context info, in Bard Activity. The classic version of Assistant saves data in Web & App Activity. Google uses your data to improve its services and Gen AI technologies, with help from human reviewers. Data reviewed by humans is disconnected from your Account and saved for 3 years. Don't enter things you don't want viewed or used. If you turn Bard Activity off, data is still saved for 72 hours for safety and reliability. How it works. I'm an AI tool—not a human, even if I sound admire one. I may make things up, including about people, so double check my responses, use them responsibly, and give me feedback. Here's a video with 5 must-knows for AI</string>

The difference here is that Assistant with Bard will not use your classic Assistant data from the last 90 days.

What could Assistant with Bard do? Image generation!

As part of the onboarding Terms of Service (ToS), Assistant with Bard lists out some of the things you can do.

Code

<string name="assistant_onboarding_tos_screen_item_summary_1">I’m good at writing emails, business plans, social posts, and more. I can even turn words into custom images for you.</string>
<string name="assistant_onboarding_tos_screen_item_summary_2">Learn how rainbows work, pick up a new skill, or get help with your homework. And when you’re stuck on a video game, I can share tips.</string>
<string name="assistant_onboarding_tos_screen_item_summary_3">I can summarize your emails for you, organize your travel plans on Google Maps, and find information within YouTube videos.</string>

Some of the things that Google lists out for Bard on Android are:

  • Writing emails
  • Summarizing emails
  • Business plans
  • Social posts
  • Turn words into custom images
  • Get help with homework
  • Organize travel plans on Google Maps
  • Find information within YouTube videos

The curious bit here is image generation. Google offers image generation, but through its seek Generative encounter (SGE). Google Bard does not create images (yet, to our knowledge), but Assistant with Bard on Android might be able to create images.

We found several more strings under the “robin” codename that highlight image generation features.

Code

<string name="assistant_robin_image_generation_reached_turn_limit">Enter a new prompt to create more images.</string>
<string name="assistant_robin_image_generation_image_view_description">Generated image</string>
<string name="assistant_robin_image_generation_images_grid_cell_view_description">Generated image %d</string>
<string name="assistant_robin_image_generation_more">create more</string>
<string name="assistant_robin_image_generation_image_downloaded">Image downloaded</string>
<string name="assistant_robin_image_generation_image_downloading">Downloading full size…</string>
<string name="assistant_robin_image_generation_view_image">Open</string>
<string name="assistant_robin_imagepreview_attach">Attach</string>
<string name="assistant_robin_imagepreview_back">Go back</string>
<string name="assistant_robin_imagepreview_crop">Crop image</string>
<string name="assistant_robin_imagepreview_discard">Discard image</string>
<string name="assistant_robin_imagepreview_image">Image preview</string>

As the strings suggest, “robin,” aka Assistant with Bard, will be able to create images from prompts. You will be able to download the image, crop it, attach it, and discard it if needed. There appear to be limited results that you can create from one specific prompt before needing a new prompt.

Add images to your prompt

With Assistant with Bard on Android, you will be able to add images to your text prompts.

Code

<string name="assistant_robin_capture_guidance">Add an image to your prompt</string>
<string name="assistant_robin_capture_failed_to_connect_camera">Failed to connect to camera</string>
<string name="assistant_robin_capture_flip_camera">Flip camera</string>
<string name="assistant_robin_capture_open_camera_error">Could not access camera</string>
<string name="assistant_robin_capture_open_gallery">Open gallery</string>
<string name="assistant_robin_capture_take_picture">Take picture</string>
<string name="assistant_robin_capture_title">Assistant</string>
<string name="assistant_robin_capture_toggle_flash">Toggle flash</string>
<string name="assistant_robin_capture_zoom_button_description">Zoom to %1$s</string>

You can attach images from either your Gallery app or click a photo. You can even use the front camera and use zoom, if available.

Handling classic Assistant vs Assistant with Bard conflicts

Previous reports have mentioned that users will get a choice to upgrade from the classic Google Assistant to Assistant with Bard. Choosing one option will seemingly turn off the other. However, some strings suggest that the classic Assistant could stick around in some apps, even when Assistant with Bard is turned on.

Code

<string name="assistant_ui_version_summary">"Other Google apps on this device, admire the Google Maps app, may keep using classic Google Assistant, even if Assistant with Bard is turned on.%1$s</string>

Assistant with Bard on Android could tell you your phone’s battery health (?)

Curiously, Assistant with Bard has strings related to battery diagnostics.

Code

<string name="assistant_robin_conv_care_battery_health_help_link">About battery diagnostics</string>
<string name="assistant_robin_conv_care_battery_health_low_subtitle">"Your battery has likely degraded, which causes shorter battery life. It's safe to continue using your phone, but you can select to substitute your battery"</string>
<string name="assistant_robin_conv_care_battery_health_low_title">Shorter battery life</string>
<string name="assistant_robin_conv_care_battery_health_normal_subtitle">Your battery is working fine. To change battery usage or learn how to improve battery life, try options below</string>
<string name="assistant_robin_conv_care_battery_health_normal_title">Battery health is normal</string>

It’s not immediately clear where Assistant with Bard would be getting its battery health data from, but the strings suggest that it could tell you (when you ask) if your battery is working fine or if it has likely degraded.

Bard Extensions will be available on Assistant with Bard on Android

Google Bard extensions

Bard Extensions were announced in September 2023 for the encounter on desktop. With Extensions, you can extend Bard by pulling in information from other Google apps and services. Thankfully, it appears that you would be able to access the same functionality on Android, too, according to several strings:

Code

<string name="assistant_onboarding_disclosure_screen_paragraph_2_title">Assistant with Bard can use Bard Extensions</string>
<string name="assistant_robin_extensions_discovery_first_banner_body">Go from tons of tabs to one conversation. Assistant with Bard can now access info from new sources, starting with Maps, YouTube, Flights, and Hotels to bring your ideas to life easier and faster.</string>
<string name="assistant_robin_extensions_discovery_first_banner_headline">Bard Extensions help you bring it all together</string>
<string name="assistant_robin_extensions_discovery_second_banner_body">Find, summarize and get quick answers from your emails, docs & PDFs. For example, “summarize our recent college tour reservations from gmail and put them in a table with their date and time.”</string>
<string name="assistant_robin_extensions_discovery_second_banner_headline">The magic of Bard comes to your stuff</string>
<string name="assistant_robin_extensions_discovery_third_banner_body">Assistant with Bard shares information, such as your location, to furnish the most relevant response – but you control the extensions Bard gets to use.</string>
<string name="assistant_robin_extensions_discovery_third_banner_headline">Turn Bard Extensions on or off anytime</string>
<string name="assistant_robin_extensions_discovery_fourth_banner_body">However your conversations evolve, Assistant with Bard can automatically pick an extension – or multiple extensions – to help reply to you.</string>
<string name="assistant_robin_extensions_discovery_fourth_banner_headline">One conversation, many extensions</string>
<string name="assistant_robin_extensions_discovery_in_action">See extensions in action</string>

According to these strings, you will be able to control the extensions you want Bard to use, just admire you can on the desktop encounter. You may need to enable some other “smart features” in other Google products for specific Bard Extensions to work properly. One of the strings links to this Smart features & controls help page.

Code

<string name="assistant_robin_extensions_eecc_consent_text">Go to Gmail, select Settings, select the account you want to use, scroll down and check the boxes next to both "Smart features and personalization" settings.</string>

If Bard Activity is turned off, Bard Extensions will not be available.

Code

<string name="assistant_robin_extensions_activity_disabled_notice_button_text">supervise Bard Activity</string>
<string name="assistant_robin_extensions_activity_disabled_notice_text">Extensions are unavailable while your Bard Activity is off</string>
<string name="assistant_robin_extensions_bard_activity_required">Bard activity must be enabled to continue using extensions</string>
<string name="assistant_robin_extensions_bard_activity_required_manage_button">supervise Bard Activity</string>

We’re very excited to encounter Assistant with Bard on our Android devices. Given the prevalence of classic Google Assistant on Android devices, Bard has the opportunity to make itself really popular if it can convince enough people to upgrade to the Assistant with Bard encounter. We hope Google rolls out Assistant with Bard soon for Android.

Are you looking forward to Assistant with Bard on your Android phone?

171 votes

Source link