- Google aims to give new chatbot access to users’ photos and internet search history
- The project, dubbed “Project Ellmann,” will be built on top of Gemini, Google’s “multimodal” ChatGPT rival, which can understand not only text but also images, video, and audio.
- Read more: AI-designed lollipop-flavored cola and digitally created beef stew
A confidential document released at a recent internal Google summit details the tech giant’s plans to create artificial intelligence (AI) designed to be a “life story teller” for its users.
But to make that happen, AI will require unprecedented access to each user’s personal data.
It’s unclear where this experimental AI, currently called “Project Ellmann,” will be placed in Google’s apps and services, but the team behind it works in Google Photos, and their presentation will include a customized AI chatbots were suggested.
Part of a presentation created by a Google product manager says, “If you can’t see the big picture of your life, you can’t answer the tough questions or tell the good stories.”
Based on the company’s ChatGPT rival Gemini, Project Ellmann uses “Large-Scale Language Models” (LLMs) to include biographies of users and their loved ones, as well as “moments” of stored photos. We plan to synthesize personal information from the context in which it is said.
But this new development comes as people outraged by Google’s secret collection of millions of individuals’ confidential medical records, codenamed “Project Nightingale” in 2019, and tips on digital privacy. This may cause alarm among those who avidly collect.
“We comb through your photos and look at their tags and locations to identify meaningful moments,” according to another presentation slide obtained by . CNBC.
“When we step back and understand your entire life, your entire story is revealed,” the slide continued.
In other words, this project wants to create a ChatGPT-style chatbot that is customized to your interests and life history, inferred from your internet search history, your phone’s camera roll, and other data. This always includes things like purchases with Google Wallet.
The Google Photos team will ingest search results, user photos, and other data to discover patterns in order to “answer previously impossible questions” about a person’s life, according to the project manager’s presentation. I spent several months verifying the suitability of the.
The team’s planned ambition to build an intimate and detailed portrait of its users is embedded in the project’s name. It is named after literary critic and biographer Richard Ellman, who won the National Book Award for his biography of novelist James Joyce.
Google’s team hopes that Ellmann will eventually be able to describe users’ personal photos in more detail than “just pixels with labels and metadata,” according to one slide. It is said that there is
In one example, Ellmann LLM scans a user’s photos and batches them into “memories” or “moments” from the birth of that user’s child, for example, or a series of images taken at a high school reunion. We discussed how to handle it. .
As the analysis of the photo explains in the presentation slides, “It’s been exactly 10 years since he graduated. It’s probably a reunion because it’s full of faces I haven’t seen in 10 years.”
As an example of a “previously impossible question” that Project Elman can help users answer, the presentation talks about a hypothetical user who wants to know when their brother last visited, or which town they should move to. A request has been presented.
Mr. Ellman was able to answer both questions based on his presentation.
Ellman also appears to be able to predict purchases and make recommendations, and also provided an overview of users’ eating habits.
“You seem to like Italian food,” Project Elman LLM noted in one slide. “There are several photos of pasta dishes and one photo of pizza.”
Given that this presentation came from a Google Photos manager, CNBC speculated that the company was planning to incorporate new AI products within the Google Photos app.
According to Google Photos, there are over 1 billion users and 4 trillion photos and videos stored there. Google Cloud blog post.
To more directly demonstrate who Google teams consider their competitors, they asked colleagues, “Imagine opening ChatGPT, but it already knows everything about your life.” ” he asked, summarizing “Ellmann Chat.”
Google feared the worst when competitor OpenAI unveiled its artificial intelligence (AI) chatbot ChatGPT a year ago.
It’s the answer to Gemini, an AI chatbot built to power Google’s chatbot Bard, which Google says outperformed ChatGPT’s GPT-4 on the majority of cognitive tests.
Judging from more common information these days, Gemini can tell users when an omelet is cooked, suggest the best design for a paper airplane, and help soccer players improve their skills. I can. research paper.
But they are particularly good at mathematics and physics, raising hopes that they may lead to scientific advances that improve human life.
Google claims Gemini outperforms GPT-4 in 30 out of 32 performance measures, including text generation, question answering, inference, image understanding, and “common sense reasoning.”
In its Gemini research paper, Google outlined various image-related AI capabilities, including what to knit from different colored threads.
Gemini only works in English for now, but the company says the technology could be easily diversified into other languages.
In addition to Project Ellmann and Bard, Google also plans to introduce Gemini to its flagship search engine.
The company appeared disappointed by the leak of the presentation to CNBC, and issued a statement through a spokesperson explaining the privacy issues.
“Google Photos has always used AI to help people find photos and videos, and we’re excited about the potential for LLM to unlock even more helpful experiences,” a spokesperson said. Ta.
“This was an initial internal investigation,” the spokesperson stressed.
“When we decide to roll out new features, we will take the time necessary to ensure they are useful to people and designed to protect the privacy and safety of our users as a top priority.”