Digging deeper into the intelligence behind Google Photos

Google Photos AI

How do you find the Google Photos app in you phone? If you are less photo savvy and have less collections of photos, probably you find your simple gallery app more useful. But what do you do in the simple gallery app? You simply open the app, choose from the latest photos by date or from the standard groups like camera, screenshot, whatsapp photos, downloads...

But suppose you are 40 years old and you have attended atleast 10 marriage ceremonies of your relatives, 10 foreign tours and many other sweet memories. And now if you are a photo savvy person, you most probably will have collections of photos in thousands. Now try finding all your photos with your friend say Leeza.

Difficult right? Actually very difficult. Lets say you had a separate folder for Leeza. But what about your other friends in general. You simply cannot create seperate folders for all of them.

And now your simple gallery app cannot help you with this. Because it cannot understand photos or recognize faces. In other words, if you have a large collection of photos you need something more than storing and viewing.

And this is where Google Photos can help you.

Google photos can understand photos. It can recognize every face in your collections. And you ask how? Well, thats a whole different topic on Neural networks and deeplearning. And the purpose of this article is not to discuss that. But if you ask how well Google photos does that, we can discuss it here.

How well Google photos recognize your face?

The great minds working in this field have set human ability and intelligence as the standard for Machines. Human Intelligence is extremely complex. A person saw a man once years ago and say things like "Maine use kahin dekha he" or "I think I saw him somewhere".

What I mean to say is a human may not need lot of samples to recognize another human by face. But with machines we need thousands of samples to recognize a person's face with human level accuracy. As William Koehrsen has mentioned in this post, quote:

Inception V3 achieved such impressive results,rivaling or besting those of humans,by using a very deep architecture, incorporating inception modules, and training on 1.2 million images. However, this model is limited to identifying only the 1000 different images on which it was trained. If we want to classify different objects or perform slightly different image-related tasks (such as facial verification), then we will need to train the parameters,connection weights and biases,of at least one layer of the network.

But this does not mean that the system will take lot of time to detect your face. Once the system has recognized your face after training on a sufficient amount of data, it can detect your face from any frame as fast as you do, may be even faster.

Leaving research aside, you can test it yourself and see how well Google Photos organize your collections . Open the Photos app, hit the search bar, and see if the app shows all the unique faces in your collection. A sample is as shown below:

Google Photos Organise photos automatically

The faces you see are kind of folders which contain all the photos containing that face. You can access that folder by clicking on a face. A sample is shown below:

Google Photos grouping photos by face

You can also do a simple name search. The interesting thing here is that chances are the system already knows the names of the faces in your collections. How? Well, the system might have known them from their own Google accounts or, if they are famous, the system knows them from the internet.

My own assessment is that the Photos app works phenomenally well. It can detect the target face from any orientation. It can detect you even if you have clicked yourself from a darker space.

How will face recognition impact our lives

Face recognition may seem like a normal thing to a layman. But for the big companies in the world, it is a ground breaking thing. Joaquin Quiñonero Candela has explained in this post how face recognition helps facebook quote:

these new features help you find photos that you’re not tagged in and help you detect when others might be attempting to use your image as their profile picture. We’re also introducing a way for people who are visually impaired to know more about who is in the photos they encounter on Facebook...

If you’re in a photo and are part of the audience for that post, we’ll notify you, even if you haven’t been tagged. You’re in control of your image on Facebook and can make choices such as whether to tag yourself, leave yourself untagged, or reach out to the person who posted the photo if you have concerns about it.

And you probably also know that using Google Lens, which is now part of Google Photos, you can extract information about an unknown object. And the scary thing is that the unknown object could be a human being. Scary because an unknown person standing next to you might extract information about you without interacting with you or without your knowledge and consent. This might be a huge boost to security agencies and national security and for the government.

But what about your privacy? We will discuss this important issue later in this article. In the meantime, let's move on to explore some more amazing things about Google Photos.

Search Photos smarter and faster

One of the best part of Google Photos is that you can search photos from your collection with any keywords. For example, suppose, you had visited a place two years back and now you want to see those photos. With the Photos app, you don't need to do a long scrolling and browsing. All you have to do is hit the search bar and type, say, visit to Los Angeles.

And the Photos app will show you all the photos at Los Angeles. A sample showing how this works is as shown below:

1 / 3
Smile Photos
2 / 3
Family Weekend
3 / 3
Dog at the beach

This feature is same as normal Google search but within the space of your collection.

Smart and efficient organisation of your photos

The Photos system can organise your photos in any fashion. The amazing thing about this is that this does not require extra space. Common groups you will normally see are People, Places, Things, Family, Wedding and so on.

A sample is as shown below:

Google Photos Smart organisation

Another very interesting thing about Google Photos is that it can not only group your photos but can also describe it. A sample is as shown below:

Google Photos describing a trip

Latest features and future updates

Google photos app was last updated on June 28, 2018. The features presented at Google I/O 2018 are not available yet. Your photos app will soon have a new feature called suggested action, essentially suggesting smart action right in context for you to act on.

Suppose you are browsing through your wedding photos. Now the photos system knows who all are there in the photos. When you browse through the photos containing your friend say Leeza, the app will suggest you to share all n pics with Leeza, and with one click the photos will be sent to Leeza. A sample is shown below:

Google Photos fast sharing

Suppose in the same wedding, there are some photos that are underexposed. When you browse through those photos, the app will suggest you to fix the brightness, and with one click the brightness will be fixed. A sample is shown below:

Google Photos adjust brightness

Suppose you took a pic of document. When you open the pic, the app will understand that the pic is a document and will ask you to fix the document. And with click you have a document in pdf format, making it much for you to use later. A sample is shown below:

Google Photos save document

Amazing right? And further, sometimes the intelligent will be able to deliver unexpected moments. It might suggest certain color effect which could make the pic cuter. Consider the sample below:

Google Photos pop color

In this sample, the system suggested you to pop the background color and the pic looked just cuter. I am sure, users will have lot of fun playing around.

And finally, suppose you have a very old pic of say your grandmother. If the photo is in black and white, the system might suggest to colorize it and with one click you will have a colored version of the pic. A sample is shown below:

Google Photos colorise photo

All these new features are not simple editing. These are done by the AI system after having trained itself on billions of photos. So lot of fun ahead!!

How secure are you with Google Photos?

I have met with quite a few people who are not comfortable using Google photos. Their primary concern being security. They fear that their private pics could be leaked and misused. Now the question is, is this fear justified?

It is true that unless your Google account is hacked, your photos are private. But can you really call it private when your photos are being used to train their intelligent systems and extract information about you? This is what worries some people.

One may argue that the more Google knows about you, the better capable Google would be to suggest you more relevant ads and information. Your experience with Google Now will be enhanced and it only adds to your advantage.

But this is not the main concern here. The point is that Google uses your information behind your back. When you Signed up for Google accounts, you did accept Google's terms and conditions, meaning you did give Google legal rights to use your information.

So the question really boils down to this: How much do you trust Google?

One irony here though. You use Google's other services like Gmail, Google maps, Google drive, Google Now without much concern. But you become extra concerned with your photos. Why? As per collecting information is concerned, Google photos is nothing new.

Using the free services of Google, you did put your trust in Google. But still Do you trust Google?

You may also like reading this nice article @ on whether you should be worried about your privacy with Google photos.


Ratul Doley
Ratul Doley
Expertised in ReactJs, NodeJS, Modern Js+Css, Php, Java. Professional Android and iOS app developer and designer. Updated Aoril 02, 2020

How intelligent is Google Duplex: latest insight, features, release date and more

How intelligent is Google Duplex: latest insight, features, release date and more

Google's latest update on Google Translate app is traveller savvy's dream

Google's latest update on Google Translate app is traveller savvy's dream

Do you know?

Your emails, which you consider private and official, are actually read by Google to collect information about you. For example, the Google app on your phone, knows when your next flight is leaving, and whether or not it has been delayed, based on emails you get from airlines and travel booking sites. You might have sometimes noticed Google Calendar automatically adding restaurant reservations and similar events to your schedules. Where do you think they got those information?