Finally the wait is over! Since the time Google CEO Sundar Pichai demonstrated smart reply feature in Gmail app at I/O 2018, we had been eagerly waiting for the release of this API. Google has just released the API and I have just completed thoroughly testing it.

In this post, I am going to build a very simple Android app to demonstrate how you can use this API on your app. But before we start building the app, I need to talk what Smart Reply really is and why is it smart.

Why smart reply?

Smart reply becomes very handy particularly if you are a manager of a company or a chat support provider or somebody who receives messages from lot of people. The smart reply here is not just a fixed set of replies for a fixed set of phrases or texts. It is actually a learning system(don't have the audacity to call it intelligent system, lol) which generates replies not only on the basis of the semantic but also on the basis of the context of the conversation.

But of course Natural Language processing is extremely complex and so it cannot be guaranteed that the model will always provide appropriate replies for all contexts or audiences. In case you encounter inappropriate reply suggestions, feel free to reach out to Firebase Support. Your feedback can definitely help improve the model further and also filter sensitive topics.

Key capabilities of ML kit smart reply API

1. It generates contextually-relevant suggestions. The reply suggestions are based on the full context of a conversation, not just a single message, resulting in suggestions that are more helpful to your users.

2. It runs on the device. The on-device model generates replies quickly, and doesn't require you to send users' messages to a remote server.

How the Smart Reply model works

• The model uses up to 10 of the most recent messages from a conversation history to generate reply suggestions.

• The mode first checks the language of the conversation and only attempts to provide suggestions when the language is determined to be English.

• The model compares the messages against a list of sensitive topics and won’t provide suggestions when it detects a sensitive topic.

• Given all the above consitions are satisfied, the model provides up to three suggestions.

• The number of number of suggested replies depends on how many meet a sufficient level of confidence(confidence score) based on the input to the model.

Building a simple demo app

You already use smart reply feature in Gmail app. Other than gamil app, have a look at this live demo:

Just to avoid unnecessary details, the app we are going to build is even a simpler one, just enough to demonstrate the working of this API. You can quickly download the code from the Downloads page and run it on your System.

1. In order to setup the project, just follow steps 1 to 7 of ML Kit Tutorial: How to detect faces with ML Kit API and identify key facial features.

2. Add the dependencies for the ML Kit Android libraries to your module (app-level) Gradle file (usually app/build.gradle):

  dependencies {
    // ...
    implementation 'com.google.firebase:firebase-ml-natural-language:18.2.0'
    implementation 'com.google.firebase:firebase-ml-natural-language-smart-reply-model:18.0.0'
  }

3. Make sure to disable compression of tflite files in your app-level build.gradle file:

  android {
      // ...
      aaptOptions {
          noCompress "tflite"
      }
  }

4. Create a conversation history object

The conversation history would be a chronologically-ordered List of FirebaseTextMessage objects, with the earliest timestamp first. You need to add the message and its timestamp to the conversation history objects whenever a user sends a message.

In the conversation we will have two users: local user for whom reply suggestions would be generated and the remote user on the basis of whose messages the suggestions would be generated. The most recent message would be from a non-local user. This is important because ML Kit suggests replies intended to be sent by the user of your app: the local user. You should be sure you're passing ML Kit a conversation log that ends with a message to which your user might want to reply.

conversation.add(FirebaseTextMessage.createForLocalUser(
        "How are you?", System.currentTimeMillis()));

Whenever the user receives a message, add the message, its timestamp, and the sender's user ID to the conversation history. The user ID can be any string that uniquely identifies the sender within the conversation. The user ID doesn't need to correspond to any user data, and the user ID doesn't need to be consistent between conversation or invocations of the smart reply generator.

conversation.add(FirebaseTextMessage.createForRemoteUser(
  "Are you coming back soon?", System.currentTimeMillis(), userId));

Obtaining the reply suggestions

In order to generate smart replies to a message, we need to get an instance of FirebaseSmartReply and pass the conversation history to its suggestReplies() method:

FirebaseSmartReply smartReply = FirebaseNaturalLanguage.getInstance().getSmartReply();
smartReply.suggestReplies(conversation)
        .addOnSuccessListener(new OnSuccessListener() {
            @Override
            public void onSuccess(SmartReplySuggestionResult result) {
                if (result.getStatus() == SmartReplySuggestionResult.STATUS_NOT_SUPPORTED_LANGUAGE) {
                    // The conversation's language isn't supported, so the
                    // the result doesn't contain any suggestions.
                } else if (result.getStatus() == SmartReplySuggestionResult.STATUS_SUCCESS) {
                    // Task completed successfully
                    // ...
                }
            }
        })
        .addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                // Task failed with an exception
                // ...
            }
        });

If this call succeeds, a SmartReplySuggestionResult object is passed to the success handler. This object contains a list of up to 3 suggested replies, which you can present to your user:

  for (SmartReplySuggestion suggestion : result.getSuggestions()) {
      String replyText = suggestion.getText();
  }

As stated earlier, ML Kit would generate replies only if it is confident about the relevance of the replies and the language of conversation is determined to be English.

That's all we have for now from the latest version of this API. I would be interested to know if this API would give your users a better conbersation experiuence.