Messaging Using Google Pub/Sub and Spring Boot

You might have come across many instances when you open your email inbox, you find a promotional mail from Amazon stating for its upcoming sale but then you might find the same mail for your mail as well (if he allows you to peek to his inbox, of course). These mails are sent across all the consumers like you and me who have subscribed the promotional notifications from Amazon. These mails are a broad example of the publisher-subscriber design pattern which the bigger organizations like Google, Amazon, Spotify, etc. use regularly to promote events to all their consumers (or rather subscribers).

Publish/Subscribe Pattern

The Publish/Subscribe pattern, also known as pub/sub, is an architectural design pattern that provides a framework for exchanging messages between publishers and subscribers. This pattern involves the publisher and the subscriber relying on a message broker that relays messages from the publisher to the subscribers. The host (publisher) publishes messages (events) to a channel that subscribers can then sign up to.

Pub/Sub Communication Infrastructure. Image by Microsoft

Cloud Pub/Sub

Google Cloud Pub/Sub is an asynchronous messaging service built on a core Google infrastructure component that many Google products have relied upon for over a decade. Google products including Ads, Search and Gmail use this infrastructure to send over 500 million messages per second, totalling over 1TB/s of data. That’s freaking awesome! Isn’t it?

It is said to be Google’s own implementation of Apache Kafka and the fact that it is fully managed by Google makes it highly reliable and scalable. 

An excellent YouTube video on the overview of Pub/Sub’s application on real life

Some Hands-on To Get Started

Recently, I created a Spring boot application for my organization that is subscribed to a topic event and would perform a task on receiving the message from the publisher. The topic, of course, is created in Google Pub/Sub service. 

In this article, I will create a Pub/Sub topic in Google Cloud Platform and publish a message and this message will be subscribed by a Spring boot application and will log the message in the console. Obviously, the article will be a bare minimum demonstration to give you a gist of the Pub/Sub service.

Prequesities

  • You need to have a GCP account. If you don’t have one, you can easily create using your Credit/Debit card, they won’t charge you anything. The free trial gives you $300 for 90 days. You must refer here for more details.
  • You must be familiar with Cloud SDK for authenticating your local machine for connecting to Pub/Sub service. Read here for authenticating your machine with Google Cloud. 
  • Basic understanding of Spring boot framework (considering the fact that you know some Java). Of course, the programming language is no bar with learning Pub/Sub service, just that, for the scope of this article and demo, I am using Spring boot.

Creating Cloud Pub/Sub Topic & Subscription

Assuming that you already have created a Project in your GCP account, as every service in Google Cloud requires a Project which can be considered as a workspace for creating VPCs, VPNs, VM machines, Databases and other Cloud services.

My project name for creating Pub/Sub topic

Then you need to Enable Pub/Sub API for using the service. You can do this by hovering down over all the services on the menu and select Pub/Sub, which would take to the enabling Pub/Sub API page.

Select Pub/Sub to enable to API and use the service.

Once Pub/Sub service is enabled, you would be able to create a topic for publishing a message. So, let’s create a topic. I gave a name pubsubdemoTopic and clicked Create Topic.

From the Pub/Sub homepage, you then need to click Create Subscription button and create Subscription pubsubdemoSubscription.Youneed to select the Cloud Pub/Sub topic pubsubdemoTopic, delivery type as Pull and click Create.

Creating a Spring Boot Application

I have created a spring boot application that uses the subscription for reading the published messages. For that, you need to use two dependencies basically.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>

<dependency>
   <groupId>org.springframework.integration</groupId>
   <artifactId>spring-integration-core</artifactId>
</dependency>

The complete pom.xml for the Event Subscriber consists of two important dependencies, spring-cloud-gcp-starter-pubsuband spring-integration-core .Theapplication is a web application and would use the embedded tomcat server. 

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.3.4.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>

	<groupId>com.viveknaskar.eventsubscriber</groupId>
	<artifactId>event-subscriber</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>Event Subscriber</name>
	<description>Spring Boot Application with Google Cloud Platform's PubSub service</description>

	<properties>
		<java.version>11</java.version>
		<spring-cloud-gcp.version>1.2.5.RELEASE</spring-cloud-gcp.version>
	</properties>

	<dependencies>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.integration</groupId>
			<artifactId>spring-integration-core</artifactId>
		</dependency>

	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-gcp-dependencies</artifactId>
				<version>${spring-cloud-gcp.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

For pulling messages from the subscription created, I created a service called EventSubscriberService. 

EventSubscriberService.java

package com.viveknaskar.eventsubscriber;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.cloud.gcp.pubsub.core.PubSubTemplate;
import org.springframework.cloud.gcp.pubsub.integration.AckMode;
import org.springframework.cloud.gcp.pubsub.integration.inbound.PubSubInboundChannelAdapter;
import org.springframework.cloud.gcp.pubsub.support.BasicAcknowledgeablePubsubMessage;
import org.springframework.cloud.gcp.pubsub.support.GcpPubSubHeaders;
import org.springframework.context.annotation.Bean;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.MessageHandler;
import org.springframework.stereotype.Service;

@Service
public class EventSubscriberService {

    /**
     *  Created an inbound channel adapter to listen to the subscription `pubsubdemoSubscription`
     *  and send messages to the input message channel.
     */
    @Bean
    public PubSubInboundChannelAdapter messageChannelAdapter(
            @Qualifier("pubsubInputChannel") MessageChannel inputChannel,
            PubSubTemplate pubSubTemplate) {
        PubSubInboundChannelAdapter adapter =
                new PubSubInboundChannelAdapter(pubSubTemplate, "pubsubdemoSubscription");
        adapter.setOutputChannel(inputChannel);
        adapter.setAckMode(AckMode.MANUAL);

        return adapter;
    }

    /**
     * Created a message channel for messages arriving from the 
     * subscription `pubsubdemoSubscription`.
     */
    @Bean
    public MessageChannel pubsubInputChannel() {
        return new DirectChannel();
    }

    /**
     * Defined what happens to the messages arriving in the message channel.
     */
    @Bean
    @ServiceActivator(inputChannel = "pubsubInputChannel")
    public MessageHandler messageReceiver() {
        return message -> {
           System.out.println("Message arrived! Payload: " + new String((byte[]) message.getPayload()));
            BasicAcknowledgeablePubsubMessage originalMessage =
                    message.getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE,
                            BasicAcknowledgeablePubsubMessage.class);
            originalMessage.ack();
        };
    }
}

The inboundChannelAdapter asynchronously pulls messages from the subscriptionpubsubdemoSubscription using a PubSubTemplate and sends the messages to inputMessageChannel. The inboundChannelAdapter sets the acknowledgement mode to MANUAL so the application can acknowledge messages after it processes them. The default acknowledgement mode of PubSubInboundChannelAdapter type isAUTO. The ServiceActivator bean messageReceiver logs each message arriving in inputMessageChannel to the standard output and then acknowledges the message.

Publishing Message & Printing from Subscription

For publishing message, you must go to the Pub/Sub homepage and then select Topics, you must click Publish Message.

A page would appear for you to publish a message in the Message Body and then Publish.

Now since the application is already subscribed to the topic, the message would be successfully logged onto the console. 

Message logged in the console

If you followed along, then congratulations, you just created a subscription application that logs the messages from Cloud Pub/Sub topics. 

You can get the full code from this GitHub repository.

That’s a Wrap!

This was just a small overview of the publisher-subscriber pattern and this is just one small application. There are so numerous things a Pub/Sub service can do and to be honest, it is barely covered in this article. I would strongly suggest reading the documentation for more insights.

Pub/Sub service has been implemented by all the other major cloud service providers including Amazon Web Services (AWS) and Microsoft Azure. For Google Cloud, Pub/Sub is one of the most used services in the platform and has been prominent in building various ‘Enterprises’ Event Delivery Systems’.

One such example is Spotify’s Event Delivery System which you read from their documentation.

Spotify’s Event Delivery – The Road to the Cloud (Part II)
Whenever a user performs an action in the Spotify client-such as listening to a song or searching for an artist-a small…engineering.atspotify.com


If you enjoyed reading this, you might also find the below articles worth your time.

A Developer’s Guide For Passing Google Cloud Associate Cloud Engineer Exam
One of the goals of mine was to get certified in Cloud. When they said, “Everything is moving to cloud” a few years…thedeveloperstory.com

Features That Every Developer Must Know About Spring Boot
If you are not living under the rock, then you must have heard about Spring Boot, the framework which provides a…thedeveloperstory.com

Everything A Developer Must Know About Microservices
Microservices have become the application platform of choice for cloud applications development. Nginx conducted a…thedeveloperstory.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close