|
Getting your Trinity Audio player ready...
|
You might have come across many instances when you open your email inbox, you find a promotional mail from Amazon stating for its upcoming sale but then you might find the same mail for your mail as well (if he allows you to peek into his inbox, of course). These mails are sent across all the consumers like you and me who have subscribed to the promotional notifications from Amazon. These mails are a broad example of the publisher-subscriber design pattern which the bigger organizations like Google, Amazon, Spotify, etc. use regularly to promote events to all their consumers (or rather subscribers).
Publish/Subscribe Pattern
The Publish/Subscribe pattern, also known as pub/sub, is an architectural design pattern that provides a framework for exchanging messages between publishers and subscribers. This pattern involves the publisher and the subscriber relying on a message broker that relays messages from the publisher to the subscribers. The host (publisher) publishes messages (events) to a channel that subscribers can then sign up to.
Cloud Pub/Sub
Google Cloud Pub/Sub is an asynchronous messaging service built on a core Google infrastructure component that many Google products have relied upon for over a decade. Google products including Ads, Search and Gmail use this infrastructure to send over 500 million messages per second, totalling over 1TB/s of data.
That’s freaking awesome! Isn’t it?
It is said to be Google’s own implementation of Apache Kafka and the fact that it is fully managed by Google makes it highly reliable and scalable.
Some Hands-on To Get Started
Recently, I created a Spring boot application for my organization that is subscribed to a topic event and would perform a task on receiving the message from the publisher. The topic, of course, is created in Google Pub/Sub service.
In this article, I will create a Pub/Sub topic in Google Cloud Platform and publish a message and this message will be subscribed by a Spring boot application and will log the message in the console. Obviously, the article will be a bare minimum demonstration to give you a gist of the Pub/Sub service.
Prequesities
- You need to have a GCP account. If you don’t have one, you can easily create using your Credit/Debit card, they won’t charge you anything. The free trial gives you $300 for 90 days. You must refer here for more details.
- You must be familiar with Cloud SDK for authenticating your local machine for connecting to Pub/Sub service. Read here for authenticating your machine with Google Cloud.Â
- Basic understanding of Spring boot framework (considering the fact that you know some Java). Of course, the programming language is no bar with learning Pub/Sub service, just that, for the scope of this article and demo, I am using Spring boot.
Creating Cloud Pub/Sub Topic & Subscription
Assuming that you already have created a Project in your GCP account, every service in Google Cloud requires a Project which can be considered as a workspace for creating VPCs, VPNs, VM machines, Databases, and other Cloud services.
Then you need to Enable Pub/Sub API for using the service. You can do this by hovering down over all the services on the menu and select Pub/Sub, which would take you to the enabling Pub/Sub API page.
Once Pub/Sub service is enabled, you would be able to create a topic for publishing a message. So, let’s create a topic. I gave a name pubsubdemoTopic and clicked Create Topic.
From the Pub/Sub homepage, you then need to click Create Subscription button and create Subscription pubsubdemoSubscription. You need to select the Cloud Pub/Sub topic pubsubdemoTopic, delivery type as Pull and click Create.
Creating a Spring Boot Application
I have created a spring boot application that uses the subscription for reading the published messages. For that, you need to use two dependencies, basically.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-core</artifactId>
</dependency>The complete pom.xml for the Event Subscriber consists of two important dependencies, spring-cloud-gcp-starter-pubsuband spring-integration-core. The application is a web application and would use the embedded tomcat server.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.4.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.viveknaskar.eventsubscriber</groupId>
<artifactId>event-subscriber</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Event Subscriber</name>
<description>Spring Boot Application with Google Cloud Platform's PubSub service</description>
<properties>
<java.version>11</java.version>
<spring-cloud-gcp.version>1.2.5.RELEASE</spring-cloud-gcp.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-core</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-dependencies</artifactId>
<version>${spring-cloud-gcp.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
For pulling messages from the subscription created, I created a service called EventSubscriberService.
EventSubscriberService.java
package com.viveknaskar.eventsubscriber;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.cloud.gcp.pubsub.core.PubSubTemplate;
import org.springframework.cloud.gcp.pubsub.integration.AckMode;
import org.springframework.cloud.gcp.pubsub.integration.inbound.PubSubInboundChannelAdapter;
import org.springframework.cloud.gcp.pubsub.support.BasicAcknowledgeablePubsubMessage;
import org.springframework.cloud.gcp.pubsub.support.GcpPubSubHeaders;
import org.springframework.context.annotation.Bean;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.MessageHandler;
import org.springframework.stereotype.Service;
@Service
public class EventSubscriberService {
/**
* Created an inbound channel adapter to listen to the subscription `pubsubdemoSubscription`
* and send messages to the input message channel.
*/
@Bean
public PubSubInboundChannelAdapter messageChannelAdapter(
@Qualifier("pubsubInputChannel") MessageChannel inputChannel,
PubSubTemplate pubSubTemplate) {
PubSubInboundChannelAdapter adapter =
new PubSubInboundChannelAdapter(pubSubTemplate, "pubsubdemoSubscription");
adapter.setOutputChannel(inputChannel);
adapter.setAckMode(AckMode.MANUAL);
return adapter;
}
/**
* Created a message channel for messages arriving from the
* subscription `pubsubdemoSubscription`.
*/
@Bean
public MessageChannel pubsubInputChannel() {
return new DirectChannel();
}
/**
* Defined what happens to the messages arriving in the message channel.
*/
@Bean
@ServiceActivator(inputChannel = "pubsubInputChannel")
public MessageHandler messageReceiver() {
return message -> {
System.out.println("Message arrived! Payload: " + new String((byte[]) message.getPayload()));
BasicAcknowledgeablePubsubMessage originalMessage =
message.getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE,
BasicAcknowledgeablePubsubMessage.class);
originalMessage.ack();
};
}
}
The inboundChannelAdapter asynchronously pulls messages from the subscriptionpubsubdemoSubscription using a PubSubTemplate and sends the messages to inputMessageChannel. The inboundChannelAdapter sets the acknowledgement mode to MANUAL so the application can acknowledge messages after it processes them. The default acknowledgement mode of PubSubInboundChannelAdapter type isAUTO. The ServiceActivator bean messageReceiver logs each message arriving in inputMessageChannel to the standard output and then acknowledges the message.
Publishing Message & Printing from Subscription
For publishing a message, you must go to the Pub/Sub homepage and then select Topics, you must click Publish Message.
A page would appear for you to publish a message in the Message Body and then Publish.
Now, since the application is already subscribed to the topic, the message would be successfully logged onto the console.Â
If you followed along, then congratulations, you just created a subscription application that logs the messages from Cloud Pub/Sub topics.
You can get the full code from this GitHub repository.
That’s a Wrap!
This was just a small overview of the publisher-subscriber pattern, and this is just one small application. There are so numerous things a Pub/Sub service can do and to be honest, it is barely covered in this article. I would strongly suggest reading the documentation for more insights.
Pub/Sub service has been implemented by all the other major cloud service providers, including Amazon Web Services (AWS) and Microsoft Azure. For Google Cloud, Pub/Sub is one of the most used services in the platform and has been prominent in building various ‘Enterprises’ Event Delivery Systems’.
One such example is Spotify’s Event Delivery System, which you read from their documentation.
If you enjoyed reading this, you might also find the below articles worth your time.
If this article provided you with value, please support my work — only if you can afford it. You can also connect with me on X. Thank you!
