From the HR analogy previously, you saw how Pub/Sub works and how it's used in decoupling systems. Let's now discuss the technical details associated. We'll start by understanding the distribution of messages and different patterns. Here, the different colors represent different messages. The first pattern is just a basic straight through flow where one publisher publishes messages into a topic, which then get consumed by the one subscriber through the one subscription. The second pattern is FAN in or load balancing. Multiple publishers can publish the same topic, and multiple subscribers can pull from the same subscription, leveraging parallel processing. What you see here are two different publishers sending three different messages all on the same topic, that means the subscription will get all three messages. The third pattern is FAN out, where you have many use cases for the same piece of data, and all data is sent to many different subscribers. As you can see here, we have two subscriptions, so both are going to get the messages, both the red message and the blue message. Pub/Sub allows for both push and pull delivery. In the pull model, your clients are subscribers, and will be periodically calling for messages and Pub/Sub will just be delivering the messages since the last call. In the pull model, you're going to have to acknowledge the message as a separate step, so what you see here is we initially make the call to the subscribers, it pulls the messages, it gets the message back, and then separately, it acknowledges that message. The reason for this is because the pull queues are often used to implement some queuing system for work to be done. You don't want to acknowledge the message until you firmly have the message and have done the processing on it. Otherwise, you might lose the message if the system goes down. Therefore, we generally recommend you wait to acknowledge until after you have gotten it. In the pull model, the messages are stored for up to seven days, in the push model it actually uses an HTTP endpoint. You register a Webhook as your subscription, and Pub/Sub infrastructure itself will call you with the latest messages. In the case of push, you just respond with status 200 Okay for the HTTP call, and that tells Pub/Sub the message delivery was successful. It will actually use the rate of your success responses to self-limit, so that it doesn't overload your worker. The way the acknowledgments work is to ensure every message gets delivered at least once. What happens is, when you acknowledge a message, you acknowledge on a pull subscription basis. If you have two subscriptions, you have one acknowledge and the other one doesn't, the one that acknowledged will continue to get the messages. Pub/Sub will continue to try to deliver the message for up to seven days until it is acknowledged. There is a replay mechanism as well, that you can rewind and go back in time and have it replay messages, but in any case, you will always be able to go back seven days. You can also set the acknowledgment deadline and do that on a pull subscription. So if you know that on average it takes you 15 seconds to process a message in your work queue, then you might set your acknowledgment deadline to 20 seconds. This will ensure it doesn't try to redeliver the messages. Configuring a topic with message retention gives you more flexibility, allowing any subscription attached to the topic to seek back in time and replay previously acknowledged messages. Topic message retention also allows a subscription to replay messages published before a subscription was created. Snapshots are utilized to make replay highly efficient. If topic message retention is enabled, storage costs for the messages retained by the topic are to be billed to the topics project. Subscribers can work individually or as a group. If we have just one subscriber, it is going to get every message delivered through that subscription. However, you can set up worker pools by having multiple subscribers sharing the same subscription. In this case it is going to distribute the message, so one and three go to subscription one, and two goes to subscription two, and it is just random based on when it pulls from messages throughout the day. In the case of a push subscription, you only have one web endpoint, so you will only have one subscriber, typically, but that one subscriber could be an app Engine Standard App or Cloud Run Container image which auto scales. It is one web endpoint, but it can have Autoscale workers behind the scenes, and that is actually a very good pattern.
