In my previous blog post, “How to Explore Prometheus with Easy ‘Hello World’ Projects,' I described three projects that I used to get a better sense of what Prometheus can do. In this post, I’d like to share how I got more familiar with Prometheus Alertmanager and how I set up alert notifications for Slack, PagerDuty, and Gmail.
Slack for Gmail sits alongside your inbox, making it easy to bring emails into a channel or DM with just a few clicks. With Slack for Gmail you can: Forward emails from. Shift is a desktop app for streamlining all of your email and app accounts. Easily manage Gmail and Google Drive Messenger, WhatsApp, Slack, and 500+ apps. Slack is a communication tool that is created to streamline and simplify conversations. It's similar to other messages apps with the benefits of being searc. Gmail for G Suite now adds chat and video calling feature to take on Microsoft and Slack News Google has refreshed the design for Gmail for G Suite as it now adds other communication tools like. Whether you're looking for a way to get an Slack notifications for only the most important emails, such as from your boss, or you want to get a message when the latest issue of your favorite newsletter hits your inbox, rely on this Business Gmail-Slack integration.
(I’m going to reference my previous blog post quite a bit, so I recommend reading it before continuing on.)
First, you can get started with Prometheus in minutes with Grafana Cloud. We have free and paid Grafana Cloud plans to suit every use case — sign up for free now.
Setting up alerts with Prometheus is a two-step process:
First, you need to create your alerting rules in Prometheus, and specify under what conditions you want to be alerted (such as when an instance is down).
Second, you need to set up Alertmanager, which receives the alerts specified in Prometheus. Alertmanager will then be able to do a variety of things, including grouping alerts of similar nature into a single notification; silencing alerts for a specific time; muting notifications for certain alerts if other specified alerts are already firing; and picking which receivers receive a particular alert.
Step One: Create Alerting Rules in Prometheus
Before you begin, you’ll want to create a dedicated Prometheus folder with server, node_exporter, github_exporter and prom_middleware subfolders. (The process is explained here.)
After that, move to the server subfolder and open the content in the code editor, then create a new rules file. In the rules.yml, you will specify the conditions when you would like to be alerted.
I’m sure everyone agrees that knowing when any of your instances are down is very important. Therefore, I’m going to use this as our condition by using up metric. By evaluating this metric in the Prometheus user interface (http://localhost:9090), you will see that all running instances have value of 1, while all instances that are currently not running have value of 0 (we currently run only our Prometheus instance).
After you’ve decided on your alerting condition, you need to specify them in rules.yml. Its content is going to be following:
To summarize, it says that if any of the instances are going to be down (up 0) for one minute, then the alert will be firing. I have also included annotations and labels, which store additional information about the alerts. For those, you can use templated variables such as {{ $labels.instance }} which are then interpolated into specific instances (such as localhost:9100).
(You can read more about Prometheus alerting rules here.)
Once you have rules.yml ready, you need to link the file to prometheus.yml and add alerting configuration. Your prometheus.yml is going to look like this:
If you’ve started Prometheus with --web.enable-lifecycle flag, you can reload configuration by sending POST request to /-/reload endpoint curl -X POST http://localhost:9090/-/reload. After that, start prom_middleware app (node index.js in prom_middleware folder) and node_exporter (./node_exporter in node_exporter folder).
Once you do this, anytime you want to create an alert, you can just stop the node_exporter or prom_middleware app.
Step Two: Set up Alertmanager
Create an alert_manager subfolder in the Prometheus folder, mkdir alert_manager. To this folder, you’ll then download and extract Alertmanager from the Prometheus website, and without any modifications to the alertmanager.yml, you’ll run ./alertmanager --config.file=alertmanager.yml and open localhost:9093.
Depending on whether or not you have any active alerts, Alertmanager should be properly set up and look something like the image below. To see the annotations that you added in the step above, you’d click on the +Info button.
With all of that completed, we can now look at the different ways of utilizing Alertmanager and sending out alert notifications.
If you want to receive notifications via Slack, you should be part of a Slack workspace. If you are currently not a part of any Slack workspace, or you want to test this out in separate workspace, you can quickly create one here.
To set up alerting in your Slack workspace, you’re going to need a Slack API URL. Go to Slack -> Administration -> Manage apps.
In the Manage apps directory, search for Incoming WebHooks and add it to your Slack workspace.
Next, specify in which channel you’d like to receive notifications from Alertmanager. (I’ve created #monitoring-infrastructure channel.) After you confirm and add Incoming WebHooks integration, webhook URL (which is your Slack API URL) is displayed. Copy it.
Then you need to modify the alertmanager.yml file. First, open subfolder alert_manager in your code editor and fill out your alertmanager.yml based on the template below. Use the url that you have just copied as slack_api_url.
Reload configuration by sending POST request to /-/reload endpoint curl -X POST http://localhost:9093/-/reload. In a couple of minutes (after you stop at least one of your instances), you should be receiving your alert notifications through Slack, like this:
If you would like to improve your notifications and make them look nicer, you can use the template below, or use this tool and create your own.
And this is the final result:
PagerDuty is one of the most well-known incident response platforms for IT departments. To set up alerting through PagerDuty, you need to create an account there. (PagerDuty is a paid service, but you can always do a 14-day free trial.) Once you’re logged in, go to Configuration -> Services -> + New Service.
Choose Prometheus from the Integration types list and give the service a name – I decided to call mine Prometheus Alertmanager. (You can also customize the incident settings, but I went with the default setup.) Then click save.
The Integration Key will be displayed. Copy the key.
You’ll need to update the content of your alertmanager.yml. It should look like the example below, but use your own service_key (integration key from PagerDuty). Pagerduty_url should stay the same and should be set to https://events.pagerduty.com/v2/enqueue. Save and restart the Alertmanager.
Stop one of your instances. After a couple of minutes, alert notifications should be displayed in PagerDuty.
In PagerDuty user settings, you can decide how you’d like to be notified: email and/or phone call. I chose both and they each worked successfully.
If you prefer to have your notifications come directly through an email service, the setup is even easier. Alertmanager can simply pass along messages to your provider – in this case, I used Gmail – which then sends them on your behalf.
It isn’t recommended that you use your personal password for this, so you should create an App Password. To do that, go to Account Settings -> Security -> Signing in to Google -> App password (if you don’t see App password as an option, you probably haven’t set up 2-Step Verification and will need to do that first). Copy the newly-created password.
You’ll need to update the content of your alertmanager.yml again. The content should look similar to the example below. Don’t forget to replace the email address with your own email address, and the password with your new app password.
Once again, after a couple of minutes (after you stop at least one of your instances), alert notifications should be sent to your Gmail.
And that’s it!
Recent Posts
Related Posts
Categories
Related Posts
Related Case Studies
Keep up with us.
Product developments and observability innovations.
Expert-led sessions and product announcements on Grafana 8, Prometheus, Loki logs, and more.
As a workplace communication hub, Slack is frequently hailed by its founder and proponents as the solution to email’s limitations.
https://www.youtube.com/watch?v=TGY9dX1Mvsw
Compared to Slack, email generally makes it harder to share information and get a quick, easy response.
Compared to Email, Slack causes more interruptions with its frequent message notifications, that can often have a false sense of urgency.
Slack & Gmail: Rather than picking one platform or the other as the “winner,” both have their use cases:
- Email presents a non-interruptive hub that is key for external communications
- Slack is a more transparent and quick communication hub that is especially well-suited for internal communications
While there are advantages and disadvantages to both channels, we believe the biggest issues for teams are caused by having to switch between Gmail, the most widely used email client, and Slack for internal chat.
Flipping between Slack and Gmail creates a dispersion of information and attention. Furthermore: team collaboration is made more difficult by this dispersion.
To solve this problem: we’ve synced Slack & Gmail through our two-way integration. The existing Slack for Gmail integration simply sends emails to Slack, and Slack messages to email.
Gmelius does that and more to make email even more collaborative. How so? By creating a new layer that translates an email into a shared conversation, and then a shared instant message, and vice-versa.
We’re convinced that syncing data between Slack and Gmail in this way can provide the ultimate workplace communication hub. We’ve released this integration because we believe that it will create more efficiency, results, and happiness when communicating.
Slack For Gmail Sign In
To show you why our version of syncing Slack and Gmail is the solution, let’s dissect those issues about the Slacklash a bit further.
Slack vs. Email: Pros and Cons of Both Platforms
The commonly-reported pain points about Slack tend to be:
- The false sense of urgency created by Slack’s chatroom interface (not every message requires an immediate response)
- Frequent chat notifications that interrupt the state of flow needed to create valuable “deep work”
- Too much irrelevant chatter (both work-related and social)
- Inefficient onboarding of the tool by companies that introduce it, resulting in endless chats and channels
- A web of integrations is needed to use Slack with other collaboration tools (like Trello, Helpscout, Basecamp, etc)
That said, we like using Slack and Gmail — a lot. Why is Slack great for business?
1. Slack gives instant access to team members. The speed of communication that happens with brief messages and quick answers is a big advantage.
2. Slack makes it easier to keep everyone on the same page. It’s easier to make a company/department/project-wide announcement to notify everyone of the relevant news (without everyone involved getting CC’d on an endless email thread).
3. Slack makes work more transparent by being able to track everything that is happening (even when someone is typing).
4. Slack gives a feeling of control. For managers, seeing the team’s green dots light up helps them feel like everyone is working (and hopefully they are).
5. Slack’s chatroom interface makes it usable for team building, and creating good vibes among coworkers. While not a top benefit, this chatroom interface can’t be discounted.
Email is and will remain king in terms of being the central hub of business communications..but it lacks these benefits to collaboration that make Slack so widely-used.
Email used on its own presents these problems as a communication hub:
- Sharing information to large teams requires CC’ing a big list
- Tracking projects and statuses requires effort, especially when multiple email threads and people are involved
- You don’t have the ability to discuss email threads privately (unless you start a new email chain or discussion in different apps)
- You can’t send automated sequences of emails and track their stats
In contrast to email, Slack makes sharing information, tracking projects and statuses, and having internal discussions quicker and easier.
Most Companies Use Both Platforms: Slack & Email — Which Causes Problems
Both tools have their benefits, and as a result most companies end up using both. This causes a problem where data is dispersed between the two platforms.
Using both communication hubs means there will frequently be messages and files on one platform — but not the other (“Where was that message: in my email or Slack?”).
Slack’s collaborative features make it highly useful for teams but most communication to clients, customers, vendors, and contractors still goes through email. As a result, teams end up discussing projects in two different platforms.
Slack-Gmail: A More Collaborative Integration
One goal for business communications should be to keep the bar for sending messages high enough to allow team members to stay focused.
At the same time, we need the ability to communicate with each other quickly enough to make business happen.
With that in mind, there are several criteria to use Slack and Email at the same time productively.
These are the problems solved by our two-way syncing integration with Slack and Gmail:
1. Above all, we’ll reduce the channel dispersion by integrating and syncing communication between Slack and Email. How would this work?
To do it: We’ve created a full sync between a shared label or shared inbox and a Slack channel.
Slack For Gmail App
For instance, a Gmail shared inbox named “Support” or a shared inbox like support@dealet.com is created with Gmelius:
Through the Slack-Gmail integration that shared label or shared inbox will be synchronized with the #help channel on Slack:
To take it further: We’ve enabled teams to assign emails to each other, add email notes for context, add tags, change statuses, and more — without leaving Gmail:
Integrating the two channels in this way solves a whole host of problems caused by the two being separate channels.
First post! . The bottom line is: this reduces the need for additional communication between team members because with our new integration, they can now see the existing context in Gmail and Slack at the same time.
2. Stop the need to search and switch between a lot of tools to collaborate as a team.
Imagine replacing the need for a to-do assigning app like Asana and project boards like Trello, a client notes app like Basecamp, or even a Helpdesk with ticket-management and automated follow ups by containing all of those features inside your inbox.
With our integration, once a conversation happens on Slack, it can be added along with any relevant email threads or notes to a Gmelius Kanban board within Gmail.
This board also has the option for:
- A due date and priority level
- An assignment to a team member
- A trigger for an automated follow-up email
- Another trigger for a sequence of further follow-up emails and internal tasks using “If/Then” logic (like in the below image):
Keeping all these features in sync between tools is the better way of working that we envision. The goal is to reduce noise from multiple channels, of which Slack is one.
Fewer transitions between tools, and fewer redundant message exchanges to keep up with between them, can clearly save a lot of time.
3. Thirdly, end the endless creation of new Slack conversations and channels by dedicating team chats to address specific internal emails.
It’s not uncommon for teams to create new Slack channels for each department, client, prospect, projects, etc.
Instead, you can create a new Slack channel to discuss a specific email internally. Then, you’ll see that conversation next to that email when you’re back in your inbox. What a time to be alive free mp3 downloadselfieodd. When you’re done, just archive the email and channel.
Plus, less messages need to be exchanged thanks to the transparency of team actions within that discussion.
While in the inbox, Gmelius offers collision-detection which shows you when someone is: currently typing with a pulsing “..” underneath their avatar:
You can also see if they have read and/or replied to an email, and whether they are currently working or away right now.
This allows teams to avoid mistakes. For example: unknowingly sending two separate messages to the same client, or even worse — ignoring a client altogether. All thanks to having any discussion about that email in Slack synced to appear next to it.
Slack-Gmail Conclusion
As you can see, we have recently released our Slack integration to enable all of the solutions that I just described.
Now, the communication data that is important for collaboration can live on both platforms at the same time.
You can choose the platform you prefer (or simply use both as needed) as your primary communication hub to end the data/time lost from switching between them.
Slack For Gmail Email
You no longer have to track team activity in multiple tools, or have separate conversations about the same topic on multiple channels. Just easy communication that is synced with the best features needed for collaboration — everything in real-time across all your team members and all their devices.