We can use Alert Webhooks to notify an HTTP endpoint when a Real Browser check or Uptime check fails. This endpoint could be a third-party alerting service, a company chat application, or even a custom API. To use Alert Webhooks, we need to:
- Create a new Alert Webhook
- Edit a check to notify the webhook
We can select from the following pre-configured services: Splunk On-Call
Configure a webhook to send to a custom endpoint
Send alerts to Splunk On-Call
Splunk Synthetic Monitoring verifies applications are performing well and alerts you to user experience problems before they impact your customers. Integrating Splunk Synthetic Monitoring with Splunk On-Call allows you to send alert notifications into the Splunk On-Call timeline.
You must be an administrator in Splunk Synthetic Monitoring and a Global or Alert Admin in Splunk On-Call to establish this integration.
In Splunk On-Call
In Splunk On-Call, select Integrations >> Splunk Synthetic Monitoring
If the integration has not yet been enabled, click the “Enable Integration” button to generate your endpoint URL. Copy this URL to your clipboard.
In Splunk Synthetic Monitoring (Rigor)
From the Rigor Monitoring web portal, select the 3 dot menu next to your profile icon in the upper-righthand corner and select Alert Webhooks.
Click the + New button to create a new webhook and select Splunk On-Call as the type.
Give the webhook a name and specify your trigger or triggers. In the “send request to” box for each trigger, replace the default
https://alert.victorops.com/integrations/<<YOUR_VICTOROPS_REST_ENDPOINT
with the URL you’d previously copied.
On each trigger, also replace the trailing $routing_key value with the Splunk On-Call routing key you’d like this webhook to hit. For more information on routing keys, see this article.
You’re welcome to leave the payloads at their default values, or if you find it more appropriate, replace the default variables with ones more catered towards your use case.
Once finished, click the Test button to test the integration out. This will send an alert with your specified payloads into the Splunk On-Call timeline.
Lastly, hit the Create button at the bottom of the screen to save your Alert Webhook. Splunk Synthetic Monitoring will now send a notification to Splunk On-Call whenever the trigger conditions are met.
Send alerts to Datadog
- Create a new Alert Webhook under Admin > Alert Webhooks
- Select the pre-built webhook for Datadog
- Select the ‘Triggers’ you want to activate. There are two types: ‘When failed’ and ‘When back online’
- Insert your Datadog API key into the Trigger endpoint URLs
- OPTIONAL: If you want, you can customize the JSON data sent in the notification via the ‘Payload’ section. This allows you to change the text and alert format to something that better suits your workflow.
- Once you have saved and named your webhook, it can be added to any Rigor monitoring check via the check notifications page in the check edit menu. From that point forward, your Datadog event’s stream will receive a POST request with the specified JSON payload any time your check fails. Example failures would look like this:
Send alerts to HipChat
-
Log in to HipChat and go to ‘Rooms’
- Select the room to receive Rigor alert notifications and click ‘Integrations’ in the left menu
-
Click Create on the ‘Build Your Own!’ option
- Name the integration and click Create
-
Copy the integration URL and click Save
- In Rigor, go to Admin > Alert Webhooks > + New and create a HipChat Alert Webhook
-
Paste the HipChat integration URL copied in step 5 in the ‘Webhook URL’ field for each trigger
- (optional) Customize the JSON data to be sent to HipChat
- Save the webhook
-
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will post notifications to HipChat:
Send alerts to OpsGenie
- Log in to OpsGenie and go to ‘Integrations’
-
Find the Rigor integration and click Add (or click here)
-
Copy the ‘API Key’ in the integration settings
- In Rigor, go to Admin > Alert Webhooks > + New and create an OpsGenie Alert Webhook
-
Paste the OpsGenie API key copied in step 3 in the ‘apiKey’ field in the ‘Payload’ section for each trigger
- (optional) Customize the JSON data to be sent to OpsGenie
- Save the webhook
-
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger alerts in OpsGenie:
When the check comes back online, the associated OpsGenie alert will get closed:
For more, see the OpsGenie integration documentation.
Send alerts to PagerDuty
-
Log in to PagerDuty and go to Configuration > Services
-
Click Add Services, name the Service, and select ‘Rigor’ as the integration type
- Click Add Service
-
Copy the ‘Service API Key’ of the new PagerDuty Service
- In Rigor, go to Admin > Alert Webhooks > + New and create a PagerDuty Alert Webhook
-
Paste the PagerDuty Service API Key copied in step 4 in the ‘service_key’ field in the ‘Payload’ section for each trigger
- (optional) Customize the JSON data to be sent to PagerDuty
- Save the webhook
-
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger incidents in PagerDuty:
When the check comes back online, the associated PagerDuty incident will get resolved:
Send alerts to Slack*
*You need to have a paid, 'Standard' Slack plan to integrate as a webhook.
-
Log in to Slack and go to ‘Integrations’
-
Click the ‘Incoming WebHooks’ option (or click here)
-
Select the Slack channel to receive alerts and click Add Incoming WebHooks Integration
-
Copy the ‘Webhook URL’ for the new Incoming WebHook
- In Rigor, go to Admin > Alert Webhooks > +New > [Slack]
-
Paste the Webhook URL copied in step 4 in the ‘Webhook URL’ field for each trigger
- (optional) Customize the JSON data to be sent to Slack
- Save the webhook
-
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will post notifications to Slack:
Send Alerts to ServiceNow
- Create a new Alert Webhook under Admin > Alert Webhooks
- Select the icon for ServiceNow
- Provide the credentials for the ServiceNow user that will be used to create an incident. Click "Generate Header" button to generate the encoded authorization header that will be used with this webhook. Note: Make sure the ServiceNow user has the permissions to create, modify and close incidents.
- Select the 'Triggers' you want to activate. There are two types: 'When failed' and 'When back online'. 'When failed' will open a new incident in ServiceNow when a check fails and 'When back online' will close that same incident when that check resumes functioning. Note: you should not use the 'When back online' trigger without also using a 'When failed' trigger.
- Find the base URL of your ServiceNow instance. In the new webhook form, edit the URLs for each trigger's endpoint, replacing "<<YOUR_SERVICENOW_BASE_URL>>" with the base URL for your ServiceNow instance.
- OPTIONAL: If you want, you can customize the JSON data sent in the notification via the 'Payload' section. This allows you to change the text and alert format to something that better suits your workflow. Note: When customizing a 'When back online' trigger, the state and close_code fields should not be changed.
- Once you have saved and named your webhook, it can be added to any Rigor monitoring check via the check notifications page in the check edit menu. From that point forward, new ServiceNow incidents will be created with the specified details any time a check fails. In turn, the incident will be closed automatically when the check is back online.
Send alerts to a custom endpoint
- In Rigor, go to Admin > Alert Webhooks > + New and create a Custom Alert Webhook
- (optional) Add Request Headers
- In the ‘Webhook URL’ field for each trigger, provide the endpoint to receive Rigor alerts
- (optional) Customize the JSON data to be sent in the webhook
- Test the webhook (check API endpoint to verify results)
- Save the webhook
-
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger POST requests to the specified endpoints.