Set up the Alert Webhook in Splunk Synthetic Monitoring
- In Splunk Synthetic Monitoring, go to Admin > Alert Webhooks > + New and click ServiceNow.
- Under Name, provide a name for your webhook.
- Click Generate Header to generate the encoded authorization header that will be used with this webhook. Provide the credentials for the ServiceNow user that will be used to create an incident. Note: Make sure the ServiceNow user has the permissions to create, modify and close incidents.
- Under Triggers, select the triggers you want to activate. There are two types: 'When failed' and 'When back online'. 'When failed' opens a new incident in ServiceNow when a check fails and 'When back online' closes that same incident when that check resumes functioning. Note: you should not use the 'When back online' trigger without also using a 'When failed' trigger.
- Paste the base URL of your ServiceNow instance and edit the Webhook URLs for each trigger's endpoint, replacing "<<YOUR_SERVICENOW_BASE_URL>>" with the base URL for your ServiceNow instance.
- (optional) Click JSON to open the JSON request body for the API request that will be sent to HipChat upon the trigger event. Edit the JSON as needed to change the text and alert format to something that better suits your workflow, and then click Save. Note: When customizing a 'When back online' trigger, the state and close_code fields should not be changed.
- Click Test test your webhook by sending a request to the ServiceNow endpoint.
- Click Create to save the webhook.
Add your webhook to a Splunk Synthetic Monitoring check
You need to add your webhook to each check you want to send notifications from. To add your webhook to a check, do the following:
- Navigate to a check in Splunk Synthetic Monitoring, click the gear icon, and select Edit from the dropdown.
- Under Notify, select your new webhook from the dropdown. Use the second toggle menu to select your desired notification frequency.
- Click Save.
With the webhook set up, new ServiceNow incidents will be created with the specified details any time a check to which the webhook has been added fails. The incident will be closed automatically when the check is back online.