Hello Everyone
Today I am going to briefly explain the Alert Manager JavaScript issue, how I debugged it and what I found to be the solution.
This in no way means it will fix your specific issue but it certainly may or at least guide you on how to find the root of your issue.
So Alert Manager is an add on app for Splunk. Its a 3rd party app so the app developers support the app, not Splunk......at least not much.
What Alert Manager does, per Splunk, "The Alert Manager adds simple incident workflows to Splunk. The general purpose is to provide a common app with dashboards in order to investigate fired alerts or notable events."
Essesntially it adds a workflow action to any given alert that creates an incident in the apps Incident Posture dashboard. That allows you to take actions like: assign a person to it, give it a status and assign a resolution.
What I found after upgrading Splunk versions is that somewhere an issue was introduced that caused Alert Manager to throw JavaScript erros when trying to take actions or edit settings. For example:
After a lot of digging and serching the internet I really couldnt find any docs or any resources that related to this issue, that had a resolution.
What I then did was reload the Incident Posture dashboard. I then opened the developer console by pressing "function button + f12".
Then I tried to edit something that would trigger the JavaScript error.
In the Devloper Console on the Console tab I saw this error:
When I clicked its ID on the right it gave me the spot in the code where the error was being thrown:
I then went to the apps appserver/static directory and grepped the JavaScript file(.js) for the stanza that was throing the error. What I found was that the script was calling select2 but select2 was not imported at the beginning:
What it should have looked like is:
What I decided to to was backup the kvstore, where all of the app settings are stored, and then uninstall the app completely. Then reinstall the app fresh from the Deployer and push it to the the Search Head Cluster. Configure it again per the install documentation via the Splunk Web app configure page. Then restart. Then restore the kvstore for the app.
Heres what that looked like starting with the backup of the kvstore(I assume you know how to uninstall an app).:
backup the kvstore
./splunk backup kvstore -archiveName splunkkvstorePreAppUpgrade -appName alert_manager
push the package from the Splunk Deployer to the SH Captain: ./splunk apply shcluster-bundle -target https://<SHCaptain>.com:8089
Configure via the Splunk Web app configure page(really its just redoing the global settings that were already thre, most importantly the index that records are being added to). Click "Perform Setup" when complete and restart Splunk.
restart Splunk
./splunk restart
restore the kvstore
./splunk restore kvstore -archiveName alertManager9272022.tar.gz -appName alert_manager
In doing this the issue with the .js files not having the proper imports called out was resolved. Once this process was completed there were no more JavaScript errors.
Again this may not solve your specific issue. However, it should help you get to where you can find the specific errors that are happening and can then find your own path to a resolution.
Hope this helps.
Have a great week
Cheers
-Todd