Welcome to another installment of Old Logs New Tricks.
Here we are going to look at routing some of the uglier non-useful data from the logs, pre-index time, to save on licensing as well as make the logs prettier.
So say we have a log like this(edited for security reasons!):
So, for example, let's say we don't want debug logs. So what we can do is go into Splunk and on the Heavy Forwarder tier(some dont have this type of deployment, in that instance it would be added to the indexer) and pipe those logs to the "nullQueue", with props.conf and transforms.conf: props.conf [new_sourcetype]
TRANSFORMS-bye_debug=kill_debug transforms.conf
[kill_debug]
REGEX=\"level\"\:\"debug\"
DEST_KEY=queue
FORMAT=nullQueue
Always test your regex:
Now in Splunk the debug logs will be gone laving only the informational messages
Doing this will save you time as well as money since sending un-necessary data to the nullQueue at index time keeps that data from being indexed and using the license that you pay for.
Thanks for following and have a great day!