Hello everyone, welcome back to Old Logs New Tricks. If you are like us you are always looking to improve your Splunk indexing or search performance. Some things you can do to improve search performance is to move search time extractions to index time. While this is not recommended it is acceptable for a few use cases. As an example we have host names that look something like: nsab000css005.domain.com In this example we call the portion "css" role. Every host has this role and extracting it at search time causes really poor performance so moving it to index time on our Intermediate forwarders, where much of our pre-processing happens, is an acceptable change. To do this I created a file called fields.conf in the $SPLUNK_HOME/etc/apps/<app_name>/local on my Intermediate tier. Its a very simple file and looks something like this:
[example_fieldname]
INDEXED=true
Next I created a transforms.conf in $SPLUNK_HOME/etc/apps/<app_name>/local for the new field like:
[extract_role]
REGEX=\D{3}\D\d{1,4}(?<example_fieldname>...)\d{1,5}
FORMAT = example_fieldname::$1
SOURCE_KEY = MetaData:Host
WRITE_META = true
Then I created a props.conf in $SPLUNK_HOME/etc/apps/<app_name>/local to apply to all hosts. Here's a new "trick." Splunk doesn't work with wildcards (*) in stanzas, so "[host::*]" won't work. TO WORK AROUND THIS you can do this:
[(?::){0}host::*]
TRANSFORMS-roleextract = extract_role Next I went to $SPLUNK_HOME/etc/apps/<app_name>/metadata and created a local.meta file to set permissions to global for the search heads. (As we will copy this app to them in the next steps.)
[example_fieldname]
export = system Lastly, you will want these configurations on the search heads as well. For us in a clustered environment we copied the app to the deployer and then deployed it to the cluster, otherwise just add it to your search head(s). Verify in search that the data works as expected: host=* | dedup example_fieldname| table example_fieldname
example_fieldname css
hdn
dst
ifd
chb
dtx
cob
hnn
hen
usp
api
itl
jes
Now that it verified and you're done! Hope that helps you save some time and improve performance, it helped us. Have a great day!
Comments