Jun 6

Using Lookups in Splunk
By:Bitsioinc

We all know, lookups are very useful in enhancing the your original event data. It will add key value pairs to your existing event to make more sense of your data. Let’s dive into below topic on how to use CSV lookups.

Limitations the csv files:
There are some restrictions to the files that can be used for CSV lookups.

  1. The table in the CSV file should have at least two columns. One column represents a field with a set of values that includes values belonging to a field in your events. The column does not have to have the same name as the event field. Any column can have multiple instances of the same value, which is a multivalued field.
  2. The characters in the CSV file must be plain ASCII text and valid UTF-8 characters. Non-UTF-8 characters are not supported.
  3. CSV files cannot have “\r” line endings (OSX 9 or earlier)
  4. CSV files cannot have header rows that exceed 4096 characters.
Upload the lookup table file:
To use a lookup table file, you must upload the file to your Splunk platform.
Steps
  1. SelectSettings > Lookupsto go to the Lookups manager page.
  2. In the Actions column, clickAdd newnext toLookup table files.
  3. Select a Destination app from the list.
    Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
  4. ClickChoose Fileto look for the CSV file to upload. The Splunk software saves your CSV file in$SPLUNK_HOME/etc/system/lookups/, or in$SPLUNK_HOME/etc/<app_name>/lookups/if the lookup belongs to a specific app.
  5. Enter the destination filename. This is the name the lookup table file will have on the Splunk server. If you are uploading a gzipped CSV file, enter a filename ending in “.gz”. If you are uploading a plaintext CSV file, use a filename ending in “.csv”.
  6. ClickSave.
Share a lookup table file with apps:
After you upload the lookup file, tell the Splunk software which applications can use this file. The default app is Launcher.
  1. SelectSettings > Lookups.
  2. From the Lookup manager, clickLookup table files.
  3. ClickPermissionsin the Sharing column of the lookup you want to share.
  4. In the Permissions dialog box, underObject should appear in, selectAll appsto share globally. If you want the lookup to be specific to this app only, selectThis app only. You can also keep your lookup private by selectingKeep private.
  5. ClickSave.
Create a CSV lookup definition :
Steps
  1. SelectSettings > Lookups.
  2. ClickLookup definitions.
  3. ClickNew.
  4. Select a Destination app from the drop-down list.
    Your lookup table file is saved in the directory where the application resides. For example: $SPLUNK_HOME/etc/users/<username>/<app_name>/lookups/.
  5. Give your lookup definition a uniqueName.
  6. SelectFile-basedas the lookupType.
  7. Select theLookup filefrom the drop-down list. For a CSV lookup, the file extension must be .csv
  8. ClickSave.
Your lookup is defined as a file-based CSV lookup and appears in the list of lookup definitions.
Share the lookup definition with apps:
After you create the lookup definition, specify in which apps you want to use the definition.
  1. SelectSettings > Lookups.
  2. Click Lookup definitions.
  3. In the Lookup definitions list, clickPermissionsin the Sharing column of the lookup definition you want to share.
  4. In the Permissions dialog box, underObject should appear in, selectAll appsto share globally. If you want the lookup to be specific to this app only, selectThis app only. You can also keep your lookup private by selectingKeep private.
  5. ClickSave.
Define an automatic lookup:
Manual lookups are applied to the results of a search when they are invoked with thelookupcommand.Automatic lookupsare applied to all searches at search time.
A lookup definition that you have defined previously.
Steps
  1. In Splunk Web, selectSettings > Lookups.
  2. Under Actions for Automatic Lookups, clickAdd new.
  3. Select theDestination app.
  4. Give your automatic lookup a uniqueName.
  5. Select the Lookup table that you want to use in your fields lookup.

    This is the name of the lookup definition that you defined on the Lookup Definition page.

  6. In theApply tomenu, select a host, source, or source type value to apply the lookup and give it a name in thenamedfield.
  7. Under Lookup input fields provide one or more pairs of input fields.

    The first field is the field in the lookup table that you want to match. The second field is a field from your events that matches the lookup table field. For example, you can have an ip_address field in your events that matches an ip field in the lookup table. So you would enter ip = ip_address in the automatic lookup definition.

  8. Under Lookup output fields provide one or more pairs of output fields.

    The first field is the corresponding field that you want to output to events. The second field is the name that the output field should have in your events. For example, the lookup table may have a field named country that you may want to output to your events as ip_city. So you would enter country=ip_city in the automatic lookup definition.

  9. You can select the checkbox for Overwrite field values to overwrite the field values when the lookup runs.
    Note: This is equivalent to configuring your fields lookup in props.conf.
  10. ClickSave.
The Automatic lookup view appears, and the lookup that you have defined is listed.
The Automatic lookup field “descritption” is shown below in the event search.
Comments:
No Comments Found!
0

Leave Us A Reply

Your email address will not be published.