Subscribe Our Newsletter

Decoding Splunk Indexes definition

By Bitsioinc 06/07/2019 10:43 pm
No Comments

Decoding Indexes.conf

Indexes.conf configuration file is used to manage and configure index settings. Use the [default] stanza to define any global settings and [<index>] stanza to define index level settings. If a setting is defined at both the global level and in a specific stanza, the value in the specific stanza takes precedence.

Below is the list of all available settings in indexes.conf categorized into settings type such as path, size, time, count, boolean and value.

Global Settings :

 tsidxStatsHomePath
 rtRouterQueueSize
 bucketRebuildMemoryHint
 memPoolMB
 serviceInactiveIndexesPeriod
 serviceSubtaskTimingPeriod
 processTrackerServiceInterval
 hotBucketTimeRefreshInterval
 indexThreads
 selfStorageThreads
 maxRunningProcessGroups
 maxRunningProcessGroupsLowPriority
 sync
 rtRouterThreads
 assureUTF8
 enableRealtimeSearch
 inPlaceUpdates
 defaultDatabase
 lastChanceIndex
 suppressBannerList

Per Index Settings:

 bloomHomePath
 summaryHomePath
 tstatsHomePath
 remotePath

 maxTotalDataSizeMB
 maxGlobalDataSizeMB
 rawChunkSizeBytes
 maxMemMB
 minStreamGroupQueueSize

 disabled
 deleted
 isReadOnly
 createBloomfilter
 enableOnlineBucketRepair
 enableDataIntegrityControl
 syncMeta
 enableTsidxReduction
 suspendHotRollByDeleteQuery
 tsidxWritingLevel
 maxBloomBackfillBucketAge
 hotlist_recency_secs
 hotlist_bloom_filter_recency_hours
 rotatePeriodInSecs
 minRawFileSyncSecs
 quarantinePastSecs
 quarantineFutureSecs
 serviceMetaPeriod
 partialServiceMetaPeriod
 throttleCheckPeriod
 maxTimeUnreplicatedWithAcks
 maxTimeUnreplicatedNoAcks
 streamingTargetTsidxSyncPeriodMsec
 tsidxReductionCheckPeriodInSec
 timePeriodInSecBeforeTsidxReduction

 maxMetaEntries
 maxConcurrentOptimizes

 splitByIndexKeys
 journalCompression


bucketRebuildMemoryHintSuggestion for the bucket rebuild process for the size (bytes) of tsidx file it will try to build
hotlist_bloom_filter_recency_hoursThe cache manager attempts to defer eviction of the non-journal and non-tsidx bucket files, such as the bloomfilter file, until the interval between the bucket's latest time and the current time exceeds this setting
hotlist_recency_secsThe cache manager attempts to defer bucket eviction until the interval between the bucket's latest time and the current time exceeds this setting
inPlaceUpdatesIf true, metadata updates are written to the .data files directly
journalCompressiongzip|lz4|zstd * Defaults to gzip. * zstd is only supported in Splunk 7.2.x and later
maxConcurrentOptimizesThe number of concurrent optimize processes that can run against the hot DB
maxGlobalDataSizeMBThe maximum amount of local disk space (in MB) that a remote storage enabled index can occupy, shared across all peers in the cluster
maxMemMBThe amount of memory to allocate for indexing
maxMetaEntriesSets the maximum number of unique lines in .data files in a bucket
maxTimeUnreplicatedWithAcksPuts an upper limit on how long events can sit unacknowledged in a raw slice
minRawFileSyncSecsHow frequently we force a filesystem sync while compressing journal slices
minStreamGroupQueueSizeMinimum size of the queue that stores events in memory before committing them to a tsidx file
partialServiceMetaPeriodRelated to serviceMetaPeriod. If set, it enables metadata sync every seconds
processTrackerServiceIntervalControls how often, indexer checks status of the child OS processes it had launched to see if it can launch new processes for queued requests
rawChunkSizeBytesTarget uncompressed size in bytes for individual raw slice in the rawdata journal of the index
rotatePeriodInSecsControls the service period: how often splunkd performs certain housekeeping tasks
rtRouterThreadsSet this to 1 if you expect to use non-indexed real time searches regularly
rtRouterThreadsSet this to 1 if you expect to use non-indexed real time searches regularly
selfStorageThreadsSpecifies the number of threads used to transfer data to customer-owned remote storage
serviceMetaPeriodDefines how frequently metadata is synced to disk, in seconds
splitByIndexKeysValid values are: host, sourcetype, source, metric_name. This setting only applies to metric indexes.
streamingTargetTsidxSyncPeriodMsecPeriod we force sync tsidx files on streaming targets
suppressBannerListSuppresses index missing warning banner messages for specified indexes
syncThe index processor syncs events every number of events
throttleCheckPeriodDefines how frequently Splunk checks for index throttling condition
tsidxStatsHomePathAn absolute path that specifies where Splunk creates namespace data with 'tscollect' command

Comments are closed.