RRD Data source - DS

A RRD Data Source is a generic description of a single variable use by RRD in a time graph. Data Source is used to fill the RRD database composed of Round Robin Archive. An association between the data source is established in the RRD Collector to assign a real variable to the Data Source (Mapping operation).

Data Sources are shortened to DS. By default RRD Collector assigns data source name, DS0, DS1 etc. Names are given arbitrary and can be changed.

A LoriotPro RRD database profile contains a single set of Data Source and one or multiple archives RRA. A set of Data source contains at least one data source. There is no limit to the number of data source that can be defined in a set but reasonably 10 is a maximum.

For example a set of two data sources of gauge type can be defined to graph the incoming and outgoing traffic of a network interface. The mapping between the DS0 and DS1 data source defined are later on mapped to the ifinoctets and ifoutoctets SNMP objects.

The types of variables that are supported by data source are explained below:��

Gauge	Variable of gauge type are fluctuant over time between a maximum and a minimum. The maximum is limited to a 64 bits value. Typically gauges are used to measure level of something, like cpu usage, connected users, waiting message in a queue etc. Depending on the SNMP agent behaviour, the value is instantaneous and cannot take in account peak values.
Counter	Variable of counter type are always increasing, when the limit is reached or detected (64 bits or 32 bits) counters wrap. The update function takes the overflow into account. The counter is stored as a per-second rate. When the counter overflows, RRDTool checks if the overflow happened at the 32bit or 64bit border and acts accordingly by adding an appropriate value to the result. They are commonly used to measure traffics on network interfaces.
Derive	The derive can be used on SNMP gauge type to reflect the trend (derive). This can be useful for gauges, for example, to measure the rate of people entering or leaving a room. Internally, derive works exactly like COUNTER but without overflow checks. So if your counter does not reset at 32 or 64 bit you might want to use DERIVE and combine it with a MIN value of 0.
Absolute	The absolute type is used on counters that are reset when they are read. This is used for fast counters which tend to overflow. So instead of reading them normally you reset them after every read to make sure you have a maximal time available before the next overflow. Another usage is for things you count like number of messages since the last update.

Data Source creation

The creation of a new data source is performed from the Data Source Settings tab.

To create a new RRD Data Source click the Create Data Source Button

RRD data source list

Click on the Create Data Source bouton to start the creation of a new set of Data Source items.

RRD data source edition

Each Data Source should have a unique name. Define simple name with alpha numeric characters, no space, no symbols.

Select the data source type , GAUGE, COUNTER, DERIVE, ABSOLUTE.

Next specify the HeartBeat. The heartbeat defines the maximum time interval between two database updates. The update interval is defined in the Global properties of the profile and set to one minute by default.

Three parameters are used to create Primary Data Point (PDP). PDP are used by RRD to generate record in the archive RRA.

1. The Polling interval defines the period at which the samples are collected by the RRD Collector. (The SNMP get or ping request)

2. The Update interval defines the period of RRD database feeding (PDP generation). The samples collected over this period are averaged.

3. The heartbeat defines the maximum allowed time between 2 samples.

We define a Heartbeat of 120 seconds two times the update interval of 60 seconds by default.

RRD PDP

The minimum value option allows you to specify the lower limit of acceptable value coming from the RRD Collector. Values under this limit are not taken and the UNKNOWN value is stored in the DataBase for this PDP.

The maximum value option allows you to specify the higher limit of acceptable value coming from the RRD Collector. Values higher than this limit are not taken and the UNKNOWN value is stored in the DataBase for this PDP.

COntribution of Don Baarda (The step is the Data acquisition interval in our terminology)

Here is an explanation by Don Baarda on the inner workings of RRDTool. It may help you to sort out why all this *UNKNOWN* data is popping up in your databases:

RRD gets fed samples at arbitrary times. From these it builds Primary Data Points (PDPs) at exact times every ``step'' interval. The PDPs are then accumulated into RRAs.

The rrd heartbeat' defines the maximum acceptable interval between samples. If the interval between samples is less than ``heartbeat'', then an average rate is calculated and applied for that interval. If the interval between samples is longer than ``heartbeat'', then that entire interval is considered ``unknown''. Note that there are other things that can make a sample interval ``unknown'', such as the rate exceeding limits, or even an ``unknown'' input sample.

The known rates during a PDP's "step'' interval are used to calculate an average rate for that PDP. Also, if the total "unknown'' time during the "step'' interval exceeds the "heartbeat'', the entire PDP is marked as ``unknown''. This means that a mixture of known and "unknown'' sample time in a single PDP "step'' may or may not add up to enough "unknown'' time to exceed "heartbeat'' and hence mark the whole PDP ``unknown''. So "heartbeat'' is not only the maximum acceptable interval between samples, but also the maximum acceptable amount of "unknown'' time per PDP (obviously this is only significant if you have ``heartbeat'' less than "step'').

The "heartbeat'' can be short (unusual) or long (typical) relative to the "step'' interval between PDPs. A short "heartbeat'' means you require multiple samples per PDP, and if you don't get them mark the PDP unknown. A long heartbeat can span multiple "steps'', which means it is acceptable to have multiple PDPs calculated from a single sample. An extreme example of this might be a "step'' of 5 minutes and a "heartbeat'' of one day, in which case a single sample every day will result in all the PDPs for that entire day period being set to the same average rate. -- Don Baarda <don.baarda@baesystems.com>