Projects/InfluxDB Setup: Difference between revisions

From Playful Exploration
Jump to navigation Jump to search
No edit summary
 
(One intermediate revision by the same user not shown)
Line 4: Line 4:
** Allow for high sampling rates e.g. 4 values/second
** Allow for high sampling rates e.g. 4 values/second
** Short retention for high sampling rate data e.g. 1 hour
** Short retention for high sampling rate data e.g. 1 hour
** Multiple levels of automatic down-sampling using aggregation with increasing retention for each level
** Multiple levels of automatic down-sampling using aggregation with increasing retention time for each level


== Retention and Down-Sampling ==
== Retention and Down-Sampling ==
Line 12: Line 12:
|-
|-
| 0.25 seconds (raw data) || 1 hour || 14400 ||  
| 0.25 seconds (raw data) || 1 hour || 14400 ||  
CREATE RETENTION POLICY default ON weather DURATION 1h REPLICATION 1 DEFAULT
CREATE RETENTION POLICY "default" ON "weather" DURATION 1h REPLICATION 1 DEFAULT
|-
|-
| 15 minutes || 10 years || 345600 ||
| 15 minutes || for ever || 345600 ||
CREATE RETENTION POLICY _10years ON weather DURATION 1d REPLICATION 1
CREATE RETENTION POLICY _15min ON weather DURATION INF REPLICATION 1


CREATE CONTINUOUS QUERY CQ_15min ON weather BEGIN SELECT median(value) AS value INTO weather._10years.:MEASUREMENT FROM weather."default"./^[a-z].*/ GROUP BY time(15m), * fill(none) END
CREATE CONTINUOUS QUERY CQ_15min ON weather BEGIN SELECT median(value) AS value INTO weather._15min.:MEASUREMENT FROM weather."default"./^[a-z].*/ GROUP BY time(15m), * fill(none) END


SELECT median(value) AS value INTO weather._10years.:MEASUREMENT FROM weather."default"./^[a-z].*/ where time > now() - 1d GROUP BY time(15m), * fill(none)
SELECT median(value) AS value INTO weather._15min.:MEASUREMENT FROM weather."default"./^[a-z].*/ where time > now() - 1d GROUP BY time(15m), * fill(none)
|}
|}
* 360000 measurement, 5.76MB at 16Byte per measurement
* 360000 measurement, 5.76MB at 16Byte per measurement

Latest revision as of 04:34, 23 May 2019

Objective

  • Provide a Database for measurements
  • Manage memory smartly:
    • Allow for high sampling rates e.g. 4 values/second
    • Short retention for high sampling rate data e.g. 1 hour
    • Multiple levels of automatic down-sampling using aggregation with increasing retention time for each level

Retention and Down-Sampling

sample rate/aggregate retention # of values InfluxDB Retention and Continuous Query Setup
0.25 seconds (raw data) 1 hour 14400

CREATE RETENTION POLICY "default" ON "weather" DURATION 1h REPLICATION 1 DEFAULT

15 minutes for ever 345600

CREATE RETENTION POLICY _15min ON weather DURATION INF REPLICATION 1

CREATE CONTINUOUS QUERY CQ_15min ON weather BEGIN SELECT median(value) AS value INTO weather._15min.:MEASUREMENT FROM weather."default"./^[a-z].*/ GROUP BY time(15m), * fill(none) END

SELECT median(value) AS value INTO weather._15min.:MEASUREMENT FROM weather."default"./^[a-z].*/ where time > now() - 1d GROUP BY time(15m), * fill(none)

  • 360000 measurement, 5.76MB at 16Byte per measurement