A long due update
A while back I setup influx DB to store the sensor and activity data on my smart home system. And i thought the raspberry pi 3 was not enough to run the server anymore. And moved to a NUC and all was fine.
But recently I wanted to upgrade from Ubuntu 18 to 20 and thought about making an image of the SSD first, as a quick and easy way to restore in case of problems.
It wrote how much?
When I checked the drive status … what do I see? Total writes to the disk – 288 TB. Yes, that is 288 terra bytes. A check with IOTOP shows that there is indeed a periodic high CPU and write to the disk from influx, some MB/s which is not really accounted for by how much sensor data is really gathered.
So on to investigating – quickly see that Influx has an internal database where it stores time series data regarding its operation parameters. And boy does it save a lot of parameters. Checking the size – indeed the internal database is 2.4GB, which can explain the rather large writes to the disk every 10 seconds. “InfluxData does not recommend using the _internal database in a production cluster. It creates unnecessary overhead…”. Well, maybe they should not have enabled it by default. So on to disabling it.
After the fix
The pictures speaks for itself: CPU load and temperature were reduced, there is more free RAM and large disk writes are gone.
So moral of the story: make sure to disable the internal influx database.