Setup explanation:
I am using jmx prometheus agent, which starts with the VM as vm argument in HDFS setup.
Version: 0.20.0
There is a jmx http port displaying the Mbeans as JSON.
Problem:
When I view the /metrics port of prometheus agent, the values of the beans changes alternatively every time I refresh.
The first time is shows diskused 68Gb, next time it's 140G ... 68 -> 140 -> 68... so on.
LiveServersCount(dn) goes from 3 to 5 then to 3.
But in jmx http json port, it's consistent: 140GB.