|Doug Hellmann||May 11, 2012 8:55 am|
|Loic Dachary||May 11, 2012 12:40 pm|
|Loic Dachary||May 11, 2012 12:55 pm|
|Doug Hellmann||May 11, 2012 1:18 pm|
|Doug Hellmann||May 11, 2012 1:21 pm|
|Loic Dachary||May 14, 2012 5:22 am|
|Doug Hellmann||May 14, 2012 7:15 am|
|Loic Dachary||May 14, 2012 10:03 am|
|Doug Hellmann||May 14, 2012 1:27 pm|
|Julien Danjou||May 15, 2012 3:04 am|
|Loic Dachary||May 15, 2012 4:52 am|
|Julien Danjou||May 15, 2012 5:21 am|
|Doug Hellmann||May 15, 2012 7:26 am|
|Nick Barcet||May 16, 2012 12:31 am|
|Loic Dachary||May 16, 2012 1:09 am|
|Julien Danjou||May 16, 2012 2:19 am|
|Doug Hellmann||May 16, 2012 8:05 am|
|Subject:||Re: [Openstack] [metering] resources metadata|
|From:||Julien Danjou (juli...@enovance.com)|
|Date:||May 15, 2012 3:04:56 am|
On Mon, May 14 2012, Loic Dachary wrote:
Each set of metering data will need to be associated with the appropriate metadata from the resource at the time the metering information was collected. The rate of change of metadata and metering events are different, though, so the timestamps of the metadata records are unlikely to match exactly with the values in the metering records. Depending on the clock resolution, it would be possible to have metadata changes and meter data with the same timestamp, resulting in an incorrect association. Indeed, good point.
We can work around that by maintaining proper foreign key references using the metadata version field as you describe in the schema above (so the resource id and metadata version value point to the correct metadata record). It will make recording the metering data less efficient because we will need to determine the current version for the resource metadata, but we can optimize that eventually through indexes and caching.
Aggregation will also need to take the metadata version into account, so
everywhere in the list of queries we say "by resource_id" we need to change that
to "by resource_id and version".
I added the idea of a format version for when the payload format changes and
tried to write down a description of the metadata storage matching this thread
in the wiki.
What do you think ?
I'm jumping in a bit late in the discussion, but there may be a point I miss in the current definition because, I think it's getting too complicated.
We now have 2 "payload" fields: one for meter and one for metadata.
For example, if you look at the c1 counter (instance) you need to store the "type" as payload of the meter. This is a metadata of the instance, but it's not currently defined as being stored in metadata, but in the "payload" field of the meter. Moreover, I'm rather sure there will soon be a counter with the need of 2 different "payload" information, and we'll have a problem since we can only store one in the current meter schema, so we'll store the second one as a metadata or something. So clearly the initial "payload" solution is not enough.
OTOH I find the metadata proposal in another table too much complicated. Why not storing what metadata in the meter.payload field in the same table (e.g. as a JSON string)?
I miss the point of the introduction of a dedicated metadata table with version string. It sounds to me like early optimization, which is the root of all evil. :) But I might miss something.
-- Julien Danjou // eNovance http://enovance.com // ✉ juli...@enovance.com ☎ +33 1 49 70 99 81