atom feed17 messages in net.launchpad.lists.openstackRe: [Openstack] [metering] resources ...
FromSent OnAttachments
Doug HellmannMay 11, 2012 8:55 am 
Loic DacharyMay 11, 2012 12:40 pm 
Loic DacharyMay 11, 2012 12:55 pm 
Doug HellmannMay 11, 2012 1:18 pm 
Doug HellmannMay 11, 2012 1:21 pm 
Loic DacharyMay 14, 2012 5:22 am 
Doug HellmannMay 14, 2012 7:15 am 
Loic DacharyMay 14, 2012 10:03 am 
Doug HellmannMay 14, 2012 1:27 pm 
Julien DanjouMay 15, 2012 3:04 am 
Loic DacharyMay 15, 2012 4:52 am 
Julien DanjouMay 15, 2012 5:21 am 
Doug HellmannMay 15, 2012 7:26 am 
Nick BarcetMay 16, 2012 12:31 am 
Loic DacharyMay 16, 2012 1:09 am 
Julien DanjouMay 16, 2012 2:19 am 
Doug HellmannMay 16, 2012 8:05 am 
Subject:Re: [Openstack] [metering] resources metadata
From:Nick Barcet (nick@canonical.com)
Date:May 16, 2012 12:31:54 am
List:net.launchpad.lists.openstack

On Tue, May 15, 2012 at 10:26 AM, Doug Hellmann <doug@dreamhost.com <mailto:doug@dreamhost.com>> wrote:

On Tue, May 15, 2012 at 8:21 AM, Julien Danjou <juli@enovance.com <mailto:juli@enovance.com>> wrote:

On Tue, May 15 2012, Loic Dachary wrote:

> On 05/15/2012 12:05 PM, Julien Danjou wrote: >> >> OTOH I find the metadata proposal in another table too much >> complicated. Why not storing what metadata in the meter.payload field >> in the same table (e.g. as a JSON string)? > I would be much simpler to store the metadata in the resource_id field > which could be renamed into resource field.

That'd be even more radical.

I like it because it would simplify the messaging. We can leave the storage optimization question to the daemon that stores the data.

> Instead of resource_id=134123 we could have resource={ 'id': 134123, > 'name': 'foobar', 'flavor': 'm1.small' etc.. } There would be no need > for versioning, format, separate table, etc. etc. The only convention > would be that it's a hash with at least one field : the id of the > resource. The rest is metadata. > > It will use a lot of disk space with highly redundant information.

Ok, so the current proposal is just early optimization, as I understood.

If you want to optimize the storage, why not use resource_id as a foreign key to the metatable table which would contains unique records of metadata?

That would allow to store identical metadata once (and therefore optimize space) and will be much simpler. There would not be any need of version, timestamp, or whatever on metadata.

I fully second Julien's analysis regarding the storage of metadata in a separate table. We are overly complicating the schema for the sole purpose of resource optimization. In fact, the current metadata table seems to be defining 5 fields where only one constitutes some valid content we would like to extract at some point, the rest existing only for relational or versioning purposes. A bit of an overkill, it seems.

I would not recommend however to overcharge a field that can be used as a key value for searches (resource_id) to store additional data, as it would block us to search on that key easily. Why mot just extend the meter schema to add a metadata field, which can use the proposed JSON format?

Nick