atom feed17 messages in net.launchpad.lists.openstack[Openstack] [metering] public API design
FromSent OnAttachments
Doug HellmannMay 11, 2012 8:55 am 
Loic DacharyMay 11, 2012 12:40 pm 
Loic DacharyMay 11, 2012 12:55 pm 
Doug HellmannMay 11, 2012 1:18 pm 
Doug HellmannMay 11, 2012 1:21 pm 
Loic DacharyMay 14, 2012 5:22 am 
Doug HellmannMay 14, 2012 7:15 am 
Loic DacharyMay 14, 2012 10:03 am 
Doug HellmannMay 14, 2012 1:27 pm 
Julien DanjouMay 15, 2012 3:04 am 
Loic DacharyMay 15, 2012 4:52 am 
Julien DanjouMay 15, 2012 5:21 am 
Doug HellmannMay 15, 2012 7:26 am 
Nick BarcetMay 16, 2012 12:31 am 
Loic DacharyMay 16, 2012 1:09 am 
Julien DanjouMay 16, 2012 2:19 am 
Doug HellmannMay 16, 2012 8:05 am 
Subject:[Openstack] [metering] public API design
From:Doug Hellmann (doug@dreamhost.com)
Date:May 11, 2012 8:55:30 am
List:net.launchpad.lists.openstack

During yesterday's meeting we discussed the API proposal at http://wiki.openstack.org/EfficientMetering#API and came up with a few missing items and other points for discussion. We should try to work out those details on the list before the meeting next week.

The original proposal included these API calls:

GET list components GET list components meters (argument : name of the component) GET list [user_id|project_id|source] GET list of meter_type GET list of events per [user_id|project_id|source] ( allow to specify user_id or project_id or both ) GET sum of (meter_volume, meter_duration) for meter_type and [user_id|project_id|source]

They can be broken down into three groups:

- Discover the sorts of things the server can provide:

- list the components providing metering data - list the meters for a given component - list the known users - list the known projects - list the known sources - list the types of meters known

- Fetch raw event data, without aggregation:

- per user - per project - per source - per user and project

- Produce aggregate views of the data:

- sum "volume" field for meter type over a time period - per user - per project - per source - sum "duration" field for meter type over a period of time - per user - per project - per source

We agreed that all of the queries related to metering data would need to have parameters to set the start and end times with the start time inclusive and the end time exclusive (i.e., start <= timestamp < end).

Callers also are likely to want to filter the raw data queries based on the meter type, so we need to add that as an optional argument there. The aggregate values have the meter type as a required argument (because it does not make sense to aggregate data from separate meters together).

There are a few other queries missing from that list. The items below are based on the meeting notes and my own thoughts from yesterday, please add to the list if I have left anything out (or suggest why we should not implement something suggested here, of course):

- list discrete events that may not have a duration (instance creation, IP allocation, etc.) - list raw event data for a resource ("what happened with a specific instance?") - aggregate event data per meter type for a resource over a period of time ("what costs are related to this instance?") - sum volume for meter type over a time period for a specific resource ("how much total bandwidth was used by a VIF?") - sum duration for meter type over a time period for a specific resource ("how long did an instance run?") - metadata for resources (such as location of instances) - aggregating averages in addition to sums

Some of these items may be better handled in the consumer of this API (the system that actually bills the user). I am curious to know how others are handling auditing or other billing detail reporting for users.

In order to support the discrete events, we need to capture them. Perhaps those could be implemented as additional counter types. Thoughts?

In order to provide the metadata we need to capture it. Some metadata may change (location), so we need to support updates.

- The interesting metadata for a resource may depend on the type of resource. Do we need separate "tables" for that or can we normalize somehow? - How do we map a resource to the correct version of its metadata at any given time? Timestamps seem brittle. - Do we need to reflect the metadata in the aggregation API?

We also discussed extensions, but we should start another thread for that topic.

Doug