I remember the times when I started to work for my first job at a big company. It was a large multinational telecommunications company, and one of the earliest concepts I have ever learned was “metering”. Up to that time, I was assuming everything one measures on a telephone switch is all about charging customers, and to understand how many minutes they have spent on the phone. Upon pouring over the code that runs on the switch and the peripherals, I came to realize the code was measuring the use of many things, and all was because of understanding the resource use of various different features, which also happens to be including the speech channels utilized.

Having a ubiquitous measuring of the resources a solution is using, that does not get in the way of the actual operations of it becomes even a more important concept, if the solution is utilizing resources from a different system, and potentially being charged for them. The urge to learn and analyze the tenants’ respective indirect utilization of Windows Azure resources the ISV is paying for the solution may have many different reasons. A few of the first that may come to mind can be:

• How can I enforce quotas for tenants, or do I have tojQuery1520412992729106918_1337901680773
• Are there any tenants consuming most of the resources?
• Do we need to introduce pricing tiers?
• How does the cost of goods for our solution get affected per tenant?
• Are certain tenants causing starvation for other tenants?
• Is my system behaving accordingly? Are we experiencing any resource usage anomalies?

The metering application block (which can be found at http://cnmb.codeplex.com), attempts to provide a solution to the ISVs asking those questions to themselves about their solution. The goal of the application block is to generate data where an ISV can compare the resource usage among the tenants.

Of course, given the wide range of pricing packages Microsoft provides, and possible changes Microsoft may take in their pricing policies, it would not be possible to calculate monetary values for those resource usages. It is also important to point out that the collected data is not the equivalent of the data Microsoft uses to calculate the bills at the end of each month. There can surely be differences in the output, since Microsoft billing has a different goal (to bill the customers) than the Cloud Ninja Metering block (to track relative usages across tenants for major resources).

What to Meter and Why

We have started with mentioning that, the CNMB is not a mechanism to analyze and double check your Azure bill details, however it is meant to provide the relative resource usage of the tenants. But what exactly do we need to measure to see the relative usage patterns? A good starting point is whatever is already there, provided by the Windows Azure page as the main pivots for a pricing discussion.

To make our point about metering not being related with the pricing, we also included a provider that measures the ingress data (data coming to the data center), which is not charged by Microsoft. The reason we chose to provide this is, heavy incoming data traffic by certain tenants may indicate additional data usage on certain other resources, such as compute time.

You can obviously start measuring many different metrics; however the block comes with 5 default providers that map to the major pivots:

• Blob size: Measured by the blob store size provider
• Database size: Measured by the database size provider
• Bandwidth : Measured by the IIS bandwidth provider, the database bandwidth provider and the blob bandwidth provider

Of course it goes without saying that depending on the solutions needs, there may be needs for measuring other aspects. Number of documents processed, active users, queue lengths could be among those.

In designing the block, we concentrated on four tenets:

• Block can be easily hosted in any hosting application, and does not depend on Azure runtime
• Does not require a code change for the related application, and be application agnostic
• Rely heavily on configuration, not coding
• Easily extensible

You can reach us at info@fullscale180.com for further questions/comments.

Have a great day!

I remember the times when I started to work for my first job at a big company. It was a large multinational telecommunications company, and one of the earliest concepts I have ever learned was “metering”. Up to that time, I was assuming everything one measures on a telephone switch is all about charging customers, and to understand how many minutes they have spent on the phone. Upon pouring over the code that runs on the switch and the peripherals, I came to realize the code was measuring the use of many things, and all was because of understanding the resource use of various different features, which also happens to be including the speech channels utilized.

Having a ubiquitous measuring of the resources a solution is using, that does not get in the way of the actual operations of it becomes even a more important concept, if the solution is utilizing resources from a different system, and potentially being charged for them. The urge to learn and analyze the tenants’ respective indirect utilization of Windows Azure resources the ISV is paying for the solution may have many different reasons. A few of the first that may come to mind can be:

• How can I enforce quotas for tenants, or do I have tojQuery1520412992729106918_1337901680773
• Are there any tenants consuming most of the resources?
• Do we need to introduce pricing tiers?
• How does the cost of goods for our solution get affected per tenant?
• Are certain tenants causing starvation for other tenants?
• Is my system behaving accordingly? Are we experiencing any resource usage anomalies?

The metering application block (which can be found at http://cnmb.codeplex.com), attempts to provide a solution to the ISVs asking those questions to themselves about their solution. The goal of the application block is to generate data where an ISV can compare the resource usage among the tenants.

Of course, given the wide range of pricing packages Microsoft provides, and possible changes Microsoft may take in their pricing policies, it would not be possible to calculate monetary values for those resource usages. It is also important to point out that the collected data is not the equivalent of the data Microsoft uses to calculate the bills at the end of each month. There can surely be differences in the output, since Microsoft billing has a different goal (to bill the customers) than the Cloud Ninja Metering block (to track relative usages across tenants for major resources).

What to Meter and Why

We have started with mentioning that, the CNMB is not a mechanism to analyze and double check your Azure bill details, however it is meant to provide the relative resource usage of the tenants. But what exactly do we need to measure to see the relative usage patterns? A good starting point is whatever is already there, provided by the Windows Azure page as the main pivots for a pricing discussion.

To make our point about metering not being related with the pricing, we also included a provider that measures the ingress data (data coming to the data center), which is not charged by Microsoft. The reason we chose to provide this is, heavy incoming data traffic by certain tenants may indicate additional data usage on certain other resources, such as compute time.

You can obviously start measuring many different metrics; however the block comes with 5 default providers that map to the major pivots:

• Blob size: Measured by the blob store size provider
• Database size: Measured by the database size provider
• Bandwidth : Measured by the IIS bandwidth provider, the database bandwidth provider and the blob bandwidth provider

Of course it goes without saying that depending on the solutions needs, there may be needs for measuring other aspects. Number of documents processed, active users, queue lengths could be among those.

In designing the block, we concentrated on four tenets:

• Block can be easily hosted in any hosting application, and does not depend on Azure runtime
• Does not require a code change for the related application, and be application agnostic
• Rely heavily on configuration, not coding
• Easily extensible

You can reach us at info@fullscale180.com for further questions/comments.

Have a great day!

In this post, I will explore how to create a new provider and plug it into to the CNMB framework. One of the requirements for your application to count the number of rows on a table per tenant periodically, and save that for future use, say to understand the heaviest users of the data.

To create a custom provider you will need to complete these steps:

1. Implement a class deriving from MeteringProviderBase that collects data, and publishes to the repository
2. Implement the configuration class deriving from ProviderElement 
3. Decorate the provider class with the configuration class’ name
4. Implement abstract method RegisterType on the configuration class
5. Add the provider’s configuration to your host’s configuration

For this, the first step is of course to get the copy of CNMB source from http://cnmb.codeplex.com. Then you will need to build the projects first. You will notice there will be a need to refresh some of the NuGet packages, like Reactive Extensions and Transient Fault Handling block from P&P. We have published the source with NuGet “Enable Package Restore” but there might be a need to manually refresh those.

Implementing the provider class

I will continue by adding a new Class Library project to the solution named “CloundNinja.Metering.RowCountsProvider” and reference the project “CloudNinja.Metering.Providers”. I will then get rid of the default Class.cs file, and add a new class derives from MeteringProviderBase , named  RowCountsProvider. The new class needs to implement the following abstract members of the base class:

public abstract string Name { get; }

public abstract void Initialize();

public abstract void Collect();

public abstract void Stop();

The base class, MeteringProviderBase has a constructor with the following signature that we will need to pass the objects for. These dependencies are used internally by the base class to log anything that happens, post the measurement results to for persisting them, and get the information for the context we need to meter. We will also need to reference the CloudNinja.Metering.Common project for the types on the signature.

protected MeteringProviderBase(IMeteringRepository repository, ILogger logger, IMeteringContext meteringContext)

Now, we create a constructor for our class, so we can pass those in while registering the provider’s class during the block’s bootstrap time.

We do not really have anything to initialize, or get rid of. So we simply say that the provider is initialized, and stopped using the LogInformation method on the ILogger interface. The major work to be done is at the Collectmethod.

With this provider, what we can do is to have a list of connection strings, one table each, with a table name to query, and the columns on where to get the tenant ID from and which column to count. We can populate this in the configuration, and pass that in at the constructor. Assuming the list of connection strings and related information as above is passed in, then we can query those data sources in parallel, get the results and write to the metering repository.

I am not going to describe the full implementation for the Collect method here. Please feel free to download the source at the link.

Implementing the configuration class

Rest is pretty straightforward. First create a class named TenantRowCountConnectionCollectionElement deriving from ConfigurationElement, and add the properties for ServerAlias, Connect string, table name to collect the data from, the column names where the tenant ID resides and the column to count (we do not want to do a COUNT(*)), and last but not least, a tenant Id pattern, as used throughout the block.

Next step is to create a collection deriving from ConfigurationElementCollection, and name it as TenantRowCountConnectionCollection, to contain the connection elements as defined above. And the last class is one deriving from ProviderElement which also implements the RegisterType method. Let’s name that class as TenantRowCountProviderSettingsElement.

In the RegisterType method, we will register the provider with the Unity container to get loaded by the metering block core.

As the last step as far as coding is concerned, we will need to tell the ProviderElement about which classes to load to read the rest of the configuration. This is going to be done through decorating the provider with the TenantRowCountProviderSettingsElement class’s type name.

Decorating the provider class for the configuration

We will need to tell the infrastructure, as we mentioned above about the configuration type for that particular provider. To do so, we will need to add this attribute to the provider class:

[ProviderConfigElementType("CloundNinja.Metering.RowCountsProvider.Configuration.TenantRowCountProviderSettingsElement, CloundNinja.Metering.RowCountsProvider, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]

Adding the configuration

The configuration of your host application should contain a provider definition like the following:

<Provider Type="CloundNinja.Metering.RowCountsProvider.RowCountsProvider, CloundNinja.Metering.RowCountsProvider, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null">

  <TenantRowCountMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

    <StorageAccounts>

      <addConnection ServerAlias="Storage1" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table1" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

      <addConnection ServerAlias="Storage2" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table2" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

      <addConnection ServerAlias="Storage3" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table3" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

    </StorageAccounts>

  </TenantRowCountMeteringProviderSettings>

</Provider>

You can find the sample project file in the downloads section of http://cnmb.codeplex.com

Have a great weekend!

-e