Cloud Ninja Metering Block

New Post: cloudninja.metering.pages.VendorDashboard.js file issue

ghugeramesh — Tue, 02 Apr 2013 15:28:17 GMT

I have published the clound ninja. i click on app level usage its not showing any statistics to me.

in Console it shows the following error message to me in the cloudninja.metering.pages.VendorDashboard.js file

Uncaught TypeError: Cannot read property 'Value' of null cloudninja.metering.pages.VendorDashboard.js:58
Uncaught TypeError: Cannot read property 'Value' of null cloudninja.metering.pages.VendorDashboard.js:62
Uncaught TypeError: Cannot read property 'Value' of null cloudninja.metering.pages.VendorDashboard.js:66

please help me solve the issue.

Source code checked in, #16714

Project Collection Service Accounts — Mon, 01 Oct 2012 20:27:17 GMT

Upgrade: New Version of LabDefaultTemplate.xaml. To upgrade your build definitions, please visit the following link: http://go.microsoft.com/fwlink/?LinkId=254563

Source code checked in, #16713

Project Collection Service Accounts — Mon, 01 Oct 2012 20:25:58 GMT

Checked in by server upgrade

Updated Wiki: Home

trentmswanson — Tue, 03 Jul 2012 15:41:26 GMT

The Cloud Ninja Metering Block is an extensible and reusable software component designed to assist software developers with the metering of tenant resource usage in a multi-tenant solution on the Windows Azure platform.

The need to meter and monitor resource usage at the tenant level is a common requirement for multi-tenant solutions. This metering data is commonly used to enforce quotas, set pricing, understand solution cost breakdown by tenant, generate alerts, or identify tenants which may be abusing the solution. Regardless of how this data is used, a core requirement is to be able to capture and store this data in a form that it can be utilized for the purposes.

Metering Block
The metering block ships as a hostable runtime component that can be deployed and hosted in an on-premise windows services, console application; or to Windows Azure either in it's own role or as part of another role for an existing solution. The metering block comes with a number of providers for collecting metering data from existing logs, and can be extended with custom providers for collecting metering data from other sources, such as custom logs, views, etc...

Metering Providers

Database Size
Database Bandwidth
W3C Logs (Bandwidth / Requests)
Storage Size
Storage Bandwidth

Repository
The Cloud Ninja metering block utilizes a storage provider implementation for writing metering data to a metering repository. The current metering block ships with an OData storage repository provider for writing the metering data to an OData service secured with HMAC authentication scheme. The solution also includes a data service project that exposes an OData metering service to the metering block and utilizes SQL Azure as the persistent store. Additional storage providers can be implemented to support peristence of metering data to other repositories.

NuGet makes is easy to install the prerequisites for the metering block. In order to build the metering block and run the OData repository and dashboard application you will need the following:

A development machine running Microsoft Visual Studio 2010 development system SP1
All required Microsoft Windows updates
Windows Azure SDK for .NET and Windows Azure Tools for Microsoft Visual Studio – November 2011 Release (1.6)
NuGet Package Manager
A Windows Azure subscription with room for at least one hosted service
A Windows Azure storage account
A SQL Azure database server
DevLabs: Code Contracts
Microsoft® SQL Server® 2012 PowerPivot® for Microsoft® Excel® 2010

Tenant View

Powerpivot Dashboard

Tenant List View (Sorted by Total Weighted Usage)

Bhushan Nene
Kashif Alam
Ercenk Keresteci
Trent Swanson

Full Scale 180 - Metering News Feed

Why Anybody Wants to Meter a Multitenant Application?

Thursday, May 24, 2012 | From Full Scale 180 - Metering

I remember the times when I started to work for my first job at a big company. It was a large multinational telecommunications company, and one of the earliest concepts I have ever learned was “metering”. Up to that time, I was assuming everything one measures on a telephone switch is all about charging customers, and to understand how many minutes they have spent on the phone. Upon pouring over the code that runs on the switch and the peripherals, I came to realize the code was measuring the use of many things, and all was because of understanding the resource use of various different features, which also happens to be including the speech channels utilized.

Having a ubiquitous measuring of the resources a solution is using, that does not get in the way of the actual operations of it becomes even a more important concept, if the solution is utilizing resources from a different system, and potentially being charged for them. The urge to learn and analyze the tenants’ respective indirect utilization of Windows Azure resources the ISV is paying for the solution may have many different reasons. A few of the first that may come to mind can be:

How can I enforce quotas for tenants, or do I have tojQuery1520412992729106918_1337901680773
Are there any tenants consuming most of the resources?
Do we need to introduce pricing tiers?
How does the cost of goods for our solution get affected per tenant?
Are certain tenants causing starvation for other tenants?
Is my system behaving accordingly? Are we experiencing any resource usage anomalies?

The metering application block (which can be found at http://cnmb.codeplex.com), attempts to provide a solution to the ISVs asking those questions to themselves about their solution. The goal of the application block is to generate data where an ISV can compare the resource usage among the tenants.

Of course, given the wide range of pricing packages Microsoft provides, and possible changes Microsoft may take in their pricing policies, it would not be possible to calculate monetary values for those resource usages. It is also important to point out that the collected data is not the equivalent of the data Microsoft uses to calculate the bills at the end of each month. There can surely be differences in the output, since Microsoft billing has a different goal (to bill the customers) than the Cloud Ninja Metering block (to track relative usages across tenants for major resources).

What to Meter and Why

We have started with mentioning that, the CNMB is not a mechanism to analyze and double check your Azure bill details, however it is meant to provide the relative resource usage of the tenants. But what exactly do we need to measure to see the relative usage patterns? A good starting point is whatever is already there, provided by the Windows Azure page as the main pivots for a pricing discussion.

To make our point about metering not being related with the pricing, we also included a provider that measures the ingress data (data coming to the data center), which is not charged by Microsoft. The reason we chose to provide this is, heavy incoming data traffic by certain tenants may indicate additional data usage on certain other resources, such as compute time.

You can obviously start measuring many different metrics; however the block comes with 5 default providers that map to the major pivots:

Blob size: Measured by the blob store size provider
Database size: Measured by the database size provider
Bandwidth : Measured by the IIS bandwidth provider, the database bandwidth provider and the blob bandwidth provider

Of course it goes without saying that depending on the solutions needs, there may be needs for measuring other aspects. Number of documents processed, active users, queue lengths could be among those.

In designing the block, we concentrated on four tenets:

Block can be easily hosted in any hosting application, and does not depend on Azure runtime
Does not require a code change for the related application, and be application agnostic
Rely heavily on configuration, not coding
Easily extensible

You can reach us at info@fullscale180.com for further questions/comments.

Have a great day!

Implementing a Custom Provider for Cloud Ninja Metering Block

Sunday, May 20, 2012 | From Full Scale 180 - Metering

In this post, I will explore how to create a new provider and plug it into to the CNMB framework. One of the requirements for your application to count the number of rows on a table per tenant periodically, and save that for future use, say to understand the heaviest users of the data.

To create a custom provider you will need to complete these steps:

Implement a class deriving from MeteringProviderBase that collects data, and publishes to the repository
Implement the configuration class deriving from ProviderElement
Decorate the provider class with the configuration class’ name
Implement abstract method RegisterType on the configuration class
Add the provider’s configuration to your host’s configuration

For this, the first step is of course to get the copy of CNMB source from http://cnmb.codeplex.com. Then you will need to build the projects first. You will notice there will be a need to refresh some of the NuGet packages, like Reactive Extensions and Transient Fault Handling block from P&P. We have published the source with NuGet “Enable Package Restore” but there might be a need to manually refresh those.

Implementing the provider class

I will continue by adding a new Class Library project to the solution named “CloundNinja.Metering.RowCountsProvider” and reference the project “CloudNinja.Metering.Providers”. I will then get rid of the default Class.cs file, and add a new class derives from MeteringProviderBase , named RowCountsProvider. The new class needs to implement the following abstract members of the base class:

public abstract string Name { get; }

public abstract void Initialize();

public abstract void Collect();

public abstract void Stop();

The base class, MeteringProviderBase has a constructor with the following signature that we will need to pass the objects for. These dependencies are used internally by the base class to log anything that happens, post the measurement results to for persisting them, and get the information for the context we need to meter. We will also need to reference the CloudNinja.Metering.Common project for the types on the signature.

protected MeteringProviderBase(IMeteringRepository repository, ILogger logger, IMeteringContext meteringContext)

Now, we create a constructor for our class, so we can pass those in while registering the provider’s class during the block’s bootstrap time.

We do not really have anything to initialize, or get rid of. So we simply say that the provider is initialized, and stopped using the LogInformation method on the ILogger interface. The major work to be done is at the Collectmethod.

With this provider, what we can do is to have a list of connection strings, one table each, with a table name to query, and the columns on where to get the tenant ID from and which column to count. We can populate this in the configuration, and pass that in at the constructor. Assuming the list of connection strings and related information as above is passed in, then we can query those data sources in parallel, get the results and write to the metering repository.

I am not going to describe the full implementation for the Collect method here. Please feel free to download the source at the link.

Implementing the configuration class

Rest is pretty straightforward. First create a class named TenantRowCountConnectionCollectionElement deriving from ConfigurationElement, and add the properties for ServerAlias, Connect string, table name to collect the data from, the column names where the tenant ID resides and the column to count (we do not want to do a COUNT(*)), and last but not least, a tenant Id pattern, as used throughout the block.

Next step is to create a collection deriving from ConfigurationElementCollection, and name it as TenantRowCountConnectionCollection, to contain the connection elements as defined above. And the last class is one deriving from ProviderElement which also implements the RegisterType method. Let’s name that class as TenantRowCountProviderSettingsElement.

In the RegisterType method, we will register the provider with the Unity container to get loaded by the metering block core.

As the last step as far as coding is concerned, we will need to tell the ProviderElement about which classes to load to read the rest of the configuration. This is going to be done through decorating the provider with the TenantRowCountProviderSettingsElement class’s type name.

Decorating the provider class for the configuration

We will need to tell the infrastructure, as we mentioned above about the configuration type for that particular provider. To do so, we will need to add this attribute to the provider class:

[ProviderConfigElementType("CloundNinja.Metering.RowCountsProvider.Configuration.TenantRowCountProviderSettingsElement, CloundNinja.Metering.RowCountsProvider, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]

Adding the configuration

The configuration of your host application should contain a provider definition like the following:

<Provider Type="CloundNinja.Metering.RowCountsProvider.RowCountsProvider, CloundNinja.Metering.RowCountsProvider, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null">

  <TenantRowCountMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

    <StorageAccounts>

      <addConnection ServerAlias="Storage1" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table1" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

      <addConnection ServerAlias="Storage2" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table2" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

      <addConnection ServerAlias="Storage3" ConnectionString="Data Source=.\sqlexpress;Initial Catalog=meteringtest;Integrated Security=SSPI;"

        TableName="table3" TenantIdColumn="tenantid" CountColumn="value"

        TenantIdPattern="tn_{.+}" />

    </StorageAccounts>

  </TenantRowCountMeteringProviderSettings>

</Provider>

You can find the sample project file in the downloads section of http://cnmb.codeplex.com

Have a great weekend!

-e

Metering block internals

Friday, May 18, 2012 | From Full Scale 180 - Metering

Greetings all. In this post I want to get a little bit deeper into the block’s data collection components’ implementation.

At the core of the metering block are two main concepts, the TickTock component and the providers. The TickTock component is a concept we borrowed from the Patterns and Practices Microsoft Enterprise Library 5.0 Integration Pack for Windows Azure, The Autoscaling Application Block which we have also contributed. The heart of the Autoscaling block is a sub-component referred as “Metronome”, we are using a similar concept, with a few changes in the Metering core as well and named it as “TickTock”.

After the block is started the Start() method gets hold of the custom configuration section. This results in getting the configuration object graph built by the .Net framework. The mandatory CreateNewElement() method on the ProvidersCollection class, derived from the ConfigurationElementCollection is the one that is responsible for loading the appropriate provider configuration type into the AppDomain with the typical GetType(), Activator.CreateInstance() pattern. The provider’s class should carry a custom attribute which specifies the type for the configuration element for the provider. Using this pattern and getting the type for the configuration element through the custom attribute, this method for the collection knows which collection element class, thus the appropriate element for the provider needs to be loaded in and activated.

At this point we will have the whole configuration section loaded, with the appropriate provider types loaded into the AppDomain.

The metering block Start() method calls the Initialize() method and this in turn continues with calling the RegisterType() method of the providers that implement the ITypeRegistration interface. Each provider configuration element, that derives from ProviderElement base class implements that interface, and is responsible for registering the respective provider type with the Unity framework.

Having completed loading in the provider types, and getting them registered with Unity, next step Initialize() does is to register the Collect() methods of the provider with the TickTock class, with their collect intervals, so TickTock can call the Collect() methods at those intervals. We utilize Reactive Extensions for the TickTock class to call the Collect() methods of the providers.

Full Scale 180 - Metering News Feed

Updated Release: v1 Release (May 19, 2012)

ercenk — Sun, 20 May 2012 21:16:39 GMT

First edition of the Cloud Ninja Metering Block.

It looks like the Nuget package restore is not working automatically for some installations for Reactive Extensions and Unity. Workaround is to uninstall the related packages in the projects manually and re-install them again.

Sample implementation of a provider is also included, with detailed steps for how to create a custom provider, posted at http://www.fullscale180.com/Blog/post/2012/05/19/Implementing-a-Custom-Provider-for-Cloud-Ninja-Metering-Block.aspx.

Released: v1 Release (May 19, 2012)

Sun, 20 May 2012 21:16:39 GMT

Updated Release: v1 Release (May 19, 2012)

ercenk — Sun, 20 May 2012 21:15:50 GMT

Updated Release: v1 Release (May 19, 2012)

ercenk — Sun, 20 May 2012 01:51:56 GMT

First edition of the Cloud Ninja Metering Block.

It looks like the Nuget package restore is not working automatically for some installations for Reactive Extensions and Unity. Workaround is to uninstall the related packages in the projects manually and re-install them again.

Sample implementation of a provider is uploaded, with steps posted at http://www.fullscale180.com/Blog/post/2012/05/19/Implementing-a-Custom-Provider-for-Cloud-Ninja-Metering-Block.aspx.

Released: v1 Release (May 19, 2012)

Sun, 20 May 2012 01:51:56 GMT

First edition of the Cloud Ninja Metering Block.

It looks like the Nuget package restore is not working automatically for some installations for Reactive Extensions and Unity. Workaround is to uninstall the related packages in the projects manually and re-install them again.

Sample implementation of a provider is uploaded, with steps posted at http://www.fullscale180.com/Blog/post/2012/05/19/Implementing-a-Custom-Provider-for-Cloud-Ninja-Metering-Block.aspx.

Updated Release: v1 Release (May 19, 2012)

ercenk — Sun, 20 May 2012 01:51:32 GMT

First edition of the Cloud Ninja Metering Block.

It looks like the Nuget package restore is not working automatically for some installations for Reactive Extensions and Unity. Workaround is to uninstall the related packages in the projects manually and re-install them again.

Sample implementation of a provider is uploaded, with steps posted at http://www.fullscale180.com/Blog/post/2012/05/19/Implementing-a-Custom-Provider-for-Cloud-Ninja-Metering-Block.aspx.

Updated Release: v1 Release (May 19, 2012)

ercenk — Sun, 20 May 2012 01:50:01 GMT

Updated Wiki: Home

ercenk — Fri, 18 May 2012 08:05:23 GMT

The Cloud Ninja Metering Block is an extensible and reusable software component designed to assist software developers with the metering of tenant resource usage in a multi-tenant solution on the Windows Azure platform.

The need to meter and monitor resource usage at the tenant level is a common requirement for multi-tenant solutions. This metering data is commonly used to enforce quotas, set pricing, understand solution cost breakdown by tenant, generate alerts, or identify tenants which may be abusing the solution. Regardless of how this data is used, a core requirement is to be able to capture and store this data in a form that it can be utilized for the purposes.

You can view a running demo deployment, which is currently metering THE cloud ninja - multi-tenant application here:
http://cnmb.cloudapp.net

Metering Block
The metering block ships as a hostable runtime component that can be deployed and hosted in an on-premise windows services, console application; or to Windows Azure either in it's own role or as part of another role for an existing solution. The metering block comes with a number of providers for collecting metering data from existing logs, and can be extended with custom providers for collecting metering data from other sources, such as custom logs, views, etc...

Metering Providers

Database Size
Database Bandwidth
W3C Logs (Bandwidth / Requests)
Storage Size
Storage Bandwidth

A development machine running Microsoft Visual Studio 2010 development system SP1
All required Microsoft Windows updates
Windows Azure SDK for .NET and Windows Azure Tools for Microsoft Visual Studio – November 2011 Release (1.6)
NuGet Package Manager
A Windows Azure subscription with room for at least one hosted service
A Windows Azure storage account
A SQL Azure database server
DevLabs: Code Contracts
Microsoft® SQL Server® 2012 PowerPivot® for Microsoft® Excel® 2010

Tenant View

Powerpivot Dashboard

Tenant List View (Sorted by Total Weighted Usage)

Bhushan Nene
Kashif Alam
Ercenk Keresteci
Trent Swanson

Full Scale 180 - Metering News Feed

Configuring CloudNinja Metering Block

Friday, May 18, 2012 | From Full Scale 180 - Metering

Greetings to all. We have recently published yet another exciting project to the Azure community at http://cnmb.codeplex.com at the end of the previous month. We will try to add more details on its usage as we find time here on our blog. This post’s topic is on configuring the block host.

The first decision is where to host the block. You can have a multitude of different combinations on the location, such as a Windows Service, an Azure worker role, or a web role. For the sake of simplicity, I will explain how you can configure to run it in a web role. We will continue this topic on a different post soon on how to continue this on the same role for the publishing part of the block.

Starting with Azure SDK 1.3, the web roles allow you to access the full IIS functionality. At the most basic level, the side effect of this is you will need to think about having two different hosts running your web role code. One host is the w3wp.exe process running your site’s code, and the other being the WaIISHost.exe process, hosting your RoleEntryPoint (which happens to be your WebRole class in the WebRole.cs file, if you have started with the Azure Web Role template in VS 2010).

Since our code is living in the .Net universe, each process, thus AppDomain need to grab their own respective .config files. Thus, there is a need to create a new file in your project called WaIISHost.exe.config, along your customary web.config.

Hmm, the team here looks like working on some new enhancements…

Anyhow, once you create the new file, you will need to think about 3 new points:

1. Add the reference for the custom configuration sections.

2. The common configuration for the block.

3. The provider configurations

Let’s start with the configuration section references.

We do use the Patterns and Practices’ Unity Application Block (http://unity.codeplex.com/), so the Unity configuration sections, and also the Transient Fault Handling Application Block from Enterprise Library 5.0 Integration Pack for Windows Azure. So you will need to add those, along with the Metering block’s custom section reference. An example will look like this:

 <configSections>

    <section name="MeteringConfiguration" type="CloudNinja.Metering.Common.Configuration.MeteringSection, CloudNinja.Metering.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"

             allowLocation="true" allowDefinition="Everywhere" allowExeDefinition="MachineToApplication" overrideModeDefault="Allow" restartOnExternalChanges="true"

             requirePermission="true" />

    <section name="typeRegistrationProvidersConfiguration" type="Microsoft.Practices.EnterpriseLibrary.Common.Configuration.TypeRegistrationProvidersConfigurationSection, Microsoft.Practices.EnterpriseLibrary.Common, Version=5.0.505.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />

    <section name="RetryPolicyConfiguration" type="Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.Configuration.RetryPolicyConfigurationSettings, Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling, Version=5.0.1118.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="true" />

  </configSections>

Following this will be the common configuration for the block within the custom configuration section marked between the tags:

  <MeteringConfiguration>

  </MeteringConfiguration>

The block uses a blob lease to determine the master for a given hosting environment who will collect the data and write to the repository. This method is required if you happen to be hosting the block in a role which runs in multiple instances. The BlockConnectionString attribute contains the Azure storage account connection string that contains this. The rest is automatic, the block creates its own container and the blob to put the leases on.

You can also specify a number of connection strings both for Azure storage accounts as well as SQL Azure connections if you want to consolidate all of your connection strings in one place and refer to them in the rest of the configuration section.

An example for this will be:

   <CommonSettings BlockConnectionString="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==">

      <ConnectionStrings>

        <clearConnectionStrings />

        <addConnectionString Name="Common" Value="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==" />

      </ConnectionStrings>

    </CommonSettings>

Now it is time to come to the providers’ configuration. Please note that you can specify a default collect interval for all of the providers here, but each provider can override this value at their own respective configuration elements. We support 5 providers out of the box. They are:

1. Database size provider

2. Database bandwidth provider

3. IIS bandwidth provider

4. Blob size provider

5. Blog bandwidth provider

They all have a reference to the type that implements the provider as the Type attribute at the Provider element, and also a CollectInterval attribute. And each provider has it’s own particular configuration after that. Most of it after that almost repeats, but there are two important points about this.

1. Bookmark support for parsing logs

2. Tenant identifier pattern

We support keeping a bookmark of the last parsed log for both IIS logs and the blob storage analytics logs. So we need a repository connection string for both IISLogsParserBookmarkRepository and BlobBandwidthLogsParserBookmarkRepository attributes. They may point to the same storage account. The providers automatically create a table on those accounts where they start marking the last log file and the line they parsed.

The other point is telling the providers on how to identify a tenant ID. The attribute DefaultTenantIdPattern as the default setting for the provider and TenantIdPattern connection location specific one both have the same template. It is a BEGIN_REGEX{MIDDLE_REGEX}END_REGEX template, where the start, middle and end of the tenant ID patter are marked with the curly braces. You can basically put in any regex here, as long as it conforms to the expected patterns by the resources the providers are measuring.

There are also a couple of details for the IIS Log provider and the blob provider. The first one supports the location of the tenant identifier. The values can be UriStem, Hostheader and UriQuery. Those will translate to the column names on the IIS logs, as defined by the W3C standar column names. The default IIS log configuration on web roles, when collected through the Windows Azure Diagnostics (WAD) subsystem. The next one, about the blob size provider is to tell the provider where to find the blobs as the base container. It is provided by the TenantsBaseContainerPrefix attribute on the blob size provider.

Rest is almost repeating the same pattern. Specify a connection string, either explicitly, or refer to one of the common connection strings by their name. Following is an example how you can specify the settings for the various providers.

    <Providers DefaultCollectionInterval="03:00:00">

      <Providers>

        <Provider Type="CloudNinja.Metering.Providers.WebRole.IISLogProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <IISLogMeteringProviderSettings IISLogsRepositoryConnStr="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ=="

              IISLogsParserBookmarkRepository="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg=="

              TenantIdentifierLocation="hostheader" TenantIdentifierPattern="{.*}.thecloudninja.com" />

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.SQLAzure.DatabaseSizeProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <DatabaseSizeMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

            <DatabaseServers>

              <clearConnectionStrings />

              <addConnectionString Name="Server1" Value="Persist Security Info=False;Integrated Security=false;server=ors6ecpxj2.database.windows.net;UID=AzureSaas@ors6ecpxj2;PWD=Bloomington!"

                  TenantIdPattern="tnt_{.+}" />

            </DatabaseServers>

          </DatabaseSizeMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.AzureBlob.BlobSizeProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <BlobSizeMeteringProviderSettings DefaultTenantIdPattern="tnt-{.+}">

          <StorageAccounts>

            <clearStorageAccount />

            <addStorageAccount Name="cn-tenantstore" TenantsBaseContainerPrefix="" ConnectionString="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ==" TenantIdPattern="tnt[ps]-{.+}"/>

          </StorageAccounts>

          </BlobSizeMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.SQLAzure.DatabaseBandwidthUsageProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <DatabaseBandwidthMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

            <DatabaseServers>

              <clearConnectionStrings />

              <addConnectionString Name="Server1" Value="Persist Security Info=False;Integrated Security=false;server=ors6ecpxj2.database.windows.net;UID=AzureSaas@ors6ecpxj2;PWD=Bloomington!" TenantIdPattern="tnt_{.+}" />

            </DatabaseServers>

          </DatabaseBandwidthMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.AzureBlob.BlobBandwidthProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"  CollectInterval="01:00:00" >

          <BlobBandwidthMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}" BlobBandwidthLogsParserBookmarkRepository="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==">

            <StorageAccounts>

              <clearStorageAccount />

              <addStorageAccount Name="cn-tenantstore" ConnectionString="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ==" TenantIdPattern="tnt[ps]-{.+}" />

            </StorageAccounts>

          </BlobBandwidthMeteringProviderSettings>

        </Provider>

      </Providers>

    </Providers>

Full Scale 180 - Metering News Feed

Updated Wiki: Home

ercenk — Fri, 18 May 2012 08:04:02 GMT

The Cloud Ninja Metering Block is an extensible and reusable software component designed to assist software developers with the metering of tenant resource usage in a multi-tenant solution on the Windows Azure platform.

The need to meter and monitor resource usage at the tenant level is a common requirement for multi-tenant solutions. This metering data is commonly used to enforce quotas, set pricing, understand solution cost breakdown by tenant, generate alerts, or identify tenants which may be abusing the solution. Regardless of how this data is used, a core requirement is to be able to capture and store this data in a form that it can be utilized for the purposes.

You can view a running demo deployment, which is currently metering THE cloud ninja - multi-tenant application here:
http://cnmb.cloudapp.net

Metering Block
The metering block ships as a hostable runtime component that can be deployed and hosted in an on-premise windows services, console application; or to Windows Azure either in it's own role or as part of another role for an existing solution. The metering block comes with a number of providers for collecting metering data from existing logs, and can be extended with custom providers for collecting metering data from other sources, such as custom logs, views, etc...

Metering Providers

Database Size
Database Bandwidth
W3C Logs (Bandwidth / Requests)
Storage Size
Storage Bandwidth

A development machine running Microsoft Visual Studio 2010 development system SP1
All required Microsoft Windows updates
Windows Azure SDK for .NET and Windows Azure Tools for Microsoft Visual Studio – November 2011 Release (1.6)
NuGet Package Manager
A Windows Azure subscription with room for at least one hosted service
A Windows Azure storage account
A SQL Azure database server
DevLabs: Code Contracts
Microsoft® SQL Server® 2012 PowerPivot® for Microsoft® Excel® 2010

Tenant View

Powerpivot Dashboard

Tenant List View (Sorted by Total Weighted Usage)

Bhushan Nene
Kashif Alam
Ercenk Keresteci
Trent Swanson

Full Scale 180 - Metering News Feed

Configuring CloudNinja Metering Block

Friday, May 18, 2012 | From Full Scale 180 - Metering

Hmm, the team here looks like working on some new enhancements…

Anyhow, once you create the new file, you will need to think about 3 new points:

1. Add the reference for the custom configuration sections.

2. The common configuration for the block.

3. The provider configurations

Let’s start with the configuration section references.

 <configSections>

    <section name="MeteringConfiguration" type="CloudNinja.Metering.Common.Configuration.MeteringSection, CloudNinja.Metering.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"

             allowLocation="true" allowDefinition="Everywhere" allowExeDefinition="MachineToApplication" overrideModeDefault="Allow" restartOnExternalChanges="true"

             requirePermission="true" />

    <section name="typeRegistrationProvidersConfiguration" type="Microsoft.Practices.EnterpriseLibrary.Common.Configuration.TypeRegistrationProvidersConfigurationSection, Microsoft.Practices.EnterpriseLibrary.Common, Version=5.0.505.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />

    <section name="RetryPolicyConfiguration" type="Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.Configuration.RetryPolicyConfigurationSettings, Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling, Version=5.0.1118.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="true" />

  </configSections>

Following this will be the common configuration for the block within the custom configuration section marked between the tags:

  <MeteringConfiguration>

  </MeteringConfiguration>

An example for this will be:

   <CommonSettings BlockConnectionString="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==">

      <ConnectionStrings>

        <clearConnectionStrings />

        <addConnectionString Name="Common" Value="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==" />

      </ConnectionStrings>

    </CommonSettings>

1. Database size provider

2. Database bandwidth provider

3. IIS bandwidth provider

4. Blob size provider

5. Blog bandwidth provider

1. Bookmark support for parsing logs

2. Tenant identifier pattern

    <Providers DefaultCollectionInterval="03:00:00">

      <Providers>

        <Provider Type="CloudNinja.Metering.Providers.WebRole.IISLogProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <IISLogMeteringProviderSettings IISLogsRepositoryConnStr="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ=="

              IISLogsParserBookmarkRepository="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg=="

              TenantIdentifierLocation="hostheader" TenantIdentifierPattern="{.*}.thecloudninja.com" />

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.SQLAzure.DatabaseSizeProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <DatabaseSizeMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

            <DatabaseServers>

              <clearConnectionStrings />

              <addConnectionString Name="Server1" Value="Persist Security Info=False;Integrated Security=false;server=ors6ecpxj2.database.windows.net;UID=AzureSaas@ors6ecpxj2;PWD=Bloomington!"

                  TenantIdPattern="tnt_{.+}" />

            </DatabaseServers>

          </DatabaseSizeMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.AzureBlob.BlobSizeProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <BlobSizeMeteringProviderSettings DefaultTenantIdPattern="tnt-{.+}">

          <StorageAccounts>

            <clearStorageAccount />

            <addStorageAccount Name="cn-tenantstore" TenantsBaseContainerPrefix="" ConnectionString="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ==" TenantIdPattern="tnt[ps]-{.+}"/>

          </StorageAccounts>

          </BlobSizeMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.SQLAzure.DatabaseBandwidthUsageProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" CollectInterval="01:00:00">

          <DatabaseBandwidthMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}">

            <DatabaseServers>

              <clearConnectionStrings />

              <addConnectionString Name="Server1" Value="Persist Security Info=False;Integrated Security=false;server=ors6ecpxj2.database.windows.net;UID=AzureSaas@ors6ecpxj2;PWD=Bloomington!" TenantIdPattern="tnt_{.+}" />

            </DatabaseServers>

          </DatabaseBandwidthMeteringProviderSettings>

        </Provider>

        <Provider Type="CloudNinja.Metering.Providers.AzureBlob.BlobBandwidthProvider, CloudNinja.Metering.Providers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"  CollectInterval="01:00:00" >

          <BlobBandwidthMeteringProviderSettings DefaultTenantIdPattern="tnt_{.+}" BlobBandwidthLogsParserBookmarkRepository="DefaultEndpointsProtocol=https;AccountName=cnm;AccountKey=YfWiIGe/BQtLZzMOUtdkth7KLKyyUlVYl83vvCSptYHJgCVcP37i03/JIOQvYblwZlmp/a0fVXT5nGi9jQzuIg==">

            <StorageAccounts>

              <clearStorageAccount />

              <addStorageAccount Name="cn-tenantstore" ConnectionString="DefaultEndpointsProtocol=https;AccountName=friday;AccountKey=q0JAToIm1qt2SnxfedcHjy6TxSPBSNYv5oVrhCwFTVacaXQ6/8d7iI34ETHk0FwBeKCz6HETlW4YjWzmg2R6lQ==" TenantIdPattern="tnt[ps]-{.+}" />

            </StorageAccounts>

          </BlobBandwidthMeteringProviderSettings>

        </Provider>

      </Providers>

    </Providers>

Full Scale 180 - Metering News Feed

New Post: Installation Guide or Documentation

kjarriv — Tue, 15 May 2012 09:59:15 GMT

Is there any plan to publish a installation guide or documentation?

Updated Release: v1 Release (Apr 28, 2012)

ercenk — Mon, 07 May 2012 04:26:42 GMT

Released: v1 Release (Apr 28, 2012)

Mon, 07 May 2012 04:26:42 GMT

Updated Release: v1 Release (Apr 28, 2012)

ercenk — Fri, 04 May 2012 07:35:41 GMT

Released: v1 Release (Apr 28, 2012)

Fri, 04 May 2012 07:35:41 GMT

Released: v1 Release (Apr 28, 2012)

Fri, 04 May 2012 07:07:30 GMT

Updated Release: v1 Release (Apr 28, 2012)

ercenk — Fri, 04 May 2012 07:07:30 GMT