Azure Diagnostic Logging


Enabling Diagnostics in Azure Cloud Services and Virtual Machines

Get logging in Windows Azure with Enterprise Library

Microsoft Azure Diagnostics Part 1: Introduction

Bare metal logging – Trace with AzureDiagnosticsTraceFilter

Following the principle of “do the simplest possible thing that works” for diagnostics is to just instrument you application using .NET’s System.Diagnostic.Trace feature. Just use Trace.WriteLine to log information as your application is executing, for example:

Trace.WriteLine(“Trace information goes here”, “Category goes here”);

The category information is optional but can help provide context or filtering information

For an application running in Azure you can add the following to your web.config or app.config to cause your trace information to be written to a an Azure Table:




type=Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35


type=“” />





This will cause all of the trace information to be written to an Azure Table called WADLogsTable. The storage account that gets used is configured as part of the diagnostic configuration.

This approach is the absolute easiest and fastest way to instrument your application for diagnostics. It’s no frills but done properly it is extremely effective. With the addition of the Azure trace listener you have the convenience of being able to review the logging information in a table. It would be really easy to write a tool to make reading the logs easier.

Enterprise Library – Custom Trace Listener for Azure Table Storage

While just using the Diagnostic Trace object will get the job done most applications benefit from something a little more sophisticated. This is where libraries like Log4Net, NLog and Logging Application Block (refered to as LAB from here on) come in to the picture.

My practice is to use the Enterprise Library in my projects so logging is performed through LAB. However, at this time LAB does not have a built in way to write messages to Azure Storage. There are some tricks involving configuring Azure Diagnostics local resources that are supposed to copy the text files in to Azure Blob storage, however I’ve not been able to make them work. Rather than continuing to hack my way through the mystery it was actually easier to just write my own TraceListener that writes what I want to Azure Table storage in the way I want it written.

In order to create the trace listener you first have to get the LAB from NUGet. Create a class library project and add references to the LAB assemblies (see the sample). You’ll also need to get the Windows Azure Storage package from NuGet and add references to it.

Inside your class library project create a class that inherits from CustomTraceListener and override TraceData, Write, and WriteLine.

You’ll need to create an entity that inherits from Azure Storage’s TableEntity. The benefit of this is that you can shape your log table to look anyway you want. The TableEntry object I created has a subset of the properties from LAB’s LogEntry object. In the constructor I set the PartitionKey and RowKey using a date time string.

For the trace listener to work it will need to be configured with information about the storage account, account key and the name of the table to write entries to. This is done as part of the TraceListener’s configuration. You can either use the EntLib configuration utility or just hand write the configuration yourself. The resulting configuration information looks like this:


<add ListenerDataType=”Microsoft….CustomTraceListenerData, Microsoft….Logging”

AccountName=”account goes here” AccountKey=”[key goes here”


type=”GuerillaProgrammer….AzureTableTraceListener, GuerillaProgrammer.AzureTableListener”

name=”AzureTableTraceListener” />


Once you have your TraceListener put together and added to your project you can then write a tool to view the log in whatever way you want. Because the partition key cuts off by the day you easily filter the log to view just a day at a time if you wish.

Inside you application you simply use LAB as usual. In fact if you have existing application that use LAB you can just throw in the new trace listener and start benefiting from it.

The sample code can be downloaded here. The packages will have to be restored from NuGet before it can be built.

Where to next?

What I’ve outlined here is very simple logging. I did not address strategies or anything else. Just plain old – I need to know what my application is doing logging. Nothing else. At this time I’m looking at the Semantic Logging Application Block to see what it offers. I’m also exploring ways to introduce Aspect Orient Programming (AOP) in to existing applications in order to add logging aspects without having to make major refactoring changing to them. New applications may be worth starting off with AOP in place from the very begging, but I need to look at how well Unity supports AOP.

Configuration Notes


The intent of this document is to lay out some very simple practices and some basic knowledge to make application configuration as straight forward as possible.  The outcome of this knowledge is that you will be able to successfully build your application for different environments without having to edit the web.config file for each build.

Only the most trivial applications avoid the need for configuration files.  Almost every application I have dealt with has the need for some type of configuration.  In the normal course of development I’ll need to be able to test out my application while developing it and the testers will need a different database to do their job.  To do this I use a configuration file.  This is the simplest case but the need for configuration gets more complex as you add environments, features, and the need to integrate with other systems.


The typical custom application development project will have five environments that will be used to control the release of the finished work product.  The names of the environments can change depending upon the particular customer but the following are common environment names.


This is your workstation or laptop where all of your work is done.  The major difference between all of the other environments is that this should be the only environment that has Visual Studio installed.  This is also the environment that the application should be able to immediately run in as soon as a new person gets the source code from whatever repository is being used by the project (TFS, subversion, etc.) and presses F5 to run the application in the debugger.


This is an environment where everybody’s code comes together and is built and deployed so it can be tested by the developers in an environment similar to what will be seen in production.  While Visual Studio shouldn’t be installed here it often ends up here so debugging can be performed.


This is the QA environment where the product is built and deployed so that QA testers can perform complete functional testing in an environment that duplicates production.  Visual Studio should absolutely not be installed anywhere in this environment.  Developers do not have access to this environment, deployment is done either via an automated script, a QA administrator, or a build master.


This is essentially a production environment.  The product is deployed here for final acceptance and testing before the product goes live.  Visual Studio is not here and developers are not permitted access to this environment*.  The deployment to this environment is done through automated scripts or performed by system administrators.


This is the live environment that end users access.    The deployment to this environment is  through automated scripts or performed by system administrators.   It should be obvious that developers are absolutely not permitted in this environment.*

*In reality developers end up with access to these environments but I personally think they are foolish accepting this access.  It only transfers risk from the administrators who are responsible for operating the environment to the developer.

Build Configurations

Inside your Visual Studio project you will need to create a build configuration for each of the above environments.  By default when a new solution is created Visual Studio automatically creates Debug and Release build configurations.  Luckily you can rename them and add additional configurations as needed.

When you create a new build configuration you can add a configuration transform to any web project in the solution.  Visual Studio will do this through the project’s context menu.  For instance an ASP.NET MVC project by default will have a web.config file along with a web.debug.config and web.release.config.  If you create a new build configuration called “Staging” you can add web.Staging.config to the project by right clicking on the project in solution explorer and selecting “Add Config Transforms”.  Visual Studio will add a skeleton transform file to your project for you.

add config transform


The self-hosted environment should be in the base configuration file (web.config) and should have the correct settings for a new developer to get going without needing to make any changes.

Everybody on the team should have the same settings and the web.config file should not need to be changed unless the change affects everybody on the team.  Nobody should need to remember to exclude web.config from the change set because they have a different database connection string or something else equally silly.

All of the other environments should have their own configuration transform and should be setup so that developers do not need to make any changes when they perform a build for that environment.

Configuration Transform

The configuration transform allows you to change the configurations depending upon the build being performed.  Unfortunately without hacking Visual Studion and MSBuild only perform transforms for web.config.  There are solutions to transform app.config but they are outside the scope of this article.

The configuration transform capability in Visual Studio offers a wide range of capabilities.  Below are the capabilities that are most commonly used.  For more information on Configuration Transforms refer to this article: Web.config Transformation Syntax for Web Project Deployment Using Visual Studio

For information on how to do transformations for other configuration files refer to this Code Project article: Transform app.config and web.config


Replaces the selected element with the element in the transform file.  For example, we can turn off debug output for the compilation tag in our web.config using the following transform


<compilation debug=”false” tragetFramework=”4.5” xdt:Transform=”Replace” />


Notice that there is no locator specified.  In this case it is not needed.  However, in app.setting or other sections where you need to target a specific setting in a dictionary you’ll need to specify a locator.  In this case I’ll use the Condition locator which uses an XPath expression.  Here is an example for appSettings:


<add key=”ErrorMode” value=”” xdt:Transform=”Replace” xdt:Locator=”Condition(@key=’ErrorMode’)” />


Generally the Condition locator is the only one you’ll need, however there are other locators you can use (see the MSDN documentation).


Insert jams in the element from the transform.  This is useful for turning on authentication in production for instance.  A simple example:



<deny users=”?” xdt:Transform=”Insert” />



Visual Studio Support

Visual Studio will help you test out your configuration transform without having to build your solution for each build type.  To see the results of your transform:

  1. Right click on the transform file in the Solution Explorerpreview config context menu
  2. Select “Preview Transform”
  3. The editor window will be split in half. On the left is the original web.config and on the right is the configuration file after the transforms have been applied.preview config

Custom Configuration Sections

The usual practice in most project is to jam every possible configuration need in to the app.settings section.  It works but quickly becomes unmaintainable as your application grows and requires additional configuration capabilities.  However, as the application’s complexity grows and its configuration needs grow the app.settings section becomes difficult to manage.  At some point along this path the use of custom configuration sections will come up.

If you search the internet for information about custom configuration sections you’ll find information about how to write a custom configuration section class so you can make your own.  However, writing code for a custom configuration section is unnecessary.  There is an easy solution to this that does not involve writing a custom configuration section class.  The class behind the app.settings section is actually just NameValueSectionHandler, which you can use for your sections.

As an example our application needed to send email to users.  We needed to store the URL for our mail server, the user name and password to access the mail server, which port to use, and a few other settings.  Instead of just putting this in app.settings we setup our own section called Messaging and then added Messaging to our configuration.

To set this up we added a new section called Messaging to the configSections of our configuration file:


<section name=”Messaging” type=”System.Configuration.NameValueSectionHandler” />


Then we added Messaging to the configuration file:custom config section

It looks just like App.Settings, but keeps our setting separated and neatly organized.  To access our settings we just have to use one extra line of code as follows:

NameValueCollection settings = ConfigurationManager.GetSection("Messaging") as NameValueCollection;

if(null == settings) {

   throw new ConfigurationErrorsException("Missing Messaging configuration");


string smtpUser = settings["SmtpUser"];

Technically the above code should include a check to make sure settings is not null, but this is just a sample.  It is not difficult to wrap this in a class to encapsulate everything above to avoid writing the same lines of code repeatedly.

External Configuration Sections

In some cases because of the volume of configuration information needed it makes sense to divide the configuration file in to multiple files.  Unfortunately the configuration transforms done by visual studio will not be done to the external configuration files but you can just store each environment’s configuration information in a separate file.  However, keep in mind in the case of a web application all of the configuration files have to be marked as content so they’ll be deployed to all environments which is not really a good idea.  You’ll need to add some custom build steps to remove unneeded configuration files from the deployment package.

In our web application we’ll add an App_Config folder to store our external configuration files.  We’ll then add whatever configuration files we want:external config files

In our web.config file we will setup our connection strings as follows:

<connectionStrings configSource=”App_Config\connectionStrings.config” />

The cool part to this approach is that the code that is written to access the configuration information does not change.  The application is not aware that the configuration information it is using is actually stored outside of the .net configuration file.

Encrypting Configuration Sections

There are occasions where sensitive information needs to be stored in configuration files.  User names and password are a common example.  Having plaintext usernames and passwords laying around a file system is generally not considered secure.  Further I as a developer really do not want to know the usernames and passwords for the production environment.  Happily there is a solution to this problem that does not require your application to encrypt and decrypt configuration information for itself.

In this case we are going to use the Pkcs12ProtectedConfigurationProvider which Microsoft released as sample code.  The provider makes use of an X509 certificate for encryption and decryption which means you’ll need to install the certificate on the build server (or workstation) and in the production environment where the application will run.

The code for Pkcs12ProtectedConfigurationProvider can be found here: Pkcs12ProtectedConfigurationProvider 1.0.1

Add the provider to your solution.  In your web.config file (not in the transform) add the following:



<add name=”Pkcs12Provider” thumbprint=”XXXX” type=”Pkcs12ProtectedConfigurationProvider.Pkcs12ProtectedConfigurationProvider, PKCS12ProtectedConfigurationProvider” />



The thumbprint value comes from the certificate.  To get the thumbprint use the certificate manager plug-in.

Now that you can read encrypted configuration information you need encrypted configuration information.  The process for creating the encrypted sections of the configuration file are as follows:

  1. Create a plaintext version of the configuration file.
  2. This can done either by doing a build and getting the web.config file or using VisualStudio to produce the configuration file. This web.config file will contain the production (or stagin) user names and passwords that we want to secure.
  3. Use the ASPNET_REGIIS utility to encrypt secure sections of the configuration file
  4. In order to use ASPNET_REGIIS to encrypt your configuration you have to do the following:
  1. Start either powershell or the command prompt
  2. Copy the Pkcs12 assembly to the .net framework folder (c:\windows\\v4.0.30319\). Also add that path to your path (PATH %PATH%;….)
  3. Create a temp directory and 1 sub directory for each config file (c:\temp\prod, c:\temp\staging)
  4. Cd in to one of the config directories (c:\temp\prod)
  5. Enter aspnet_regiis –pef “connectionStrings” “.” –prov “Pkcs12Provider”
  6. Repeat for any other sections that need to be encrypted

After they have encrypted everything that needs to be encrypted they return the configuration files to development.  You extract the encrypted sections and put them into the corresponding configuration transform using the Replace transform.  For example:encrypted config

We now have secured our configuration information and our application does not have to make any code changes.

When the production version (or staging version) is built the above encrypted connection strings will be transformed in to the web.config file.

One limitation of encrypting configuration information is that you may not encrypt app.settings.

Also keep in mind that the web.config file cannot be more than 250Kb in size.  For most applications this is not an issue.  However, if you find your web.config file growing too large you can use the external configuration file feature previously discussed to move parts of the configuration information out of web.config.


Creating and maintaining configuration information for applications can be made easier by using the previously discussed features of the .NET platform and Visual Studio.  Configuration Transforms allow the web.config file to change depending upon the type of build being performed.  Using the same class that App.Settings uses allows you to easily create custom configuration sections without writing extra code.  You can move configuration sections in to external files to make it easier to manager.  Sensitive information can be encrypted in your configuration files without needing to write additional code.

Making use of this knowledge will reduce errors and make deployment to production easier.

Microsoft is doing it backward with iOS & Android apps

It seems like a pretty cool idea: take the app you wrote for iOS, make a few minor changes, rebuild and upload to the Windows Store. The problem though is that it keeps Windows on phones as a second class citizen. Why should I invest in developing applications that take advantage of the strengths of the Windows platform if all I have to do is recompile for the crappy iOS platform to get there?

Why not do it the other way? Why not provide tools that make it easy for me to write my applications on Windows first and then retarget to iOS and Android? Make XAML and the other parts of the .NET framework work on iOS and Android so the task of retargeting is easy?

I’ve not actually looked at the toolkit yet, but I also have a feeling that there is a lot more involved in retargeting an iOS application to Windows than just hitting F5 in Visual Studio. Guess I’ll have to go see for myself though. But I still think it is backward to Microsoft’s long term interests.

Perhaps you ought to reconsider those bad feeling you have toward those weird Libertarian folks.

From this:

Where Is Human Evolution Taking Us?

Not so much an article but some quick quotes on a favorite topic of mine.

My opinion, if it matters, is that humanity is either going to stop evolving or has already stopped evolving. We are going to start developing instead. Natural selection and mutation will give way to personal choice and economics.

Gene manipulation could allow you great choice in your physical appearance and conditioning. Want to be able to pack on muscle and burn fat like a hyperactive 5 year old? We have a gene for that. Want eyes that see near infrared? Nikon will make them for you.

How about eyes that also act as heads up displays. Software could provide you with facial recognition so as you walk up to that person whose name you keep forgetting your personal assistant (Cortana) reminds you that its “Bob” who works for RBH consulting you met him 5 weeks ago in San Diego and so on.

Hopefully the result of these advancements will be cures for most diseases. However, on the heels of those cures will be new problems. Integrating biological systems with digitals systems will deliver us the problem of who owns you, and how do we manage privacy? Apple lays claim to owning iOS, they also reserve the right to determine what applications you can and cannot run on your phone and if you jail break your phone you are cut off. What happens when that bit of technology has been integrated directly in to your mind? What happens when snapchat is performed just by thinking messages to a contact? That would mean the device could in some fashion read your mind. How do you feel about the NSA? Further, in order for the digital stuff to work it will need to be connected to a wireless network that would mean you could be physically tracked. Again, what are your feeling toward the government? In fact would it be unreasonable to expect a country like China to run a digital agent that monitors people’s feeling toward the government? Why stop there, why not help them feel better toward the government? Hell, why should Apple help you make the right buying decisions and feel good about them? Perhaps you ought to reconsider those bad feeling you have toward those weird Libertarian folks.

Tomorrow’s party: who is bringing the beer?

Regarding nearly everything going on in society: the hand wringing about gays getting married, trannies coping a squat in the wrong bathroom, hippies torching some bud, prom dresses showing too much. All of it misses everything that is actually going to change everything.

Technology has been advancing at a rate the economists don’t understand and its affects are barely understood. It’s not a matter of me printing off a barely functional gun in my house. Screw that noise. If I’m going to do something I’ll mix up something like Sarin nerve gas or a biological agent using stuff purchased from eBay at pennies on the dollar. Even that misses the changes.

More important is this: What happens as we automate all physical labor? We already have self-driving cars. The biggest hurdle left isn’t technological: it’s legal. What happens and who is responsible when your dumbass kid decides to run out in front of a self-driving car and is killed? However, that’s a mere quibble that can actually be solved technologically as well. What about all of the various drivers who will be unemployed with this technology? What about people who work in warehouses who will be replaced by robots who do the job better, faster, more cheaply and reliably? What about the kitchen staff we will do away with? Bartenders and Servers too? Construction workers will also be dispensed with (think huge 3D printers). Assembly lines are well down this road already. Hospital staffs can be greatly reduced. There isn’t an aspect of our economy whether legal or not that will not be affected by the rapid advancements already occurring.

Even aspects of high end intellectual work will be automated away. Most professionals no longer have secretaries or assistants. Attorneys no longer need staffs of paralegals to handle research, they use search engines that have everything needed already indexed. Accountants use ERP systems and other software to automate away the drudgery.

What will be left is the creative side. Software developers won’t be affected. Chefs, musicians, photographers, graphic artists, writers and other creatives will be in greater demand. Teachers, and tutors will continue. Sales people will be needed more than ever and so will entertainers (so much for killing off Hollywood).

In some cases ‘low end’ menial work will become super high end and exclusive. High end restaurants or other businesses that want to offer that high touch experience will keep human staffs but they will cater to a smaller client base. Some people will want that ‘authentic’ experience: the shirt sewn by an actual human (who uses a sewing machine BTW), or have the chef actually hand mix the ingredients rather than just programming the device (probably a 3d printer of some sort). These services will be available, but they won’t be cheap.

So what happens to everybody else? Soylent green? Some dystopian matrix style nirvana? Through innovation do we accidently realize Karl Marx’s dream of a ‘workers’ paradise? Robots doing all of the work while we humans sit around eating strawberries so big you can eat them like hand fruit?

The book Accelerando deals with some of these questions. Nearly all of William Gibson’s body of work searches for the questions and answers. Same with most Cyberpunk writing. The movie “The Matrix” paints a bleak picture of the outcome, but I personally don’t believe it has to go that way. I believe we can actually achieve something really cool.   How we get there is not entirely clear to me, but I don’t think socialist programs are the right way. I believe those will lead to a Matrix outcome.

Microsoft Band Impressions

When Microsoft initially announced the Band I thought it looked really cool and that I would want one. However, I don’t like dealing with rushes so I figured I’d wait a while until things settled down and then I’d get one. I failed. After two trips to the mall I decided to just buy one off EBay.

I hate buying stuff like this from eBay because the people who generally run out buy up all of a new product and then resell it on eBay are part of the problem. They’ll buy up all of the new iPhones at launch and then resell them at twice the price. However, people buy them and pay the markup. I got lucky, somebody had bought two but his wife didn’t want one so he was sell it pretty much at cost (band + sales tax + screen protector + shipping).

Anyway, I have one now and the Microsoft Stores are still pretty much out of stock. Although the online store briefly showed that they were in stock, but today it’s back out of stock again.

First some background: I have a FitBit and have been using it a lot for the past 6 months. I really like the FitBit and think that it’s a good product.

Now my impression of the Band: I like the hardware. It’s bigger than a FitBit but smaller than a watch. The display is touch sensitive and there are two buttons on the side. There is a power button and an action button. Then there are all of the sensors they crammed in to the Band: GPS, heart rate, motion, temperature, UV, clock and whatever else. It also has Bluetooth to communicate with your phone or other devices.

The basic hardware in the Band blows the FitBit out of the water. The FitBit just has a motion detector, clock and Bluetooth. So the only thing it will ever really do is detect is when you are moving and how much movement is going on. Which is actually really useful. The Band however could go so much further. Which brings me to the Band’s short coming: software.

The Band’s software at the moment feels quite limited. However, I’m still learning my way around. For instance, the steps taken goal out of the box is 5000 steps. I do 5000 steps in my sleep. The screen says you can change the goal but it is not immediately obvious how you do that. I looked at preferences and other areas and didn’t see anything. Then I went in to the Steps screen and finally noticed the small pencil icon on the right edge. That takes you to the screen that lets you set your steps. While it wasn’t immediately obvious to me, I had been on the screen and not noticed the pencil thing, given the direction I think they are going I can see the logic of not having all of the goal settings put in a preferences screen.

Because the band monitors your heart rate the steps screen can show you your heart rate while you were out running. Or in my case walking the dogs (peaked at 115, not exactly a workout). I think that’s pretty cool. Given the GPS data I’d love to know if there were elevation changes.

The Sleep screen is actually a lot more detailed than FitBits. Again: heart rate is tracked so you can see that information. That combined with the motion detector you can get a better idea of how restful your sleep really was. Overall really cool.

I’ve not used the Guided Workout yet. I’m going to try out the Couch to 5K workout. What I don’t like is that I do not see a way to create my own workout. I’m doing Mark Rippetoe’s Starting Strength program, I can’t find anything like it in the existing work outs and I don’t see a way to add it or make it myself. Hopefully this will be rolled out in the next release. Given the capabilities of the Band it should be possible to enter what exercise you did, how much weight you are working with, and how many repetitions you did. Perhaps the other Guided Workouts are already doing this and maybe there is something for this already, but I’ve not found it yet.

The big pieces of the application I see missing are: diet/food tracking and weight tracking. Perhaps I’ve not found them yet, but I don’t see it. What I don’t want is to have to use another app that is not integrated with this…For instance: with the FitBit I could enter in what I’ve been eating and my weight and track my progress.   At the moment I can’t do this in the Microsoft Health application running on my phone.

I could get a different application to track that information but now I have to go to two places. Worse, I hate entering data in to my cell phone. On screen keyboards suck and voice input ain’t happening. What I really want is a set of applications that work on my phone, tablet and PC so I can do what I want to do where it makes the most sense to do it. Throw in the apps on the band too now I guess.

What I see in the Band and its software is a great start. It’s not a finished product though. From what I see and what I’ve read Microsoft is doing two things that are really smart. First: they are acting like a startup. They’ve released the Band as quickly as possible instead of waiting until they had a polished product. The Hardware is polished pretty well with only a few improvements needed (the glass scratches too easily is a common complaint I’ve read) which makes sense b/c it will be hard to get people to turn around and buy new hardware quickly. But the software is still beta in my opinion. It has some really cool features that I like but given the capabilities of the hardware they have barely scratched the surface.

The second thing they are doing is making it a platform. Based on Microsoft’s existing HealthVault platform and the messaging around Microsoft Health it seems that there will be ways to create applications that extend the Health application. Hopefully they will also make it possible to write applications that run directly on the Band as well.

At this point I’ve had the Band for less than 24 hours. I like it and I think as it stands it’s a kick ass product. However I also see a ton of room for growth.


Back when I was actively blogging this was a software development focused blog. Going forward there will still be software development material here but my focus is going to shift. A lot.

Here’s the thing: I’ve spent the last uncounted years being a wantrepreneur. Like most software developers I have tons of ideas for great products. I’ve even put in long hours building a few of those products. One I even got as far as being ready to market it, it even had a web-site. For lots of reason I just completely failed at marketing it so it never sold and here I am still playing the same game.

So here’s the deal: My goal is to end my role as an employee and wantrepreneur and move in to the new role of entrepreneur this year (2015). My goal is actually to make it happen really fast. It’ll probably fail and I’ll be doing another blog post like this for 2016, but for whatever reason I can’t make this itch go away and I think I’ll go mad if I don’t keep trying until I succeed.