Category Archives: Development


Everyone who writes systems at any reasonable scale needs a high performance logging solution. A number of these exist with varying degrees of functionality and complexity. I have been using NTrace which is a managed wrapper around the windows Event Tracing for Windows (ETW) libraries. ETW is a high performance trace system implemented as a device driver that was first introduced in Windows 2000. There is not a lot of info out there on ETW although it is beginning to gain some traction. I know that it is used internally at Microsoft as a tracing and instrumentation solution in large systems. For a background on ETW see this MSDN article.

NTrace is an elegant wrapper with an easy to use syntax.

EtwTrace.Trace("Item {0} Was not added to the database. Message: {1}", itemId, ex.Message);

The messages are compiled into TMF files which combine the text with the message Ids and data from the ETW system when viewing a trace. Some other awesome things are adjusting the trace level on the fly…a must for high performance applications. ETW supports circular log files which is highly useful. The performance of ETW is ample for even the most demanding applications. It is possible to instrument an application at a very detailed level and with very little performance cost.

Microsoft allows WCF and WPF both to log to ETW. I am moving all my WCF logging into ETW. The Service Trace Viewer can still open the resulting files.

The only downside is the precompiler does not provide a lot of information as to the nature of any errors that you made.

See Andy Hooper’s blog.

Is SQL Dead?

Well on the heels of my post last night is one on the advantages of specialized databases for warehousing applications. Is SQL dead? Not yet but with companies like Vertica historical databases will continue to not be chosen in new applications. Interestingly, MySQL does have one of the characteristics of these systems, horizontal scaling.

Technorati tags:

The future platform

I want for a moment to address a weakness that I think exists in Microsofts tool offerings. Microsoft does not offer tools for building ultra high performance, extremely fault tolerant and widely distributed applications. While some may consider this to be a niche market I do not. And further more the applications, platforms and tools that require such technology will be integral to a much wider array of applications through the services that they offer. A more concrete example would be a search engine. Search engines have massive computing needs and need to be infinitely scalable. Consider an application like an Electronic Medical Record being run as an ASP. In order to handle the plethora of data from thousands of sources and to enable the application to always be available new computing approaches need to be developed.

Forget everything that you have learned about traditional applications. N-Tier applications have difficulty scaling up to the size that is required by a large scale web application. Traditional components like databases will continue to exist and be important but the features that are important will change. For instance startups and large companies building webscale applications have increasingly turned to MySQL. MySQL enables horizontal scaling so that to increase capacity you simply add servers. This is very different from something like Microsoft SQL Server where you need to scale up onto larger machines. Don’t think that MySQL is ready for prime time? Yahoo, Amazon, Nokia, and Google disagree.

However that is not a significant departure from traditional applications. Enter Hadoop. Hadoop is a distributed computing platform that has many features that are similar to Google’s base technology. It implements Map Reduce. It has a distributed file system. Now this is an advanced filesystem that is fault tolerant. It can copy and replicate data on demand for frequently accessed objects. Hadoop provides a simple infrastructure for developers to use to build applications that can deal with enourmous volumes of data. Why would companies that are not search engines be interested? Simple, Hadoop can be used to solve very demanding problems very cheaply. Want to build an EMR with advanced data mining functionality? Hadoop would enable the data to be analyzed in a fast and inexpensive manner.

Hadoop is built in Java and runs natively on Linux. For Microsoft that is a problem. Already many web applications, especially Web 2.0 run on LAMP. Microsoft has competitors to these though that are good, even if more expensive. The argument that Microsoft tools have developer time may make up for more expensive software but that is a seperate issue for another time. However when deploying applications that run on a large number of machines can lead to huge licensing costs. Each computer needs its own copy of Windows even though the only thing that these machines will be used for is computing and storage. Few Windows services will ever be used. So Linux makes sense in this situation.

Microsoft should do three things. Microsoft should sponsor a port of Hadoop to .Net. That will give Microsoft developers the same tools as their open source associates. Microsoft will need to hire a bunch of FTEs to make this happen. Second SQL Server needs to easily scale horizontally across many machines cheaply. Third their needs to be a version of Windows to support this. Windows Compute Cluster is not what we are looking for. Windows Compute Cluster does not support .Net natively (although it can use PInvoke) and is targeted at people I think with legacy applications. Information from the website is short on details. If someone wants to correct this please do.

Microsoft needs to do something to address this niche but rapidly growing market segment.

Technorati tags: , , , , , ,

Vista and ASP.Net Membership Providers

So I have started working on a prototype for a new application. I recently switched to a Vista machine for my development work. I started with setting up an authentication system for a website using .net 2.0’s membership provider. Easy right? Not so much. After fighting with a message saying Sever Cannot Map Path ‘/’. for a couple of days I had the brilliant idea of disabling UAC. Presto. It started working. So I have disabled UAC on all my machines.

Technorati tags: , , ,

.net’s Achilles Heel

I have been a .net developer since .net beta 1 back in 2001. I love the platform and consider it to be superior as a single platform to anything else out there. However right now I am learning Java. Why? I am leveraging some opensource toolkits and pieces of software that do not have versions available in .net. I am using Nutch as the basis for a new search engine. Nutch is a fairly good piece of software. The underlying distributed computing components known as Hadoop are awesome. It is fairly easy to build a cluster of dozens to hundreds of machines to power a search engine.

So in this case the availability of a piece of open source software is driving my adoption of a new platform. I think that Microsoft should think long and hard about sponsoring opensource projects.


There is a lot of opensource software out there for .net. I just think there is a real lack of it in “high end” areas, like distributed computing. There is also not a lot of standardization in projects. I mean how many business frameworks and O/R Mappers do we really need?

Technorati: , , , ,