Well on the heels of my post last night is one on the advantages of specialized databases for warehousing applications. Is SQL dead? Not yet but with companies like Vertica historical databases will continue to not be chosen in new applications. Interestingly, MySQL does have one of the characteristics of these systems, horizontal scaling.
Technorati tags: distributed_computing
I want for a moment to address a weakness that I think exists in Microsofts tool offerings. Microsoft does not offer tools for building ultra high performance, extremely fault tolerant and widely distributed applications. While some may consider this to be a niche market I do not. And further more the applications, platforms and tools that require such technology will be integral to a much wider array of applications through the services that they offer. A more concrete example would be a search engine. Search engines have massive computing needs and need to be infinitely scalable. Consider an application like an Electronic Medical Record being run as an ASP. In order to handle the plethora of data from thousands of sources and to enable the application to always be available new computing approaches need to be developed.
Forget everything that you have learned about traditional applications. N-Tier applications have difficulty scaling up to the size that is required by a large scale web application. Traditional components like databases will continue to exist and be important but the features that are important will change. For instance startups and large companies building webscale applications have increasingly turned to MySQL. MySQL enables horizontal scaling so that to increase capacity you simply add servers. This is very different from something like Microsoft SQL Server where you need to scale up onto larger machines. Don’t think that MySQL is ready for prime time? Yahoo, Amazon, Nokia, and Google disagree.
However that is not a significant departure from traditional applications. Enter Hadoop. Hadoop is a distributed computing platform that has many features that are similar to Google’s base technology. It implements Map Reduce. It has a distributed file system. Now this is an advanced filesystem that is fault tolerant. It can copy and replicate data on demand for frequently accessed objects. Hadoop provides a simple infrastructure for developers to use to build applications that can deal with enourmous volumes of data. Why would companies that are not search engines be interested? Simple, Hadoop can be used to solve very demanding problems very cheaply. Want to build an EMR with advanced data mining functionality? Hadoop would enable the data to be analyzed in a fast and inexpensive manner.
Hadoop is built in Java and runs natively on Linux. For Microsoft that is a problem. Already many web applications, especially Web 2.0 run on LAMP. Microsoft has competitors to these though that are good, even if more expensive. The argument that Microsoft tools have developer time may make up for more expensive software but that is a seperate issue for another time. However when deploying applications that run on a large number of machines can lead to huge licensing costs. Each computer needs its own copy of Windows even though the only thing that these machines will be used for is computing and storage. Few Windows services will ever be used. So Linux makes sense in this situation.
Microsoft should do three things. Microsoft should sponsor a port of Hadoop to .Net. That will give Microsoft developers the same tools as their open source associates. Microsoft will need to hire a bunch of FTEs to make this happen. Second SQL Server needs to easily scale horizontally across many machines cheaply. Third their needs to be a version of Windows to support this. Windows Compute Cluster is not what we are looking for. Windows Compute Cluster does not support .Net natively (although it can use PInvoke) and is targeted at people I think with legacy applications. Information from the website is short on details. If someone wants to correct this please do.
Microsoft needs to do something to address this niche but rapidly growing market segment.
Technorati tags: .net, distributed_computing, hadoop, java, linux, microsoft, windows
So I have started working on a prototype for a new application. I recently switched to a Vista machine for my development work. I started with setting up an authentication system for a website using .net 2.0’s membership provider. Easy right? Not so much. After fighting with a message saying Sever Cannot Map Path ‘/’. for a couple of days I had the brilliant idea of disabling UAC. Presto. It started working. So I have disabled UAC on all my machines.
Technorati tags: .net, development, vista, software
I covered PeerVision back at its launch in June. I have checked back on it a couple of times. There has been very little activity on the site, at least as far as the forums go. I don’t know if their blogs get much traffic. I am not sure that PeerVision really has an attractive value curve for visitors that would allow it to effectively compete with other internet communities such as AuntMinnie. It is an interesting experiment but I think that it needs better content to draw people to the site.
Come on. Hire a guy to walk around with a video camera and interview people, ala Robert Scoble. Being a little naked in front of your customers is not bad. Bring us something that does not exist anywhere else.
Technorati tags: GE, healthcare, radiology, web