Bring It On

Considering how much functionality is in the .NET Framework and how .NET PE files’ metadata is highly compressed, one would expect most .NET applications to be very small.

You can empirically observe this by trying to create a simple “Hello World” in C#: the resulting executable is just 3.5KB (or even smaller with versions prior to 2.0). The size does not increase by much as you continue adding code and reasonably sized (in functionality) .NET applications are just a couple of hundred KB or even less.

Just the other day I checked to see how big the core part of my P2P BitTorrent app was – 160KB. Hm, this doesn’t look right; I keep this library lean in general, it should be smaller.

So I fired up ILDASM (IL DisAsseMbler) and checked the overall statistics (View->Statistics). Here’s the important part:

 File size            : 163840
 PE header size       : 4096 (496 used)   ( 2.50%)
 PE additional info   : 1219              ( 0.74%)
 Num.of PE sections   : 3
 CLR header size      : 72                ( 0.04%)
 CLR meta-data size   : 81268             (49.60%)
 CLR additional info  : 0                 ( 0.00%)
 CLR method headers   : 11081             ( 6.76%)
 Managed code         : 54814             (33.46%)
 Data                 : 8192              ( 5.00%)
 Unaccounted          : 3098              ( 1.89%)

Well, well, well… 50% of the image is actually metadata! Despite heavy metadata “compression” (this is a custom kind of compression) in the .NET Framework, metadata occupies half of the total space, while code takes only 34%. Interesting…

I guess if I obfuscate the image using an IL round-tripping obfuscator that I could get quite a smaller image.

Quite some time after incorporating (considering they already had a product that worked) BitTorrent Inc has finally started selling content (which was their intention from the start). The real question is: will users follow?

BitTorrent as a technology has a reputation of mainly enabling illegal file swapping. As a developer who analyzed and independently implemented the protocol I can confirm that as a file-exchange mechanism it is very efficient and scales extremely well. Most of the BitTorrent related places on the Internet that I am aware of indeed facilitate swapping of copyrighted material including movies, TV series, console and computer games and software.

Let’s compare for a moment a typical usage scenario for an end user who’d like to watch popular TV shows like “Lost”, “Prison Break” or “24”. If possible, the user would probably like to choose the quality – for HDTV owners the only acceptable choice is HDTV but for analog TV owners smaller files of lesser quality probably suffice. Nobody likes to manually check if the new episode is out, so some kind of RSS feed would be nice. Finally, if the files were not protected by a DRM or similar mechanism and if they could work on Macs and/or Linux machines (and not only Windows) that would be great, too. Awesome if the files could be transcoded and uploaded to an iPod or PSP.

If you’re just a bit Internet savvy, you can find several (illegal) places on the Internet that would offer you all of this.

BitTorrent Inc offers some of the important elements, but not all. For example, they give you the RSS feed of the vendor’s account, not yours (maybe they do, since I’m not in the USA, and the business is still USA only, can’t even check). The files are in WMV format, Windows only and protected by a DRM system. At least for TV series, the episodes are bought and not rented and can be watched on 2 machines at most – but the movies are rented and you only have a limited 24 hour window to watch them on one machine. You can not transfer any content to a portable device. At $1.99 per TV episode and for a level of quality they are offering (481MB per episode, that’s slightly better than the standard illegal rip quality) it is expensive. I won’t even comment on $4.99 per movie rental.

So, I can choose between USA only service with lowish quality DRM Windows only video with no RSS feed and $1.99 per episode or HDTV quality unprotected multi-platform video with RSS updates for free?

There’s one other non-obvious element here – motivation for sharing. BitTorrent is a P2P technology that only works if as many as possible people are uploading for as long as possible. When you download illegal content, you’re getting it for free thanks to the kindness of others so you’re motivated to give back too. Thus most of the BitTorrent clients have a minimal upload/download ratio that each of the downloads will have to achieve. At the same time, due to the way BitTorrent works, it is not efficient to share large number of files simultaneously, so it’s important to prune the download list. You need to balance these two things, uploading as much as you can without hurting your overall download speeds.

When you pay to BitTorrent there’ s no moral obligation whatsoever. You paid for the damned things and you don’t care how much you’ll upload. You will upload during the download due to the nature of the protocol, but as soon as you’re done, there’s zero incentive to keep uploading – why would you donate your upload bandwidth to BitTorrent Inc – you paid already, didn’t you? I don’t see how is BitTorrent going to manage this problem. If the point of P2P distribution is that it simplifies the distribution for content producers lowering the prices for the users, I don’t see that in practice. The content is too expensive and the model does not motivate users to share, which made the distribution model theoretically cheap in the first place.

P2P has its uses, but for selling content? I don’t think so.

In the previous post I tried to describe as good as I can all the things that make ASP.NET 2.0 great platform, but also all those that make it very irritating and tiresome.

I ended up choosing Ruby on Rails (RoR), but the decision was not nearly as simple. I like Ruby as a language just like many people do – there’s something magic about it that seduces you from the moment you write your first non-trivial script. But the hype surrounding Rails was so strong that I was put off by it. Finally, when I realized that building my site with ASP.NET will most likely become an exercise in frustration I decided to give Ruby on Rails a chance.

Note that unlike ASP.NET, I did not yet build anything serious with RoR. However, looking back at the project I did build with ASP.NET it is quite clear that it would have been almost trivial to build in RoR. To understand “almost” you’ll need to check out the list of what’s good and what’s bad about RoR. First the good stuff:

• RoR is an application framework and not a class framework; this means that there is a certain way you build your apps and especially if you are inexperienced this will help to guide you
• RoR’s MVC (model-view-controller) implementation is very good, especially its ORM (object-relational mapping); ORM is not the most flexible one, but for most of the purposes it works really well – as long as you follow the conventions; ORM supports multiple database backends: MySQL, PostgreSQL, DB2 but also Microsoft SQL Server
• RoR has a concept of conventions over configuration – by following the naming of database columns, classes, modules and methods you can achieve a lot; for those that like flexibility and don’t like rules this is irritating, but for practical folks (this is my opinion) this is great because all your “configuration” information stays embedded in your source code files
• RoR insists on good testing practices; the skeletons for your unit tests are generated along the rest of the generated files and the community around RoR seems to pay just right dose of attention to tests, both unit and functional
• RoR is very practical by supporting three different kinds of development environment, each with its own database – development, production and testing
• RoR has feature parity for most of the features a modern Web framework should have (just like ASP.NET) – things like ASP.NET master pages (they are not called like that but any page that consists of other components is effectively a master page), good caching, session support, AJAX…
• If you do know HTML, writing the pages will be very easy; RoR embeds Ruby in HTML, not unlike ASP (not ASP.NET!) did years ago, but there’s a difference – because RoR is an application framework, you do not end up mixing business logic and presentation code, you just mix code and markup; after thinking about this a lot, especially after my struggle with some of ASP.NET controls while trying to tweak their behavior I think this way is better
• RoR has a strong emphasis on DRY (don’t repeat yourself); if you are experienced, this won’t matter because you’ve already learned the lesson the hard way, but if you are a beginner, this is a very nice habit to adopt
• RoR is constantly developed and if you are inclined to do so, you can “live on the edge” and always run the latest and the greatest code; the release cycle is also faster than for ASP.NET because there are no other products to tie the launch with
• There are several IDEs to choose from, including extensions for the two giants of modern IDEs – Eclipse (RadRails) and Visual Studio 2005 (Ruby in Steel)
• Ruby is a great language that fits the needs of a Web developer very well; speed is not as important here as much as flexibility and expressiveness
• RoR is free, runs on variety of free Web servers (like Apache, lighttpd and Mongrel) and free Operating Systems (practically all popular versions of Linux); it generally a lot cheaper to find a Web hosting deal on Linux with MySQL or PostgreSQL than it is for Windows with IIS and Microsoft SQL Server

Now for the bad stuff. You’ll notice that just like with ASP.NET, both the good and the bad list are just as long. Programming is hard – deal with it . Here we go:

• RoR is hyped beyond belief; some of the screencast will make it seem almost like magic; people will take it for granted that RoR is the magic silver bullet they’ve been waiting for and be very disappointed when it does not fit their needs
• The concept of conventions over configuration works great as long as you can fit your app into the convention – if you can’t, tough luck; example is composite primary keys – can’t do it in RoR without a lot of trouble, RoR likes its keys as a single column named id of type integer and auto-incremented
• If you don’t know HTML, you’re in a tough spot; there is no concept of controls spitting out markup and the IDEs do not have nice designers – you’ll have to build all your markup manually
• RoR is not documented well and there is no alternative to MSDN library; the situation is improving, but for the meantime your best bet is to buy books (that are not free and cost money)
• When I said “my app was almost trivial to build in RoR”, the missing feature I was referring to was i18n (internationalization); this is actually a problem with Ruby itself, not RoR, but doesn’t matter – building properly localized Web apps with RoR is incomparably more complicated than with ASP.NET; this is the biggest problem with RoR at the moment
• RoR is not thread-safe, so if you want to serve several requests at the same time you have to set this up yourself (non-trivial, see next point)
• Setting up (or deploying) RoR application is complicated just because there’s so much choice (regardless of what Open Source zealots will tell you, my opinion is that too much choice hurts) – do I use Apache or lighttpd or maybe Mongrel, or maybe I should combine them with a load-balancing scheme; should I deploy to Windows or Linux and if Linux, which distribution and which version etc
• RoR is young and sometimes changes a lot; if you want to stay on top of it all, you have to run the latest code which in turn means that you’ll run into bugs and won’t know if it’s your bug or framework’s
• IDE support is not as nearly as sophisticated as it is for ASP.NET, including integration with databases
• Third parties do support RoR in the form of plugins, but there are so many and most are a one-off job from a single dev just playing with the technology so you have to be careful which ones to pick to base your code on
• Ruby is a great and flexible language but it is slow – if you need stuff done fast, you might have to resort to writing C based extensions which is hard if your background is pure Web development with scripting languages only
• Ruby as a language evolves very slowly; the i18n problem that I mentioned is supposed to be fixed Real Soon Now^TM yet it took more than a year to move from 1.8.2 to 1.8.4 (with a small fiasco with 1.8.3)

Looking at all of this, it’s obvious that RoR is not a clear winner over ASP.NET. In fact, the site I built with ASP.NET, despite all the frustration would probably be harder to build with RoR just because of bad i18n support.

In the end, having in mind my needs and the size of the site I am about to build, I decided to go with RoR. It just feels right. Everything makes sense. The framework is there to help you, not to fight you. If you can live with all the conventions and restrictions of course.

So choose wisely. Ignore the hype from all sides. Try building something and definitely read a few books and search the Internet. There’s a lot of information out there. Hopefully the last two posts will help you decide too.

I need a Web framework to build my site. I am proficient in C++ and C# and Microsoft SQL Server 2005. I work on Windows XP Professional 90% of the time and I like the .NET Framework very much.

Based on all this, I should have chosen ASP.NET (2.0) for my product’s Web site, yet I didn’t. Note that I have already built a (small) site with ASP.NET 2.0 so the experience is my own, not something that I have read on the Internet.

In this post, I’ll list the pros and the cons of ASP.NET 2.0. First, the good things about ASP.NET 2.0:

• The underlying framework (.NET Framework a.k.a. Framework Class Library or FCL) is great – it’s the same class library I use for the desktop product so I “know it” already (I put quotes because you can never really know a huge framework like that)
• The tools support is excellent, plus I use the same IDE as I use for my desktop product: Visual Studio 2005
• The debugging experience is flawless
• If you don’t know HTML, getting a decent site up fast is trivially easy
• Graphical preview of the site’s look while in the IDE is almost perfect
• There is Intellisense everywhere and the IDE can check compliance with various HTML/XHTML versions; the Intellisense even adapts to the schema chosen, so if you pick XHTML 1.0 Transitional you’ll get errors for each element or attribute that is not allowed
• There is a big community around it and many, many third party products to complement what Microsoft has offered us with the ASP.NET
• The internationalization support is way better than anything I’ve seen, on any platform
• There is a small Web server bundled with the IDE so you don’t need to have a “real” Web server on your development machine
• The documentation is good and the latest MSDN library is now free for anyone to download
• There is a free edition of Visual Studio 2005 (Web developer) and a free edition of SQL Server 2005 (Express) so technically it costs nothing to develop for this platform

Now for the list of what’s not so good:

• The deployment is confusing and the fiasco with the new project management is irritating; instead of using the same build engine (MsBuild) as for the other types of project, Microsoft decided to “innovate” and gave us Web Site “project”, only to subsequently give us Web Application project (as an add-on), plus the Web Deployment project (are you confused yet?); some of these add-ons simply do not work as advertised and generally things get really confusing with so much choice
• The SQL Server Express 2005 user instances feature is great, but the deployment has a tragic flaw – the idea is that you’ll simply put your database in the App_Data folder and xcopy it with the rest of your Web project (or deploy it using one of the several ways, see previous point), but this does not work; something needs to be registered somewhere, the easiest thing to do is to install Management Studio Express 2005 and attach the database at least once (!?!)
• While the general architecture of the ASP.NET pipeline is great, the UI generation is sub-optimal at best; if you do know HTML, you’ll end up fighting against the framework more than using it to get things done - just one example: instead of getting less markup in your “template” code than when using plain HTML, you end up having more markup; this applies mainly to the “customized” UI – if all you’ll do is to have labels on the columns of your data table (for example) identical to the names of the columns in the database and you’ll never customize any styles, then the generated markup is short and nice
• More on the UI generation – the Controls are a nice idea, but the markup they spit out is mostly hideous and not CSS friendly; Microsoft knows that and they’ve been working on Control adapters (I have even deployed one, the Tree control’s output was very ugly and bloated using tables), but this is a patch of something that should have been done properly in the first place and only complicates deployment even further; additionally, the control adapters project appears to be dead, with no updates since end up April so I can assume this to be just yet another CYA (cover your ass) side-project and not an honest effort
• The skins feature is nice but ends up putting all the styles inline, instead of in a CSS file where it can be shared (and cached by browser); additionally, the editor for skins which are nothing more than stripped down ASP pages does not support Intellisense unlike the rest of the IDE; fortunately, you can safely ignore skins and make a pure CSS site that is still skinable
• There is no object-relational mapping (ORM) layer per se, but there is a good support in the shape of ObjectDataSource that allows you to treat many things as data sources and then bind all of your UI to the ObjectDataSource; it almost works – try the most trivial, yet very frequent thing like inserting a row into a table which has an auto-incrementing id column (which may or may not be a primary key at the same time), then try to get back that id – it’s gonna get too complicated for such a simple thing
• The data adapters’ (the layer supporting the data sources) editor is very rude and will often overwrite the markup you’ve added or changed yourself (the underlying format is XML and is human readable)
• When things work, great; when they don’t, the documentation does not help – if you don’t do things the way the designers anticipated, prepare yourself for a lot of frustration, mainly because you’re trying to tweak something that should be trivial; I have previously posted two examples of this here and here
• While there’s a lot of code to assist you, there is no real framework to be used (a la MVC or something similar); in other words, the framework is mostly physical (lots of classes for a lots of uses) instead of logical, trying to guide you (and yes, even punish you when you do something stupid)

In general, looks like ASP.NET has been built mainly for Intranet applications and in-house developers or beginners. If you don’t care about the way the pages will end up internally (including the size), will work on IE and the rest be damned and just want to get the results quick, by all means choose ASP.NET. If you do care about standards, page size and using the framework instead of fighting it, you’ll have to look elsewhere. That may well be Ruby on Rails, but more on that in the next post.

Two lucky coders and one blogger will attend PDC for free, but none of them will be me :( Nevertheless, it was fun competing and having a deadline really pushed me to finally release something.

So now it's out - I have been playing with the idea for quite some time, developed most of it during last year, then abandoned it for almost half a year only to refactor and change a lot more than I expected before attaching a GUI onto the download engine I thought I had. If you'd love to use BitTorrent but absolutely must have a .NET solution, you can try NTorrent now - just head on to the Channel9 PDC contest and download my entry. I will soon publish here a version that does not require Shareware Starter Kit and that is further improved. The contest pushed me over to .NET 2.0 so I'm afraid that will be mandatory.

Speaking of coding contests, Mike Gunderloy is having one over at Larkware. Mike is actually very bold and was teasing Microsoft yesterday:

I note that Microsoft only managed 16 entries to a contest giving away a trip to the PDC. So here's a challenge to the Larkware community: I'll bet we can get more entries than that for the Larkware 2005 Developer Tool Programming Contest.

No disrespect Mike but I think that this contest is a bit contradictory.

First of all, just like with Channel9 contest it is required that all the entries be freely downloadable. This rules out any kind of serious development, if we don't count some kind of crippled version of a regular product. Second, the deadline is short - nobody can develop anything serious with a tool that is still in a beta in two months (I should know, been struggling with beta 2 bugs quite a bit). Thus I conclude that the only entries will be from single developers working on their "for fun" projects at home, after work.

But if we look at the prizes they are all what is known as enterprise class software, stuff for teams or unlimited site licenses. It might have high nominal value in dollars, but it has very little value to me as a single developer of a future microISV. Therefore I find it hard to motivate myself to apply and I think others might come to the same conclusion too. On the other hand, a trip to PDC has just right value for this kind of competition even if it has lower nominal dollar value. So I wouldn't laugh just yet ;)

In any case I would still recommend people to compete if they have something already and feel the itch to try Visual Studio 2005. I learned  a lot while porting my app and by the time Visual Studio 2005 ships it will take me no time to adjust - I already have.

What should one do if one finds out that his/her creation is shared on a P2P network? You can't sue, as you generally have no idea who put it there^*, you can't remove it from the network - heck, you can't even estimate the number of people who have it.

Some have tried polluting the network with fake versions of the same thing, some even tried infecting the file(s) with viruses (!) but the latest trick I saw made me smile - if you have to do something to protect your work, here's how you should do it...

I recently found out about Edward Tufte's Visual Display Of Quantitative Information and decided I need it (badly and urgently). It took me few minutes to order it through Amazon (FR) but the availability is at the moment 1-2 weeks (!). Impatiently I went to eDonkey network and searched for any electronic version - I needed to make a quick look as soon as possible. Sure enough, I found a PDF version even though of suspiciously small size.

Few hours later the "book" was sitting on my hard drive. But it was not really a book ;) It was a 5-6 page document consisting of:

• the cover of the book
• the link to the Amazon page of the book
• the PDF form (!) to help you order it from Amazon
• list of very positive reviews of the book on Amazon

I don't know about you, but for me this is pure genius, assuming it's the author (or someone he hired) who did it. The message was clear - if you want the book, here's how to get it - yet not rude.

Hey Scoble, if you decide to publish your book in the paper form and are afraid that P2P networks might cannibalize sales by providing electronic version for free you might consider doing this same thing ;)

^*Actually if it's your software that is being shared, the best thing to do is to display the registered user's name prominently (in the about box, on the caption etc). Then you'd at least know who leaked it...

As is customary, Scoble has gathered several links to the interesting opinions on Grokster case. In the following post he gives us his view:

I was wondering to myself today: "Is Bram Cohen packing his bags?" I can just imagine the lawyers are working overtime to write up lawsuits against BitTorrent that he developed. Yes, even if he could prove that he wrote it for legitimate file sharing (for instance, for putting your own podcasts up and sharing those with your friends -- Adam Curry on Saturday says he's going to support using BitTorrent to distribute his podcasts since each podcast he puts up is eating up hundreds of dollars of bandwidth fees and his site was unreachable earlier this morning) I imagine that Bram Cohen will get taken to court to prove that. Lawyers are expensive. Defending a court case like that -- even if you win -- will cost millions.

Don't worry Robert - this decision has nothing to do with BitTorrent, and I'll tell you exactly why. First, let's see what did Supreme Court exactly say:

We hold that one who distributes a device with the object of promoting its use to infringe copyright, as shown by clear expression or other affirmative steps taken to foster infringement, is liable for the resulting acts of infringement by third parties.

First of all, BitTorrent is not a network. If you want to share something using BitTorrent you need to do the following:

1. Decide what to share (one file or a hierarchy of folders), from now on referred to as data.
2. Gather all metadata about the data you will share. This metadata is kept in a file with the extension "torrent" aka torrent file. Whoever wants to download that data needs to get the metadata first (in other words, torrent file). The torrent file can be downloaded from a Web page, sent by email, FTP or any other means.
3. The metadata file is then fed to BitTorrent application. There are dozens of implementations (including my own, soon to be published) and they all (to my knowledge) interoperate even though they run on multitude of operating systems.
4. The crucial part of the metadata is server address/URL (server is also called tracker). The only use of the tracker is to give each BitTorrent application information on which other users (BitTorrent applications that is) are interested in downloading the same data. Get this - tracker is usually a simple, single PHP or ASP.NET page running on a plain standard Web server.
5. From that point on, BitTorrent apps communicate between themselves and share pieces of the data until the download is completed.
   

What this all means is that there is no such thing as BitTorrent "network" - trackers for various files (data) are scattered around the world and each tracker, well, tracks a (usually) small number of files. There is no single individual or company that stands behind all (or most) of these servers. Without servers, BitTorrent apps cannot find^* other BitTorrent apps and thus can't share anything. Therefore, BitTorrent applications cannot themselves do anything to "promote its use to infringe copyright". The servers that index copyrighted material and provide both torrent files and trackers are the ones the above court ruling applies to. One of the best known ones (recently shut down exactly for the reasons stated above) was suprnova.org.

That said, the Supreme Court decision is vague enough to induce further debate and complicate the decisions of the lower courts and personally, I don't like it at all.

^*I am aware that the latest beta of official BitTorrent client does not require servers to find data, but it's a topic for another post.