Bring It On

Weird title? Not really. I’ve said before that multi-threading programming is hard and despite advances in programming languages and libraries I still think we haven’t reached a comfortable point yet. Meanwhile the desktop processor market is moving towards multiple cores and only slight adjustments in the clock speed so it makes a lot of sense to make your apps multi-thread-able (if problem domain permits, that is).

While working on my highly asynchronous P2P core, I noticed a weird thing – the debugger would complain here and there that a collection was changed while iterating through it, even though I’ve protected all access to a given collection with a lock. Here’s a code sample:

lock(coll.SyncRoot)
{
  foreach(SomeObj so in coll)
    so.SomeMethod();
}

I double-checked that:

• coll is a private member of a class it belongs to and is not exposed in any way to the outside world
• all method calls that have code like above are being called from different threads (generally a random thread from a thread pool)
• all access to coll is protected with a lock

So what could be the problem? Well, turns out I did not really check the second bullet right. It was from the SomeMethod that I called back into a class that contained coll and unfortunately into one of few methods that modify coll, so the debugger was right (as usual). So when I called SomeMethod, it (sometimes, rarely – that’s why it was hard to spot) called another method which tried to acquire the same coll.SyncRoot lock, I am still on the same thread from which I already acquired the lock and the lock succeeds – collection gets modified and on the next MoveNext (iteration step) the debugger throws an exception.

The moral of the story – beware of reentrancy of locks and always double, hell, triple check the second bullet.

While developing my (BitTorrent based) P2P product, I have stumbled upon a technique that should help shape the network traffic so that the application does not slow down (more than it does by its share of throughput) overall Internet access or jeopardize other network services. It is a well known fact that P2P traffic can be quite heavy and sometimes even bring to its knees network equipment. Thus any effort that could help shape the traffic should be welcomed.

Most of the other BitTorrent based P2P applications decided to use the Type Of Service of the IP header. That seems to work well according to reports from individuals who were capable of using this to regulate their networks so I decided to implement this in my app too. It’s only a single call to set this, once for incoming sockets and once for outgoing.

But nothing is as it seems – the TOS cannot be set by default on all post-Windows NT4 machines. Before anyone starts talking about Microsoft’s secret evil plans, it’s nothing of that sort – turns out Microsoft is following the newer “spec” (I put it in quotes because we are talking about RFCs here) that obsoletes usage of TOS bits. More details can be found in a support article Q248611.

That said, we all know that specs are one thing and reality another. Setting TOS does in fact work on today’s equipment (again, I don’t have hard data but people with no reason to lie are reporting success on several P2P forums) and it would be nice if one could set it.

Not surprisingly, Microsoft does provide a workaround – there is a registry setting that does the trick and it is detailed in the support article I linked above. Note that you must restart your machine before the setting will be effective. Unfortunately, this also means that users of my app will have to do the same.

War driving used to be fun – wherever* you go, you could find dozens of unprotected wireless networks to connect to. Technology was immature, standards still being shaped out, home network appliances buggy and people were not informed well about the dangers of unprotected networks.

Not any more. I live in a small town in the south of France called Antibes-Juan Les Pins about half way between Nice and Cannes. Few hundred meters from the sea, nice climate and surroundings, the city is a magnet for elderly to retire in. There are young people here of course, but not as many as is an average. Thus I was surprised to discover four to five other wireless networks when I booted up my laptop this morning. Note that I live in the small building and my bedroom (my new office ) faces north and looks into a large garden (so no other buildings around me, at least not close).

But even greater surprise was to find all four other networks on the screenshot protected while two of them were protected properly using WPA (the other two most likely used WEP which can be hacked, but works against free riders and encrypts the traffic). Apparently the awareness of the nature of the wireless networks protection as well as quality of home equipment has reached a level decent enough so that any consumer can set her network up properly.

Great for the wireless network owners, not so for those looking for free wireless Internet

*Assuming you are driving through a relatively urban area.

I have been rather disappointed with most of the EAP builds of the Resharper 2.0 to the point that I’ve been considering moving to another add-in with similar functionality.

The worst was slowness of Visual Studio 2005 with R# installed and really, really weird coloring (which is a bug in Visual Studio 2005, but nevertheless). I am one of those that can’t work productively without fully colored source code (using my own color scheme).

Build after build, I’ve been installing EAP Resharper 2.0 (into a virtual machine) and immediately uninstalling it.

Not any more. Just this morning I received a notification about a beta release and decided to try it on my production machine. The coloring bug has been fixed via hotfix from Microsoft and besides from a few irritating R# crashes that went away after I restarted Visual Studio 2005, it works as good as 1.5 did when I used it with Visual Studio 2003!

This is great news. What I like about Resharper the most is all those small assistants that pop up constantly reminding you about unused members, redundant usings, redundant initializations etc. Fixing all these means more readable code and as we all know code is read many times and only written once.

There is only one really strange thing – once R# crashes, you get a dialog detailing what happened and an opportunity to send the call stack to the authors. However, you must be a member of their developer network (or whatever they call it) in order to do so?!? Come on guys, I am helping you with the bug report, don’t make me register in order to help you. Take a look at how Microsoft is doing this – all crash submissions are completely anonymous - does it really matter to you who I am?

Before I begin, a disclaimer – I am getting a free copy of the software library I am about to mention. I normally don’t fall for this stuff (companies giving free licenses to bloggers) but in this case I am making an exception for two reasons – it’s a fellow µISV and the product looks really interesting.

Have you ever tried developing a shell extension, in .NET or not? It really is complicated, mainly because of all the registry poking and a myriad of COM interfaces. Unfortunately, many companies decided that it wasn’t so tricky after all which only led to a number of badly implemented extensions that would bog down your machine, sometimes even crash the shell (which would subsequently be blamed on Microsoft’s “crappy OS”). Yet it would be really great to be able to customize some aspects of the shell, like taskbar bands, Internet Explorer toolbands, customized Windows Explorer right-click menu, custom file icons generated on the fly, custom picture for the tile view, custom icon overlay, custom column extension… and the list goes on and on.

All this is offered in a single package called EZShellExtensions.Net, already in version 2.0, from the company Sky Software. Looking at the price for what you’re getting (about $99 without support and $149 with 1 year support), it’s actually quite reasonable, assuming everything works as advertised. You get integration in all Visual Studio versions from 2002 onward with skeleton code generated for you for each kind of extension. It looks really easy and you can develop in any .NET language.

On a first look, I thought – why would I need this? But then while skimming through the list of features, I found several different shell extensions that would fit quite nicely into my future product. You might want to give it a look at least – you never know, it might give you ideas too.

We’ve all been in this situation – you need to make a simple code generation step during the build, most likely to generate some code from XML, but you want to do it the simplest way possible.

If you search the Web, you will run into several articles and blog posts, all of which will tell you to make your own custom tool, register it with Visual Studio and you’re good to go. That is true, but it’s complicated – you need to learn a little bit about Visual Studio extensibility model, build a library and poke the registry to integrate it correctly with Visual Studio. But what if you just want to run the external tool and be done with it?

There is a way, and it is only possible because of the new build system in Visual Studio 2005. The best feature of this new system called MsBuild is that you can open any project file (they are all written using MsBuild markup), edit it manually and still open it from Visual Studio later (assuming the file is valid according to schema).

Note that if you are building an ASP.NET application, you can ignore what follows because there is a better way of doing custom builds called the build providers.

Let’s state the goal first – I have a code generation task to do: one XML file gets translated to C# via custom XSLT script. I can run the script from the command line using Oleg Tkachenko’s most excellent nxslt2 utility. I also want to see the generated file as a dependency of the XML file, like on the picture to the right. The generated file has a suffix Designer just like most of the generated files that Visual Studio 2005 supports. You can also see a few other supporting files – XSD (schema for the XML) and XSLT script that actually performs translation.

So, how does it work? First, just add XML, XSD and XSLT files to the project. Then, close (unload) the project and open it as an XML file and add the following:

<!— inside the ItemGroup where other files are —>
<Compile Include="Connection.Designer.cs">
  <AutoGen>True</AutoGen>
  <DependentUpon>Connection.xml</DependentUpon>
</Compile>
<!— just prior to Import node, for example —>
<Target Inputs="Connection.xml;Connection.xsd;Connection.xslt" Outputs="Connection.Designer.cs" Name="Codegen">
  <Exec Command="nxslt2 Connection.xml Connection.xslt -o Connection.Designer.cs" />
</Target>

The first block describes the Connection.Designer.cs as an auto-generated file (AutoGen tag, technically not necessary but I put it in for completeness) and dependent upon Connection.xml (DependentUpon tag). Thanks to the latter tag you get nice visual dependency in the tree control as on the picture above.

The second block describes actual build task and states: if any of the files Connection.xml, Connection.xsd, or Connection.xslt changes, execute the command “nxslt2 …”. The tasked is called Codegen but can be anything you want.

The only thing left is to add the task to the list of tasks that will be built by default:

<Project DefaultTargets="Codegen;Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Important note: even though it looks like a better place would be to put the task in the InitialTargets attribute of the Project element, that does not work and causes extremely weird behavior – all of the sudden all of the projects in the solution started referencing the generated file as if it was a module (?!?). This looks like a bug (but I didn’t bother checking with Microsoft forums or bug database).

Let’s get this out of the way – I think exceptions are better error handling technique than error codes. There – I’ve said it. Even though I respect Joel and his writing is most excellent, I disagree with all of his posts that denounce exceptions in favor of error codes.

That said, exceptions are not the silver bullet for the error handling, but then again nothing ever is. This posts’ goal is to help with a small annoyance that you can run into when using a library that throws the same 2–3 exceptions most of the time, you call it on many places and handle exceptions in the same way all the time.

Take Socket class for example – most of its methods will throw SocketException or ObjectDisposedException. If you have dozens of calls to any of these methods, your code will look like this

try
{
  byte[] buff = new byte[1024];
  s.Receive(buff);
}
catch(SocketException se)
{
  // do some cleanup

}
catch(ObjectDisposedException ode)
{
  // do (the same or) some other cleanup
}

Repeat, ad nauseam. Isn’t there a better way to handle this? There is, but note one important thing: the cleanup code is always the same (as comments state) for each exception handled, otherwise what follows does not help.

If you take a good look at all these calls, you will notice that the common part is what is inside the try block, while the rest is identical. So the problem boils down to capturing the lines of code in the try block. This is easy to achieve using an anonymous method. Code first, discussion later.

internal delegate void VoidMethodNoParams();
internal static void CommonSocketMethod(VoidMethodNoParams vmnp)
{
  try
  {
    vmnp.Invoke();
  }
  catch (SocketException se)
  {
    // handle socket exception
  }
  catch (ObjectDisposedException ode)
  {
    // handle object disposed exception
  }
}

// the above code can be used like this (assuming the method is in class Helper)
Helper.CommonSocketMethod(
delegate {
           byte[] buff = new byte[1024];
           s.Receive(buff);
         });

That's it! Take few lines of code, enclose them in curly brackets, put keyword delegate in front and you’ve got a suitable method. Another .NET 2.0 feature is at play here too – you don’t have to create a delegate and then assign a method (including anonymous methods) to it – all of it can be inferred from your code. Even better, if your app is a WinForms app, there is already a delegate with no return value that accepts no parameters called MethodInvoker so you don’t have to declare your own (like I did with VoidMethodNoParams).

Disclaimer: this is not a technique to help you to generically ignore exceptions. It is only useful as a way to clean up the code when the exception handling code repeats many times. In all other cases, you should have “normal” try/catch/finally blocks.

I have been avoiding using the simplified Control Panel in Windows XP probably out of habit – as an old time Windows user, a single/flat list view of all icons suits me well.

But I must admit that simplified version (the one you get by default) is really useful for the first time (or inexperienced) users – it allows them to navigate through the UI as if it was a tree view of choices.

Before someone starts rambling about how tree views are inferior to tagged lists, think about this – what if the elements of the tree/list are not known to you? You’d probably like better a simplified tree that would guide you to the right choice.

One day, when user starts to know most of the choices by hart, the tree/simplified navigation becomes nuisance. Then you need the choice to switch to the flat list because that becomes faster to navigate.

It is miraculous how Joel Spolsky can turn a topic everyone knows about into a very interesting piece. His The Development Abstraction Layer lays out the situation most of us have been, are now, or will be in the (near) future. How many developers are really aware of the infrastructure needed to run a company?

Anyway, how does all this apply to µISVs? The whole point of µISV is not to have the (comfy) abstraction layer and to handle all that yourself. Is it possible and if yes, how? Let’s see…

True µISV consists of a single developer/manager/boss and maybe one assistant, usually a spouse. The office is the bedroom, or the spare room, or the attic. We build our stuff on a computer, and it is very easy to buy one with a decent maintenance contract. Getting a fast Internet connection has never been easier and the uptime of one is generally very high for a very low cost. Since you work at your place, you can organize things like quiet office and adequate ambient temperature yourself.

What about graphic designers and testers? The former can be outsourced, while the latter unfortunately can’t (at least it’s harder to get the right person that’ll do the job well) – you’ll have to do the testing yourself. Marketing you don’t want to outsource as it’s a primary component of your business, but yes, there’s nobody to do it but yourself.

As long as you have a partner to do the tech support and misc administrative things, you should be just fine assuming you don’t hate marketing and testing. Selling is easy, once you’ve made yourself visible – there are zillion of payment processors out there and shipping software is trivial over the Internet.

That said, look what happened to the few µISVs that got absorbed by NewsGator (Bradbury Software for example) – they seem to agree that while µISV model was great for them, having someone else to take care of “the rest” and leaving them to do just the product development has been even better.

As long as you are aware that the abstraction layer exist and that you can’t ignore it you’ll be fine running a µISV. However, if you’re one of those that see the management, marketing, sales, tech support and administration as “overhead”, think again about establishing your own company. There’s more to product development than meets the eye of the average software developer, because the product is much more than just the source code/technology.

If you haven’t seen “Omega Man” or have read “I Am Legend” (from Richard Matheson), please do so. The book is great and the movie captures the spirit, even if it is a bit naive from today’s perspective (but it gave me the creeps when I saw it as a little kid). There’s even an announcement that there will be a remake in 2007, which if executed well would be great. Of course, there’s still a chance they’ll turn it into an action movie with Arnold Schwarzenegger

Anyway, the book and the movie are about the futuristic world where everyone contracts a deadly disease and only few “normal” humans remain. The basic premise is that, in a situation like that, it’s the majority that’s normal and not you.

Well, that’s exactly how I felt in a corporate environment. Politics, sucking up to your superiors, useless meetings… Everything but honest work and results counts. Even if that was not normal, if you don’t fit into that, it’s you who is not normal.

I am happy to report that I no longer work for “The Man” and that I am from the beginning of April officially going independent . As hinted on this blog before, I will create a small company and launch my own product, but in the meantime will take occasional contract work to keep the cashflow relatively steady.

It feels a bit strange to stay at home every day, but I can at least now dedicate all of my time to my product. It is essential not to change the working habits (like getting up at exactly the same time every day) but since I’ve been in the situation like this before, I know what to (and what not to) do.