Some people have started just recently thinking about how multi-core CPUs will change the way we developers create applications. Me, I’ve realized it after the article The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software (published at the beginning of 2005) by Herb Sutter, a C++ guru and an architect at Microsoft. Definitely worth a read and some pondering.

In order to use the multiple cores we have and will have the need to start executing code on more than one thread at a time. But this does not come for free – now all the objects/resources accessed from multiple threads need to be protected from access from more than one thread at a time. To do that, all Operating Systems and all programming languages offer constructs for locking.

A detailed analysis and a good tutorial on the .NET (C#) threading is here.

A good example on how not to implement thread safety is Java’s Vector. Just do a Google search on “Java slow Vector” and you’ll see what I mean. Most of the methods in Java’s Vector are synchronized, thus an expensive lock is taken on each call. There is a great two-line snippet in the .NET tutorial linked above that shows why this is not a good idea (the intent is to inset element in a list only if the list is empty):

if (!myList.Contains (newItem)) myList.Add (newItem);

If the thread gets preempted between if expression and the block, the list can still contain some elements because another thread can add them. It does not matter that Contains and Add could (in a naive thread-safe implementation) both be internally protected with a lock. Differently put, the thread safety of collections cannot be implemented properly in the collections library, but by collection user – that’s you.

If you’ve successfully ignored the multi-threading issues, it’s probably time to start improving your skills in this area. 4–core CPUs should be available as soon as this year with 8–core and 16–core CPUs just around the corner.

The free lunch is over indeed.