The raw speed of the processor hasn’t significantly increased in years. The efficiency increases, slowly, but there’s no “free lunch” as it was before – getting a 200MHz processor used to get you twice the speed of the 100MHz processor. If you want better performance today, you go for a multi-core processor (Intel has recently launched a consumer-level four core processor).

For an app twice as fast as before, without a double-clock-speed at your disposal, you resort to parallelism, which is a hard topic. If I have a four core processor, I’d want to run my code on all four cores at once, if possible (note that not all problems can be parallelized). The way programming languages and processors expose parallelism is through threads.

Now, we are all used to working with threads in this way or another. Me, I’ve been doin’ some threadin’ mostly in C++ and C#. In C++, it was a major PITA if you did not have at least boost at your disposal. With C#/.NET, it’s mostly been a completely transparent experience. In some cases, you could be using threads and thread pools even without knowing it (asynchronous delegates are a great example).

Having this kind of background, I expected no real surprises in the chapter 20 of the very good Ruby Cookbook. But there were quite a few .

First of all, if you know some Ruby, but would like to improve your knowledge and you like to learn by example, get this book. It’s chock full of code snippets for almost any kind of real-life scenario. You don’t have to read it in order and can jump straight to, well, chapter 20 if you like

Section 7 talks about the thread pools. Thread pools are relatively easy beasts to comprehend, having in mind the following facts:

• running T threads on a single core processor will not make your code faster
• running T threads, where each is blocked most of the time (waiting for something, like file read or a database access) assuming each threads takes about X seconds to complete will in general execute in X + overhead seconds (overhead is relatively small), compared to T times X seconds if the code was run sequentially
• starting a thread is generally cheap, unless you do it all the time – there is a cost to setting the thread up
• therefore, it’s beneficial to pre-create P threads, keep them in a pool (just a fancy name for a group of threads) and then schedule work to each of the threads in a pool

The classic use case for a thread pool is a Web server. Most of the time, there will be either less than P requests (where P is the size of the thread pool) or, during short bursts, at most N times P requests (N relatively small). Usually, the requests will be I/O bound, most threads will wait for the disk, and the tasks will eventually execute.

Having all this in mind, I was shocked to see the thread pool example given in the book (source code is freely available from the web site above). In short, the scheduling looks like this:

class ThreadPool
  # ...
  def dispatch(*args)    
    Thread.new do
      # Wait for space in the pool.
      # ...
    end
    # ...
  end
  # ...
end

I omitted most of the code for clarity. What’s wrong with this picture? Well, for each new task/job/work item (names are used as synonyms in most threading texts), the class first creates a new thread, then waits for the “space in the pool”. For a 10 thread pool, if there is a short burst of activity, and you have 100 tasks to schedule, 100 threads will be created, of which only 10 at most will execute at any point in time.

This is obviously a different definition of a thread pool. Any programming book will tell you not to go overboard with threads and to create as few as is necessary. Why is the example written this way then? It’s definitely not because the book authors are incompetent as many great code examples from the book clearly show. I must be missing something…

Suddenly I realized that I recently read about Ruby providing green threads. Green thread is a term used to describe a “fake” thread, something that does work as a thread but does not use underlying OS thread facility. Note that in .NET for example, it is not guaranteed that a .NET thread will correspond to a real OS thread, even though it does at this point in time. Some Java JVMs allow you to choose between native and green thread approaches.

Ruby apparently provides green thread model only. This is very easy to confirm with the following short code snippet:

require 'thread'

def spawn(*args)
  Thread.new do
    yield(*args)
  end
end

1.upto(100) do |i|
  spawn(i) do
    puts "Running thread #{i}..."
    sleep(10)
    puts "Finished thread #{i}"
  end
end

sleep(15)

(I simplified the code by putting a longer sleep at the end, what you should do is join each of the threads and wait for their completion). Run this and if you are on Windows, have a look at the number of threads of this small script in Process Explorer. It’s 1 (one)!

This could imply that thread creation is cheap, which would explain the approach as given in the book. But it still goes against my instinct as a developer – it can’t be that cheap, at least you’d waste some memory, right? I don’t have fancy memory profiling tools, but assuming that Ruby’s garbage collector runs between the time all of the threads finish and the last sleep finishes (5 seconds gap) you should see the rough memory cost of the creation of 100 threads. Let’s see…

When all 100 threads are running, the working set of the process is 4904KB (running on Windows XP, on a 32bit Intel processor). When all of the threads exit, the memory consumption drops to 3464KB. The numbers are identical over many runs of the test. If I increase the number of threads significantly, both numbers increase significantly (for a 1000 threads, before and after numbers are ~20MB and ~9MB respectively).

So 100 threads cost you 1440KB, roughly. Note that there are many blocks created and it’s quite possible that it’s the block creation that costs memory, not threads. The testing methodology is obviously not scientific

Tomorrow, I’ll build a traditional thread pool and we’ll see if it makes any difference. To reiterate, in a traditional thread pool, only P threads are created and then work is queued and scheduled to run on one of the threads in a pool.