Overview
There are two alternative ways of executing a given piece of code concurrently:
- Create a thread specifically for that piece of code and call the code from its entry point (heavy-weight)
- Generate tasks to be executed by a thread pool (light-weight)
What are the benefits of each type of execution?
As discussed in The Language Structure, CLIP provides active objects that wrap your sequential code and allow it to be executed at the right time and by the most appropriate resource. Two different active objects are provided in which you can place your processing code and it is very important that you understand the difference between them and select the most appropriate one in order to achieve optimal performance.
Heavy-Weight Execution
Creating an operating system thread for a concurrently executing object is a heavy-weight approach because each thread has significant resource requirements. Threads require allocation of a stack and they introduce a processing overhead as they are scheduled/suspended due to their context (stack, registers etc.) being switched in and out. When large numbers of threads are created, the context switching overhead can become significant.
The main benefit of the heavy-weight approach is that it is possible to make a blocking call without unnecessarily stalling execution as the thread is dedicated to just one task.
Light-Weight Execution
Thread pools are much more efficient because they only allocate a small number of threads (typically one per core at each priority). Concurrently executing object code is processed as a task by the thread pool and it can generate additional tasks to be added to the thread pool's task queue(s).
When using light-weight execution, care must be taken to ensure that additional threads are started when making blocking calls or blocking calls are avoided, otherwise execution may stall.
How are they supported in CLIP?
Threads
The Thread is a heavy-weight execution object because every instance of a thread creates a real operating system thread for each element (i.e. a single N-dimensional thread creates N O/S threads). You provide custom sequential code for the thread object and this code is executed by the thread on circuit activation. The thread of execution is completely independent of other activity within the program and therefore the thread can block indefinitely without unexpectedly stalling other unrelated activities. Typically a thread will consist of a main loop, which waits on a particular input and when it arrives, does some processing and then waits for more input again.
It is important to note that each O/S thread can consume a significant amount of resources in terms of memory usage and context-switching overhead and therefore it is advisable to use them only where necessary.
Methods
The Method is a light-weight execution object. Unlike the thread, each method is not designated its own operating system thread but rather is scheduled execution time within a thread pool. This means that an almost limitless number of asynchronously executing entities can be described (releasing very high concurrencies) without a huge resource overhead.
In many respects, methods can be considered to be the same as threads. They execute asynchronously and preemptively and they can block on CLIP objects. However, there are two important differences to note:
- Methods execute on arrival of a designated trigger event rather than on circuit activation
- Methods should not block outside of CLIP because their worker thread will be stalled and CLIP cannot know to start another worker
The trigger mechanism for a method means that it can remain idle without consuming any resources until it is required to run, in contrast to a thread that must block an operating system thread and wait to be signaled.
Blueprint provides two types of method icons. The first type has a single trigger connection point and the second has two. In the latter case the method implicitly collects both trigger connections. This type is referred to as a compound method and there are two cases of compound methods, each with a separate icon. The first collects sequentially and the second collects simultaneously. The three cases are shown below. The explicit circuitry equivalents of the two compound collector cases (M2 and M3) are also shown.
How do you decide which execution type is appropriate?
The decision on whether to use a heavy or light-weight execution is usually fairly straightforward. You should always look to use a light-weight method unless:
- The code needs to block outside of CLIP (e.g. blocking on a hardware device call or using Sleep() or equivalent)
The most common use of threads is to wait on input from a device and write it to a data store. In this case it may be tempting to incorporate additional processing into the thread code but this should be avoided because servicing the device should always take priority over subsequent processing in order to ensure timeliness.
Examples
The example circuit below shows a towed array sonar's Tracker sub-circuit.
In this circuit the NBTrk method has dimensions NB, which might typically be set to a value of 64. Instead of creating 64 threads to implement NBTrk, this circuit creates 64 light-weight method elements, whose tasks are processed by a thread pool with one thread per core for each priority:
