Operating System Scheduling On Multi-core Architectures

Seminar “Parallel Computing“

Summer term 2008

Seminar paper Parallel Computing (703525)

Optimisation: Operating System Scheduling on multi-core architectures

Lehrveranstaltungsleiter: T. Fahringer

Name Thomas Zangerl

Matrikelnummer

1

Seminar “Parallel Computing“

Summer term 2008

Abstract As multi-core architectures begin to emerge in every area of computing, operating system scheduling that takes the peculiarities of such architectures into account will become mandatory. Due to architectural differences to traditional multi-processors, such as shared caches, memory controllers and smaller cache sizes available per computational unit, it does not suffice to simply schedule tasks on multi-core processors in the same way as on SMP systems. Furthermore, current research motivates architectural changes in CPU design, such as multicore processors with asymmetric core-performance and so called many-core architectures that integrate up to 100 cores in one package. Such architectures will exhibit a fundamentally different behaviour with regard to shared resource utilization and performance of nonparallelizable code compared to current CPUs. It will be the responsibility of the operating system to spare the programmer as much platform-specific knowledge as possible and optimize overall performance by employing intelligent and configurable scheduling mechanisms.

2

Seminar “Parallel Computing“

Summer term 2008

Abstract......................................................................................................................................2 1. Introduction............................................................................................................................4 1.1 Why use multi-core processors at all?..............................................................................4 1.2 What’s so different about multi-core scheduling?............................................................5 2. OS process scheduling state of the art...................................................................................7 2.1 Linux scheduler................................................................................................................7 2.1.1 The Completely Fair Scheduler ................................................................................7 2.1.2 Scheduling domains...................................................................................................7 2.2 Windows scheduler..........................................................................................................9 2.3 Solaris scheduler.............................................................................................................10 3. Ongoing research on the topic.............................................................................................11 3.1 Cache-Fairness................................................................................................................11 3.2 Balancing core assignment..............................................................................................12 3.3 Performance asymmetry.................................................................................................13 3.4 Scheduling on many-core architectures..........................................................................15 4. Conclusion...........................................................................................................................18 5. References............................................................................................................................20

3

Seminar “Parallel Computing“

Summer term 2008

1. Introduction 1.1Why use multi-core processors at all? In the last few years, multi-core CPUs have become a standard component in nearly all sorts of computers – not only servers and high-end workstations but also desktop and laptop PCs for consumers and even game consoles nowadays usually come with CPUs with more than one core. This development is not surprising; already in 1965 Gordon Moore predicted that the number of transistors that can be cost-effectively built onto integrated circuits were going to double every year ([1]). In 1975, Moore corrected that assumption to a period of two years; nowadays this period is frequently assumed to be 18 months). Moore's projection has more or less been accurate up to today and consumers have gotten used to the constant speedup of computer hardware – it is expected by buyers that a new computer shows a significant speedup to a two years older model (even though an increase in transistor density does not always lead to an equal in increase in computing speed). For chip manufacturers, it has become increasingly difficult however to keep up with Moore's law. In order to implement the exponential increase of integrated circuits, the transistor structures have to become steadily smaller. On the one hand, the extra transistors were used for the integration of more and more specialized instruction sets on CISC chips. On the other hand, smaller transistor sizes led to higher clock rates of the CPUs, because due to physical factors, the gates in the transistors could perform faster state switches. However, since electronic activity always produces heat as an unwanted by-product, the more transistors are packed together in a small CPU die area, the higher the resulting heat dissipation per unit area becomes ([2]). With the higher switching frequency, the electronic activity was performed in smaller intervals, and hence more and more heat-dissipation emerged. The cooling of the processor components became more and more a crucial factor in design considerations and it became clear, that the increasing clock frequency could no longer serve as the primary reason for processor speedup. Hence, there had to be a shift in paradigm in order to still make applications run faster; on the one hand the amazing abundance of transistors on processor chips was used to increase the cache sizes. This alone, however, would not result in an adequate performance gain, since it only helps memory intensive applications to a certain degree. In order to effectively counteract the heat problem while making use of the small structures and high number of transistors on a chip, the notion of multi-core processors for consumer PCs was introduced. Since the CMOS technology met its limits for the further increase of CPU clock frequency and the number of transistors that could be integrated on a single die allowed for it, the idea emerged, that multiple processing cores could be placed in a single processor die. In 2006, Intel released the Core™ microprocessor, a die package with two processor cores with their own level 1 caches and a shared level 2 cache ([3]). Also in 2006, AMD, the second major CPU manufacturer for the consumer market, released the Athlon™ X2, a processor with quite similar architecture to the Core platform, but additionally featuring the concept of also sharing a CPU-integrated memory-controller among the cores ([[4]).

4

Seminar “Parallel Computing“

Summer term 2008

Both architectures have been improved and sold with a range of consumer desktop and laptop computers - but also servers and workstations - up to today; therefore the presence of multicore processors in a large number of today's PCs can be assumed.

1.2What’s so different about multi-core scheduling? One could assume that the scheduling process on such multi-core processors wouldn’t differ much from conventional scheduling – intuitively the run-queue would just have to be replaced by n run-queues, where n is the number of cores and processes would simply be scheduled to the currently shortest run-queue (with some additional process-priority treatment, maybe). While that might seem reasonable, there are some properties of current multi-core architectures that speak strongly against such a naïve approach. First, in many multi core architectures, each core manages its own level 1 cache (Figure 1). By just naïvely rescheduling interrupted processes to a shorter queue which belongs to another core (task migration), parts of the processes cache working set may become unnecessarily lost and the overall performance may slow down. This effect becomes even worse if the underlying architecture is not a multi-core but a NUMA system where memory access can become very costly if the process is scheduled on the “wrong” node.

Figure 1: Typical multi-core architecture

A second important point is that the performance of different cores in a multi-core system may be asymmetric regarding the performance of the different cores ([5]). This effect can emerge due to several reasons:

5

Seminar “Parallel Computing“ •

•

•

Summer term 2008

Design considerations. Many slow cores can be used for increasing the throughput in parallel computation while a few faster cores contribute to the efficient processing of costly tasks which can not be parallelized. ([6]). Even algorithms that are parallelizable contain parts that have to be executed sequentially, which will benefit from the higher speed of the fastest core. Hence performance-asymmetry has been shown to be a very efficient approach in multi-core architectures ([7]). Transistor failures. Some parts of the CPU may get damaged over time and become automatically disabled by the CPU. Since such components may fail in certain cores independently from the other cores, performance asymmetry may arise in symmetric cores over time ([5]). Power-saving policies. Different cores may switch to different P- or C-power-states at different times in order to save power. At different P-states, equal cores show a different clock-frequency. If an OS scheduler manages to take this into account for processes not in need of all system resources, the system can remain more energyefficient over the execution time while giving away only little or no performance at all. ([8]).

Hence, performance asymmetry, the fact that various CPU components can be shared among cores, and non-uniform access to computation resources such as memory, mandate the design of efficient multi-core scheduling mechanisms or scheduling frameworks at the operating system level. Multi-core processors have gone mainstream and while there may be the demand that they are efficiently used in terms of performance, the currently fashionable term Green-IT also motivates the energy-efficient use of the CPU cores. Section 2 will explore how far current operating systems have evolved in support of the new architectures.

6

Seminar “Parallel Computing“

Summer term 2008

2. OS process scheduling state of the art 2.1 Linux scheduler 2.1.1The Completely Fair Scheduler The Linux scheduler in versions prior to 2.6.23 performed its tasks in complexity O(1) by basically just using per-CPU run-queues and priority arrays ([9]). Kernel version 2.6.23, which was released on October 9 2007, introduced the so-called completely fair scheduler (CFS). The change in scheduler was mainly motivated by the failure of the old scheduler to correctly predict whether applications are interactive (I/O-bound) or CPU-intensive ([10]). Therefore the new scheduler has completely abandoned the notion of different kinds of processes and treats them all equally. The data-structure of a red-black tree is used to align the tasks according to their “right” to use the processor resources for a predefined interval until context-switch. The process positioned at the leftmost node in the data structure is entitled most to use the processor resources at the time it occupies that position in the tree. The position of a process in the tree is only dependent on the wait-time of the process in the runqueue (including the time the process is actually waiting for events) and the process priority ([11]). This concept is fairly simple, but works with all kinds of processes, especially interactive ones, since they get a boost just by getting account for their I/O-waiting time. However, the total scheduling complexity is increased to O(log n) where n is the number of processes in the run-queue, since at every context-switch, the process has to be reinserted into the red-black tree. The scheduling algorithm itself has not been designed in special consideration of multi-core architectures. When Ingo Molnar, the designer of the scheduler, was asked, what the implications on HT/SMP/NUMA architectures would be, he answered, that there would inevitably be some effect, and if it is negative, he will fix it. He admits that the fairnessapproach can result in increased cache-coldness for processes in some situations. ([12]). However, the red-black trees of CFS are managed per runqueue ([13]), which assists in cooperation with the Linux load-balancer.

2.1.2Scheduling domains Linux load-balancing takes care of different cache models and computing architectures but at the moment not necessarily of performance asymmetry. The underlying model of the Linux load balancer is the concept of scheduling domains, which was introduced in Kernel version 2.6.7 due to the unsatisfying performance of Linux scheduling on SMP and NUMA systems in prior versions ([14]). Basically, scheduling domains are hierarchical sets of computation units on which scheduling is possible; the scheduling domain architecture is constructed based on the actual hardware resources of a computing element ([9]). Scheduling domains contain lists with scheduling groups that share common properties.

7

Seminar “Parallel Computing“

Summer term 2008

For example, the way scheduling should be done on two logical processors of a HT-systems and two physical processors of a SMP system is different; the HT “cores” share a common cache and memory hierarchy, therefore task migration is not a problem, if one of the logical cores becomes idle. However, in a SMP or multi-core system, in which the cache or parts of the cache is administrated by each core separately, migrating tasks with a large working set may become problematic. This applies even more to NUMA machine, where different CPU may be closer or more remote to the memory the process is using. Therefore all this architectures have to be treated differently. The scheduling domain concept introduces scheduling domains, a logical union of computing resources that share common properties, with whom it is reasonable to treat them equally and CPU groups within these domains. Those groups contain hardware-addressable computing resources that are part of the domain on which the balancer can try to even the domain load out. Scheduling domains are hierarchically nested – there is a top-level domain containing all other domains of the physical system the Linux is running on. Depending on the actual architecture, the sub-domains represent NUMA node groups, physical CPU groups, multicore groups or SMT groups in a respective hierarchical nesting. This structure is built automatically based on the actual topology of the system and for reasons of efficiency each CPU keeps a copy of every domain it belongs to. For example, a logical SMT processor that at the same time is a core in a physical multi-core processor on a NUMA node with multiple (SMP) processors would totally administer 4 sched_domain structures, one for each level of parallel computing it is involved in ([15]).

Figure 2: Example hierarchy in the Linux scheduling domains

Load-balancing takes place at scheduling domain level, between the different groups. Each domain level is sensitive with respect to the constraints set by its properties regarding load balancing. For example, load balancing happens very often between logical simultaneous 8

Seminar “Parallel Computing“

Summer term 2008

multithreading cores, but very rarely on the NUMA level, where remote memory access is costly. The scheduling domain for multi-core processors was added with Kernel version 2.6.17 ([16]) and especially considers the shared last level cache that multi-core architectures frequently possess. Hence, on a SMP machine with two multi-core packages, two tasks will be scheduled on different packages, if both packages are currently idle, in order to make use of the overall larger cache. In recent Linux-kernels, the multi-core processor scheduling domain also offers support for energy-efficient scheduling, which can be used if e.g. the powersave governor is set in the cpufreq tool. Saving energy can be achieved by changing the P- and the C-states of the cores in the different packages. However, P-states are transitions are made by adjusting the voltage and the frequency of a core and since there is only one voltage regulator per socket on the mainboard, the P-state is dependent on the busiest core. So, as long as any core in a package is busy, the P-state will be relatively low, which corresponds to a high frequency and voltage. While the P-states remain relatively fixed, the C-states can be manipulated. Adjusting the Cstates means turning off parts of the registers, blocking interrupts to the processor, etc. ([17]) and can be done on each core independently. However, the shared cache features its own Cstate regulator and will always stay in the lowest C-state that any of the cores has. Therefore, energy-efficiency is often limited to adjusting the C-state of a non-busy core while leaving other C-states and the packages’ P-state low. Linux scheduling within the multi-core domain with the powersave-governor turned on will attempt to schedule multiple tasks on one physical package as long as it is feasible. This way, other multi-core packages will be allowed to transition into higher P- and C-states. The author of [9] claims, that generally the performance impact will be relatively low and that the performance loss/power saving trade-off will be rewarding, if the energy-efficient scheduling approach is used.

2.2 Windows scheduler In Windows, scheduling is conducted on threads. The scheduler is priority-based with priorities ranging from 0 to 31. Timeslices are allocated to threads in a round-robin fashion; these timeslices are assigned to highest priority threads first and only if know thread of a given priority is ready to run at a certain time, lower priority threads may receive the timeslice. However, if higher-priority threads become ready to run, the lower priority threads are preempted. In addition to the base priority of a thread, Windows dynamically changes the priorities of low-prioritized threads in order to ensure “felt” responsiveness of the operating system. For example, the thread associated with the foreground window on the Windows desktop receives a priority boost. After such a boost, the thread-priority gradually decays back to the base priority ([21]). Scheduling on SMP-systems is basically the same, except that Windows keeps the notion of a thread’s processor affinity and an ideal processor for a thread. The ideal processor is the processor with for example the highest cache-locality for a certain thread. However, if the ideal processor is not idle at the time of lookup, the thread may just run on another processor. In [21] and other sources, no explicit information is given on scheduling mechanisms especially specific to multi-core architectures, though. 9

Seminar “Parallel Computing“

Summer term 2008

2.3Solaris scheduler In the Solaris scheduler terminology, processes are called kernel- or user-mode threads dependant on the space in which they run. User threads don’t only exist in the user space – whenever a user thread is created, a so-called lightweight process is set up that connects the user thread to a kernel thread. These kernel threads are object to scheduling. Solaris 10 offers a number of scheduling classes, which may co-exist on one system. The scheduling classes provide an adaptive model for the specific types of applications which are intended to run on top of the operating system. ([18]) mentions Solaris 10 scheduling classes for: • • • • • •

Standard (timeshare) threads, whose priority may be adapted by the scheduler. Interactive applications (the currently active window in the graphical user interface). Fair sharing of system resources (instead of priority-based sharing) Fixed-priority threads. The priority of threads scheduled with this scheduler does not vary over the scheduling time. System (kernel) threads. Real-time threads, with a fixed priority and time share. Threads in this scheduling class may preempt system threads.

The scheduling classes for timeshare, fixed-priority and fair sharing are not recommended for simultaneous use on a system, while other combinations of scheduling classes on a set of CPUs are possible. The timeshare and interactive schedulers are quite similar to the old Linux scheduler (before CFS) in their attempt of trying to identify I/O bound processes and providing them with a priority boost. Threads have a fixed time quantum they may use once they get the context and receive a new priority based on whether they fully consume their time quantum and on their waiting time for the context. Fair share scheduling uses a fixed time quantum (share) allocated to processes ([19]) as a base for scheduling. Different processes (actually collection of processes, or, in Solaris 10 terminology, projects) compete for quanta on a computing resource and their position in that competition depends on how large the value they have been assigned is in relation to the total quanta number on the computing resource. Solaris explicitly deals with the scenario, that parts of the processor’s resources may be shared, as it is likely with typical multi-core processors. There is a kernel abstraction called “processor group” (pg_t), that is built according to the actual system topology and represents logical CPUs that share some physical properties (like caches or a common socket). These groupings can be investigated by the dispatcher e.g. in order to maintain logical CPU affinity for the purpose of cache-hotness where it is reasonable. Quite similar to the concept of Linux’s scheduling domains, Solaris 10 tries to simultaneously achieve load balancing on multiple levels (for example if there are physical CPUs with multiple cores and SMT) ([20]).

10

Seminar “Parallel Computing“

Summer term 2008

3. Ongoing research on the topic Research on multi-core scheduling deals with a number of different topics, many of which are orthogonal (e.g. maximizing fairness and throughput). The purpose of this section is to present an interesting selection of different approaches to multi-core scheduling. Sections 3.1 and 3.2 summarize proposals to improve fairness and load-balancing on current multi-core architectures while sections 3.3 and 3.4 concentrate on approaches for scheduling on promising new computing architectures, such as multi-core processors with asymmetric performance and many-core CPUs.

3.1Cache-Fairness Several studies (e.g. [22], [23]) suggest that operating system schedulers insufficiently deal with threads that allocate large parts of the shared level 2 cache and thus slow-up threads running on the other core that uses the same cache. The situation is unsatisfactory due to several reasons: First, it can lead to unpredictable execution times and throughput and second, scheduling priorities may loose their effectiveness because of threads running on cores with aggressive “co-runners” (i.e. threads running on another core in the same package). Figure 3 shows such a scenario: Thread B uses the larger part of the shared cache and thus maybe negatively influences the cycles per instruction that thread A achieves during its CPU time share. L2-cache-misses are more costly than L1-cache-misses, because the latency to the memory is bigger than to the next cache-level. However, it is mostly the L2-cache that is shared among different cores. The authors of [22] try to mitigate the above mentioned effects by introducing a cachefairness aware scheduler for the Solaris 10 operating system.

Figure 3: Unfair cache utilization by thread B

In their scheduling algorithm, the threads on a system are grouped into a best effort class and a cache-fair class. Best effort threads are penalized for the sake of performance stability of cache-fair threads, if necessary, but not vide-versa. However, it is taken care, that this does not result in inadequate discrimination of best effort threads. Fairness is enforced by allocating longer time shares to cache-fair threads that suffer from cache-intensive co-runners at the expense of these co-runners, if they are best effort threads. Figure 4 illustrates that process.

11

Seminar “Parallel Computing“

Summer term 2008

Figure 4: Restoring fairness by adjusting timeshares

In order to compute the quantum that the thread is entitled to, the algorithm uses an analytical model to estimate a few reference values that would hold, if the thread had run under fair circumstances, namely the fair L2 cache miss rate, the fair CPI rate and the fair number of instructions (during a quantum). All those values are based on the assumption of fair circumstances, so the difference to the actual values is computed and a quantum extension is calculated which should serve as “compensation”. Those calculations are done once for new cache-fair class threads – their L2 cache miss rates and other data is measured with different co-runners. Subsequently, the dispatcher periodically selects best-effort threads from whom he takes parts of their quanta and assigns them as “compensation” to the cache-fair threads. New cache-fair threads are not explicitly scheduled with different co-runners in the analysis phase, but whenever new combinations of cache-fair threads with co-runners occur, analytical input is mined from the CPU’s hardware counters. The authors of [22] state that according to their measurements, the penalties on best-effort threads are low, while the algorithm actually enforces priority better than standard scheduling and improves fairness. These claims are supported by experimental data gathered using the SPEC CPU2000 suite on an UltraSPARC T1 processor. The experiments measure the time it takes a thread to complete a specific benchmark while a cache-intensive second thread is executed in the same package. The execution times under these circumstances are compared with the times of threads running with co-runners with low-cache requirements. This comparison shows differences of up to 37% in execution time between the two scenarios on a system with a standard scheduler, while using the cache-fair scheduler, the variability decreases to 7%. At the same time, however, measurements of the execution times of threads in the best-effort scheduling class reveal a slowdown of up to 8% for some threads (while some even experience a speed-up).

3.2Balancing core assignment Fedorova et al. ([27]) argue, that during operating system scheduling, several aspects have to be considered besides optimal performance. It is shown that scheduling tasks on the cores in an imbalanced way results in jittery performance (i.e. unpredictability of a tasks completion time) and, as a consequence, insufficient priority enforcement. It could be part of the operating system scheduler to ensure that the jobs are evenly assigned to cores. In the approach described in [27] this is done by using a self-tuning scheduling algorithm based on a per-core benefit function. This benefit function is based on three input components: 12

Seminar “Parallel Computing“

Summer term 2008

1) The normalized core preference of a thread, which is based on the instructions per cycle that a thread j can achieve on a certain core i ( IPC j ,i ), normalized by max( IPC j ,k ) (where k is an arbitrary CPU/core) 2) The cache-affinity, a value which is 1 if the thread j was scheduled on core i within a tuneable time period and 0 otherwise 3) The average cache investment of a thread on a core which is determined by inspecting the hardware cache miss counter from time to time This benefit function can then be used to determine whether it would be feasible to migrate threads from core i to core j. For each core, a benefit function Bi , 0 is computed that represents the case of no thread migrations taking place. For each thread k on a core, and updated benefit value for the hypothetical scenario of the migration of the thread onto another core Bi ,k − , is computed. Of course, the benefit will increase, since fewer threads are executed on the core. But the thread that would have been taken away in the hypothetical scenario would have to be migrated to another core, which would influence the benefit value of the target core. Therefore, also the updated benefit value of any other system core j to which the thread in question would be migrated to, has to be computed and is called B j ,k + . The hypothetical migration of thread k from core i to core j becomes reality if Bi ,k − + B j ,k + > Bi , 0 + B j ,0 + a * DCAB + b * DRTF . DCAB represents a system-wide balanceconstraint, while DRTF ensures a per-job response-time-fairness (i.e. the slowdown that results for the thread in question from the thread-migration does not exceed some maximum value). These two constants, together with the criterions included in the benefit function itself (most notably cache-affinity) should help to ensure that the self-tuning fulfils the three goals of optimal performance, core-assignment balance and response-time-fairness. However, the authors have not yet actually implemented the scheduling modification in Linux and hence the statements on its effectiveness remain somehow theoretical.

3.3Performance asymmetry It has been advocated that building multi-core chips with asymmetric performance of the different cores can have advantages for the overall processing speed of a CPU. For example, it can prove beneficial if one fast core can be used to speed up parts that can hardly be parallelized while multiple slower cores come to play when parallel code parts are executed. By keeping the cores for parallel execution slower than the core(s) for serial execution, diearea and cost can be saved ([24]) while power consumption may be reduced. [25] closely analyzes the impact of performance asymmetry on the average speedup of an application with increase of cores. The paper concludes, that “[a]symmetric multicore chips can offer maximum speedups that are much greater than symmetric multicore chips (and never worse)”. Hence, performance asymmetry at the processor core level seems to be a promising approach for future multi-core architectures. [26] suggests that asymmetric multiprocessing CPUs do exceptionally well for moderately parallelized applications, but don’t scale much worse with highly parallel programs. (see Figure 5)

13

Seminar “Parallel Computing“

Summer term 2008

Figure 5: Comparison of speedup with SMP and AMP using highly parallel programs (left), moderately parallel programs (middle) and highly sequential programs (right)1

Apart from explicit design, performance asymmetry can also occur in initially symmetric multi-core processors by either power-saving mechanisms (increasing the C-state) or failure of transistors that leads to disabling of parts of the core’s components. Problems arise by the fact that the programmer usually assumes symmetric performance of the processor cores and designs her programs accordingly. Therefore, the operating system scheduler should support processor performance asymmetry, which is currently not the case for most schedulers. However, it would be imaginable to see this as a Linux scheduling domain in the future. [5] describes AMPS, an approach on how a performance-asymmetry aware scheduler for Linux could look like. Basically the scheduler consists of three components: An asymmetryspecific load balancing, a faster-core first scheduler and a migration mechanism specifically for NUMA machines that will not be covered in detail here. The scheduling mechanism tries to achieve a better performance, fairness (with respect to the thread priorities) and repeatability of performance results. In order to conduct load-balancing, the core performance is assessed in a first step. AMPS measures core performance at boot time, using benchmarks, and sets the performance quantifier for the slowest core to 1 and for faster cores, to a number higher than 1. Then the scaled load of a core is the run-queue length of a core divided by its performance quantifier. If this scaled load is within a maximum and a minimum threshold, then the system is considered to be load-balanced, otherwise threads will be migrated. By the scaling factor, faster cores will receive more workload than slower ones. Besides the load-balancing, cores with lower scaled load are preferred in thread placement. Whenever new threads are scheduled, the new scaled load that would apply if the thread was scheduled to one specific core is computed. Then the thread is scheduled to the core with the least new scaled load; in case of a tie, faster cores are preferred. Additionally, the loadbalancer may migrate threads even if the source core of migration can become idle by doing it. AMPS only migrates threads, if the new scaled load of the destination core does not exceed 1

Image source: http://www.intel.com/technology/magazine/research/power-efficiency-0206.htm

14

Seminar “Parallel Computing“

Summer term 2008

the scaled load on the source-core. This way, the benefits of available fast cores can be fully exploited while not overburdening them. It can be expected, that frequent core migration results in performance loss by cache misses (e.g. in the level cache). However, experimental results in [5] reveal no excessive performance loss by the fact that task migration among cores occurs more often than in standard schedulers.

Figure 6: Speedup of the AMPS scheduler compared to Linux scheduler on AMP with two fast and six slow cores2

Instead, performance on SMP systems measurably improves (see Figure 6, standard scheduler speed would be 1, median speedup is 1.16); while fairness and repeatability is preserved better than with standard Linux (there is less deviation in the execution times of different processes).

3.4Scheduling on many-core architectures Current research points at the potential of future CPU architectures that consist of a multitude of different cores (tens to hundreds) in order to prolong the time the industry can keep up with Moore’s prediction. Such CPU architectures are going to require novel approaches in thread scheduling, since the available shared memory per core is potentially smaller, while main memory access time increases and single-threaded performance decreases. Even more so than with current CMP chips, schedulers that treat such large scale CMP architectures just like SMP systems, are going to fail with respect to performance. [28] identifies three major challenges in scheduling on many-core architectures, namely the comparatively small cache sizes which render memory access costly, the fact that nonspecialized programmers will have to program code for them and the wide range of application scenarios that have to be considered for such chips. The latter two challenges result from the projected general-purpose use of future many-core architectures. In order to deal with these challenges, an OS-independent experimental runtime-environment called McRT is presented in [28]. The runtime environment was built from scratch and is 2

Picture taken from [5]

15

Seminar “Parallel Computing“

Summer term 2008

independent of operating system kernels – hence the performed operations occur at user-level. The connection to the underlying operating system is established using so called host adapters while programming environments like pthreads or OpenMP can invoke the scheduling environment via client adaptors. For programmability it provides high level transactions (instead of locks) and the heterogeneity is alleviated by giving the user the choice among different runtime policies (which influence the scheduling behaviour). The overall target of the scheduler is to maximize resource utilization and provide the user with flexible scheduler configurability. Basically the scheduler is configured using three parameters P, Q and T which respectively denote the number of (logical) processors, task queues and threading abstractions. P and Q change the scheduling behaviour from strict affinity to work stealing. T can be used to specify different behaviour for different threads. This way, the concept of scheduler domains can be realized. It is notable, that the scheduling system does not use pre-emption; instead threads are expected to yield at certain times. This design choice has been motivated by the authors’ belief that pre-emption stems from the need of time-sharing expensive and fast resources which will become obsolete with many-core architectures. The runtime actively supports constructs such as barriers, which are often needed in parallel programming. Barriers are designed to avoid busy waiting – for example, a thread yields once it has reached the barrier but won’t be re-scheduled until all other threads have reached the barrier. With a pre-emptive scheduling mechanism, the thread would receive the context from time to time just to check whether other threads have reached the barrier – with the integrated barrier support based on a co-operative scheduling approach used in McRT, this won’t happen. The client-side adaptor, e.g. for OpenMP, promises to directly translate many OpenMP constructs to the McRT API. [28] also contains a large section on experimental results from benchmarks of typical desktop and scientific applications, such as the XviD MPEG4 encoder, singular value decomposition (SVD) and neural networks (SOM). The results were gathered on a cycle-accurate many-core simulator with 32 Kbyte L1 cache shared among 4 simultaneous multithreaded cores which form one physical core. 32 such physical cores share 2 Mbyte L2 cache and 4 Mbyte off-chip L3 cache. The simulator provides a cost-free MWait instruction that allows a thread to tell the processor that it is blocked and the resource it is waiting for. Only if the resource becomes available the CPU will execute the thread. Hence, threads that are waiting barriers and locks don’t consume system resources. It is important to consider that such a mechanism does not exist on current physical processors, when viewing the experimental speedup results for McRT

16

Seminar “Parallel Computing“

Summer term 2008

Figure 7: Speedup of XviD encoding in the McRT framework (compared to single core performance)3

The experiments reveal that XviD encoding scales very well on McRT (Figure 7, 1080p and 768p denote different video resolutions; the curve labelled “linear” models the ideal speedup). However, the encoding process was explicitly tweaked for the many-core scenario – under the condition, that only very little fast memory exists per logical core, parallelization wasn’t conducted at frame level – instead single frames were partitioned into parts of frames which were encoded in parallel using OpenMP. The scalability of SVD and SOM is quite similar to that of XviD; more figures and evaluations of different scheduling strategies can be found in [28].

3

Picture is from [28]

17

Seminar “Parallel Computing“

Summer term 2008

4. Conclusion The probability is high, that processor architectures will undergo extensive changes in order to keep up with Moore’s law in the future. AMPs and many-core CPUs are just two proposals for innovative new architectures that may help in prolonging the time horizon within which Moore’s law can stay valid. Operating system schedulers are going to have to adapt to the changing underlying architectures. The scheduler domains of Linux and Solaris add some urgently needed OS-flexibility – because computing architectures that exhibit different behaviours with regard to memory access or single-threaded performance can be quite easily integrated into the load-balancing hierarchy; however, it must be argued that in the future probably more will have to be done in the scheduling algorithms themselves. But scheduler domains at least provide the required flexibility at the level of threads. Sadly, it has to be said, that current Windows schedulers won’t scale with the number of cores or performance asymmetry at any rate. Basically, the Windows scheduler treats multi-core architecture like SMP systems and hence can not give proper care to the peculiarities of such CPUs, like the shared L2 cache (and possibly, the varying or simply bad single-threaded performance that is going to be a characteristic of emerging future architectures). Ty Carlson, director of technical strategy at Microsoft, even mentioned at a panel discussion that current Windows releases (including Vista) were “designed to run on 1,2, maybe 4 processors”4 but wouldn’t scale beyond. He seems to be perfectly right, when he says that future versions of Windows would have to be fundamentally redesigned. Current research shows the road such a redesign could follow. The approach described in section 3.3 seems to perform pretty well for multi-core processors with asymmetric performance. The advantage is that the load balancing algorithm can be (and was) implemented as a modification to current operating system kernels and hence, can be made available quickly, once asymmetric architectures gain widespread adaption. Experimental evaluation of the scheduling algorithm reveals promising results, also with regard to fairness. Section 3.4 presents the way, in which futuristic schedulers on upcoming many-core architectures could operate. The runtime environment McRT makes use of interesting techniques and the authors of the paper manage to intelligibly explain why pre-emptive scheduling is going to be obsolete on many-core architectures. However, their implementation is realized in user-space and burdens the programmer/user with a multitude of configuration options and programming decisions that are required in order for the framework to guarantee optimal performance. [29] introduces an easier-to-use thread scheduling mechanism based on the efforts of McRT, experimental assessments which could testify on its performance, although planned, haven’t been conducted yet. It will be interesting to keep an eye on the further development of scheduling approaches for many-core architectures, since they might gain fundamentally in importance in the future. Achieving fairness and repeatability on today’s available multi-core architectures are the major design goals of the scheduling techniques detailed in sections 3.1 and 3.2. The first approach is justified by a number of experimental results that show that priorities are actually enforced much better than with conventional schedulers; however it remains to be seen, 4

See: http://www.news.com/8301-10784_3-9722524-7.html

18

Seminar “Parallel Computing“

Summer term 2008

whether the complexity of the scheduling approach and the amount of overhead potentially introduced by it, justify that improvement. Maybe it would be advantageous to consider implementing such mechanisms already at the hardware level, if possible. The algorithm mentioned in section 3.2 hasn’t been implemented yet, so it is an open question whether such a rather complicated load-balancing algorithm would be feasible in practice. From the description one can figure, that it takes a lot of computations and thread migrations in order to ensure the load-balance and it would be interesting to see the overhead from the computations and the cache-misses imposed by the mechanism on the system. Without any experimental data, those figures are hard to assess.

19

Seminar “Parallel Computing“

Summer term 2008

5. References [1]

G. E. Moore: „Cramming more components onto integrated circuits“, Electronics, Volume 38, Number 8, 1965.

[2]

„Why Parallel Processing“: http://www.tc.cornell.edu/Services/Education/Topics/Parallel/Concepts/ 2.+Why+Parallel+Processing.htm

[3]

O. Wechsler: „Inside Intel® Core™ Microarchitecture“, Intel Technology Whitepaper, http://download.intel.com/technology/architecture/new_architecture_06.pdf

[4]

„Key Architectural Features AMD Athlon™ X2 Dual-Core Processors“, http://www.amd.com/usen/Processors/ProductInformation/0,,30_118_9485_13041%5E13043,00.html

[5]

Li et al.: „Efficient Operating System Scheduling for Performance-Asymmetric MultiCore Architectures“, In: International conference on high performance computing, networking, storage, and analysis, 2007.

[6]

Balakrishnan et al.: „The Impact of Performance Asymmetry in Emerging Multicore Architectures”, In Proceedings of the 32nd Annual International Symposium on Computer Architecture, pages 506–517, June 2005.

[7]

M. Annavaram, E. Grochowski, and J. Shen: “Mitigating Amdahl’s law through EPI throttling”. In Proceedings of the 32nd Annual International Symposium on Computer Architecture, pages 298–309, June 2005.

[8]

V. Pallipadi, S.B. Siddha: “Processor Power Management features and Process Scheduler: Do we need to tie them together?” In: LinuxConf Europe 2007

[9]

S.B. Siddha: “Multi-core and Linux Kernel”, http://oss.intel.com/pdf/mclinux.pdf

[10]

http://kernelnewbies.org/Linux_2_6_23

[11]

http://lwn.net/Articles/230574/

[12]

J. Andrews: “Linux: The Completely Fair Scheduler“, http://kerneltrap.org/node/8059

[13]

A. Kumar: “Multiprocessing with the Completely Fair Scheduler”, http://www.ibm.com/developerworks/linux/library/l-cfs/index.html

[14]

Kernel 2.6.7 Changelog: http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.7

[15]

Scheduling domains: http://lwn.net/Articles/80911

[16]

Kernel 2.6.17 Changelog: http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.17 20

Seminar “Parallel Computing“

[17]

Summer term 2008

T. Kidd: “C-states, C-states and even more C-states”, http://softwareblogs.intel.com/2008/03/27/update-c-states-c-states-and-even-more-c-states/

[18]

Solaris 10 Process Scheduling: http://www.princeton.edu/~unix/Solaris/troubleshoot/schedule.html

[19]

Solaris manpage FSS(7): http://docs.sun.com/app/docs/doc/816-5177/fss-7?l=de&a=view&q=FSS

[20]

Eric Saxe: “CMT and Solaris Performance”, http://blogs.sun.com/esaxe/entry/cmt_performance_enhancements_in_solaris

[21]

MSDN section on Windows scheduling: http://msdn.microsoft.com/en-us/library/ms685096%28VS.85%29.aspx

[22]

A. Fedorova, M. Seltzer and M. D. Smith: “Cache-Fair Thread Scheduling for Multicore Processors”, Technical Report TR-17-06, Harvard University, Oct. 2006

[23]

S. Kim, D. Chandra and Y. Solihin: “Fair Cache Sharing and Partitioning in a Chip Multiprocessor Architecture”, In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques, 2004

[24]

D. Menasce and V. Almeida: “Cost-Performance Analysis of Heterogeneity in Supercomputer Architectures”, In: Proceedings of the 4th International Conference on Supercomputing, June 1990.

[25]

M. D. Hill and M. R. Marty: “Amdahl’s Law in the Multicore Era”, In: IEEE Computer, 2008

[26]

B. Crepps: “Improving Multi-Core Architecture Power Efficiency through EPI Throttling and Asymmetric Multiprocessing”, Intel Technology Magazine, http://www.intel.com/technology/magazine/research/power-efficiency-0206.htm

[27]

A. Fedorova, D. Vengerov and D. Doucette: “Operating System Scheduling on Heterogeneous Core Systems”, to appear in Proceedings of the First Workshop on Operating System Support for Heterogeneous Multicore Architectures, 2007.

[28]

B. Saha et al.: “Enabling Scalability and Performance in a Large Scale CMP Environment”, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, 2007.

[29]

M. Rajagopalan, B. T. Lewis and T. A. Anderson: “Thread Scheduling for Multi-Core Platforms”, in: Proceedings of the Eleventh Workshop on Hot Topics in Operating Systems, 2007.

21