Wince Realtime

KURT KENNETT MICROSOFT CORPORATION

Real-Time Overview Basic Terminology Practical Usage Example Windows CE Interrupt Model Threads, Processes, IST, ISR, Priorities

Real-Time Overview Basic Terminology Practical Usage Example Windows CE Interrupt Model Threads, Processes, IST, ISR, Priorities

Real-Time specific features Flexible Thread Quantum Priority Inversion Variable System Tick

Real-Time Overview Basic Terminology Practical Usage Example Windows CE Interrupt Model Threads, Processes, IST, ISR, Priorities

Real-Time specific features Flexible Thread Quantum Priority Inversion Variable System Tick

Measurement Tools

Real-Time Overview Basic Terminology Practical Usage Example Windows CE Interrupt Model Threads, Processes, IST, ISR, Priorities

Real-Time specific features Flexible Thread Quantum Priority Inversion Variable System Tick

Measurement Tools Canned Demo

Interrupt Hardware signal indicating a real-world event has happened The corresponding hardware device needs to be serviced in some way by the computer system

Interrupt Hardware signal indicating a real-world event has happened The corresponding hardware device needs to be serviced in some way by the computer system

Latency The time from when the interrupt occurred to when the hardware begins to be serviced

Interrupt Hardware signal indicating a real-world event has happened The corresponding hardware device needs to be serviced in some way by the computer system

Latency The time from when the interrupt occurred to when the hardware begins to be serviced

Jitter Range of allowable variation in service time Usually defined by the “tolerance” of a mechanical system for the variability in response

Interrupt Hardware signal indicating a real-world event has happened The corresponding hardware device needs to be serviced in some way by the computer system

Latency The time from when the interrupt occurred to when the hardware begins to be serviced

Jitter Range of allowable variation in service time Usually defined by the “tolerance” of a mechanical system for the variability in response

Bounded Extreme limits are known precisely

Interrupt Hardware signal indicating a real-world event has happened The corresponding hardware device needs to be serviced in some way by the computer system

Latency The time from when the interrupt occurred to when the hardware begins to be serviced

Jitter Range of allowable variation in service time Usually defined by the “tolerance” of a mechanical system for the variability in response

Bounded Extreme limits are known precisely

Bounded low latency and jitter = “hard” real time

Consumers want to know if CE is hard real-time Want to know if CE is capable of simultaneously running radio and UI Some system developers were concerned that CE was not hard real-time enough to meet the requirements

Consumers want to know if CE is hard real-time Want to know if CE is capable of simultaneously running radio and UI Some system developers were concerned that CE was not hard real-time enough to meet the requirements

Requirements Run cellular radio DSP Meet “tight” timing requirements ARM <= 250Mhz CPU Full performance Windows CE UI And play simultaneous video and audio

OMAC represents Industrial Automation Community 100 ms

Soft Real-Time

Cycle Time

20 ms

10 ms

Hard Real Time

5 ms

1 ms 500 us 0

100 µs

1,000 µs

5,000 µs

Cycle Variation or Jitter (µs)

10,000 µs

OMAC represents Industrial Automation Community 100 ms

Soft Real-Time

Cycle Time

20 ms

Windows NT

10 ms

Hard Real Time

5 ms

1 ms 500 us 0

100 µs

1,000 µs

5,000 µs

Cycle Variation or Jitter (µs)

10,000 µs

OMAC represents Industrial Automation Community 100 ms

Soft Real-Time

Cycle Time

20 ms

Windows CE 2.X

10 ms

Hard Real Time

5 ms

Windows NT

1 ms 500 us 0

100 µs

1,000 µs

5,000 µs

Cycle Variation or Jitter (µs)

10,000 µs

OMAC represents Industrial Automation Community 100 ms

Soft Real-Time

Cycle Time

20 ms

Windows CE 2.X

10 ms

Hard Real Time

5 ms

1 ms

Windows NT

Windows CE .NET

500 us 0

100 µs

1,000 µs

5,000 µs

Cycle Variation or Jitter (µs)

10,000 µs

OMAC represents Industrial Automation Community 100 ms

Soft Real-Time

Cycle Time

20 ms

Windows CE 2.X

10 ms

Hard Real Time

5 ms

1 ms

Windows NT

90% of Apps

Windows CE .NET

500 us 0

100 µs

1,000 µs

5,000 µs

Cycle Variation or Jitter (µs)

10,000 µs

So what were some actual requirements for a real project? 200Mhz ARM Windows CE with Full UI Running WMV Video playback Interrupt as frequent as every 4.6 ms Allowable jitter < 0.5ms Actual Application Requirements

Interrupt every 4.6 ms

0.5 ms Jitter

Windows CE Real-Time Test Results Minimum Average Maximum

ISR starts

IST starts

1.2 µs 3.3 µs 13.3 µs

31.7 µs 67.2 µs 103.0 µs

Time in microseconds (µs)

In terms of the 0.5 ms jitter alone CE’s longest ISR response time was 13.3 µs (2.6% of max allowed) CE’s longest IST response time was 103 µs (20.6% of max allowed)

Windows CE Real-Time Test Results Minimum Average Maximum

ISR starts

IST starts

1.2 µs 3.3 µs 13.3 µs

31.7 µs 67.2 µs 103.0 µs

Time in microseconds (µs)

In terms of the 0.5 ms jitter alone CE’s longest ISR response time was 13.3 µs (2.6% of max allowed) CE’s longest IST response time was 103 µs (20.6% of max allowed)

Conclusion CE’s response time was well within the requirements Project in case study went ahead and did well

Thread A piece of code that can be scheduled to run by the kernel A “unit” of execution May be launched by a process or a driver

Thread A piece of code that can be scheduled to run by the kernel A “unit” of execution May be launched by a process or a driver

Process Launched from an executable file A collection of threads (at least one) with a common execution environment Can create threads to handle interrupts

Thread A piece of code that can be scheduled to run by the kernel A “unit” of execution May be launched by a process or a driver

Process Launched from an executable file A collection of threads (at least one) with a common execution environment Can create threads to handle interrupts

Driver A DLL, (dynamically loaded library) loaded into the kernel or into a user-mode driver host process Can create threads to handle interrupts

Interrupt Service Routine (ISR) A piece of code built into or loaded into the kernel Logically assigned to a particular hardware interrupt Called within O(10) CPU instructions to service an interrupt Should be written to run quickly with few or no outside dependencies and no synchronization with or unbounded calls to other code Can be chained together if multiple devices might use the same hardware interrupt line (IRQ) Logically notifies the kernel which IST should run

Interrupt Service Routine (ISR) A piece of code built into or loaded into the kernel Logically assigned to a particular hardware interrupt Called within O(10) CPU instructions to service an interrupt Should be written to run quickly with few or no outside dependencies and no synchronization with or unbounded calls to other code Can be chained together if multiple devices might use the same hardware interrupt line (IRQ) Logically notifies the kernel which IST should run

Interrupt Service Thread (IST) A thread registered specifically to handle an interrupt Can be created by either a process or a driver Scheduled like any other thread on the system Should be written to do the bulk of the interrupt handling work

ISRs and ISTs usually work as pairs ISR handles the critical time-sensitive work IST typically handles the rest

ISRs and ISTs usually work as pairs ISR handles the critical time-sensitive work IST typically handles the rest

They synchronize by using an OS EVENT Object The IST creates an EVENT Object The IST associates the EVENT with a logical interrupt It uses the WaitForSingleObject() to wait for the EVENT object to be signaled The ISR tells the kernel which EVENT object to signal When the EVENT is signaled, it unblocks the IST and makes it able to be scheduled by the kernel to run

ISRs and ISTs usually work as pairs ISR handles the critical time-sensitive work IST typically handles the rest

They synchronize by using an OS EVENT Object The IST creates an EVENT Object The IST associates the EVENT with a logical interrupt It uses the WaitForSingleObject() to wait for the EVENT object to be signaled The ISR tells the kernel which EVENT object to signal When the EVENT is signaled, it unblocks the IST and makes it able to be scheduled by the kernel to run

If the IST is the highest priority run-able thread, it will get scheduled to run immediately

Windows CE 6.0 has 256 levels of priority

Levels

Description

0 through 96

Real-time above drivers

97 through 152

Default used by CE device drivers

153 through 247

Real-time below drivers

248 through 255

Non-real-time priorities

Windows CE 6.0 has 256 levels of priority Level 0 is the highest and 255 is the lowest

Levels

Description

0 through 96

Real-time above drivers

97 through 152

Default used by CE device drivers

153 through 247

Real-time below drivers

248 through 255

Non-real-time priorities

Windows CE 6.0 has 256 levels of priority Level 0 is the highest and 255 is the lowest The default level for a thread is 252

Levels

Description

0 through 96

Real-time above drivers

97 through 152

Default used by CE device drivers

153 through 247

Real-time below drivers

248 through 255

Non-real-time priorities

Windows CE 6.0 has 256 levels of priority Level 0 is the highest and 255 is the lowest The default level for a thread is 252 Levels 0 through 248 can be reserved by OEM Levels

Description

0 through 96

Real-time above drivers

97 through 152

Default used by CE device drivers

153 through 247

Real-time below drivers

248 through 255

Non-real-time priorities

Is responsible for determining which thread will run Has a queue for threads for each priority level Always schedules first thread at the highest priority level

Is responsible for determining which thread will run Has a queue for threads for each priority level Always schedules first thread at the highest priority level

A thread gets to run for set length of time, called a “quantum” Typically 100 milliseconds A quantum of 0 means the quantum never runs out The thread can run until blocked or interrupted

Is responsible for determining which thread will run Has a queue for threads for each priority level Always schedules first thread at the highest priority level

A thread gets to run for set length of time, called a “quantum” Typically 100 milliseconds A quantum of 0 means the quantum never runs out The thread can run until blocked or interrupted

A thread runs until— Its quantum runs out It is interrupted by a higher priority thread It is blocked by a resource contention such as access to a critical section or a mutex

Interrupt Occurs

Interrupt Handler calls registered ISR

ISR runs, tells kernel which event to signal

IST runs and resets the interrupt

Scheduler sets up the IST to run

Kernel signals event, IST becomes runable

The maximum amount of time between a hardware interrupt being signaled and when the code in an IST begins to run is:

The maximum amount of time between a hardware interrupt being signaled and when the code in an IST begins to run is: Maximum time before ISR can run

The maximum amount of time between a hardware interrupt being signaled and when the code in an IST begins to run is: Maximum time before ISR can run +

Maximum time taken by ISR

The maximum amount of time between a hardware interrupt being signaled and when the code in an IST begins to run is: Maximum time before ISR can run +

Maximum time taken by ISR +

Maximum time taken by kernel to be able to get to a state where it can schedule another thread

The maximum amount of time between a hardware interrupt being signaled and when the code in an IST begins to run is: Maximum time before ISR can run +

Maximum time taken by ISR +

Maximum time taken by kernel to be able to get to a state where it can schedule another thread +

Maximum time taken by kernel to switch from whatever thread was running to the IST thread

MAXIMUM ISR LATENCY

INTERRUPT!

Normal Thread

Int Off

Normal Thread

IST ISR

OAL

Scheduler ISH

ISR Latency

ISH

Scheduler

KERNEL

IST Latency

Interrupts Disabled Preemption Disabled

MAXIMUM ISR LATENCY

INTERRUPT!

Normal Thread

Int Off

Normal Thread

IST ISR

OAL

Scheduler ISH

ISR Latency

ISH

Scheduler

KERNEL

IST Latency

Interrupts Disabled Preemption Disabled

MAXIMUM IST LATENCY

INTERRUPT!

Normal Thread

Normal Thread

IST ISR

OAL

KCall

KCall ISH

ISR Latency

Scheduler

ISH

Scheduler

KERNEL

IST Latency

Interrupts Disabled Preemption Disabled

MAXIMUM IST LATENCY

INTERRUPT!

Normal Thread

Normal Thread

IST ISR

OAL

KCall

KCall ISH

ISR Latency

Scheduler

ISH

Scheduler

KERNEL

IST Latency

Interrupts Disabled Preemption Disabled

Higher priority ISRs can preempt lower ISRs

Higher priority ISRs can preempt lower ISRs Based on support by the CPU, additional hardware, and/or OEM code

Higher priority ISRs can preempt lower ISRs Based on support by the CPU, additional hardware, and/or OEM code ARM Uses a vectored interrupt table Only two CPU interrupt levels, each with its own hardware pin IRQ – Normally only one used. FIQ – Supports ISR-only scenarios for critical time-sensitive tasks

Interrupts are not turned on before entering an ISR OEM can re-enable CPU interrupt within ISR when/if they want

OEMs can prioritize the interrupts with an external Interrupt Controller and bit masks to turn on and off the different interrupts

Hardware designers often attach several devices to the same interrupt line (IRQ)

Hardware designers often attach several devices to the same interrupt line (IRQ) Multiple ISRs can be chained together to handle shared interrupts

Hardware designers often attach several devices to the same interrupt line (IRQ) Multiple ISRs can be chained together to handle shared interrupts Each ISR in turn determines if it can handle the interrupt If it can, it does its work and either completes the interrupt or returns the logical SYSINTR value representing which IST is to run If not, it returns SYSINTR_CHAIN indicating the kernel should try the next ISR in the chain. The default OEMInterruptHandler on a system typically will disable an interrupt that occurs that does not have a relative IST assigned to it (it is not claimed by an ISR)

A high priority thread can get stuck waiting for a lower priority thread to release a resource Such as a critical section, semaphore, or mutex This causes priority inversion

A high priority thread can get stuck waiting for a lower priority thread to release a resource Such as a critical section, semaphore, or mutex This causes priority inversion

The Kernel detects priority inversion and handles it with priority inheritance, or “boosting” The lower priority thread temporarily inherits the higher priority thread’s priority The lower priority thread’s quantum is set to 0, which lets it run until it releases the resource

A high priority thread can get stuck waiting for a lower priority thread to release a resource Such as a critical section, semaphore, or mutex This causes priority inversion

The Kernel detects priority inversion and handles it with priority inheritance, or “boosting” The lower priority thread temporarily inherits the higher priority thread’s priority The lower priority thread’s quantum is set to 0, which lets it run until it releases the resource

Supports only one level of inheritance Kernel will only boost one thread If the boosted thread is also in turn blocked by a third thread, the third thread is not boosted Care must be taken when acquiring locks on shared resources

There is a per-thread quantum

There is a per-thread quantum The default is set by the OEM in the OAL dwDefaultThreadQuantum

There is a per-thread quantum The default is set by the OEM in the OAL dwDefaultThreadQuantum

APIs to set Quantum Ce(Set/Get)ThreadQuantum

There is a per-thread quantum The default is set by the OEM in the OAL dwDefaultThreadQuantum

APIs to set Quantum Ce(Set/Get)ThreadQuantum

Quantum of 0 sets a thread to “run-to-completion” This can occur at any priority Preempted only by higher priority threads

Normally there is a 1 ms timer tick

Normally there is a 1 ms timer tick A tick interrupt can cause a reschedule When a system tick occurs the kernel checks for a reschedule or No-Op The kernel will make sure one of the highest priority run-able threads is executing If there is no higher priority thread and the current thread has not exhausted its quantum, the current thread is allowed to continue to run

Normally there is a 1 ms timer tick A tick interrupt can cause a reschedule When a system tick occurs the kernel checks for a reschedule or No-Op The kernel will make sure one of the highest priority run-able threads is executing If there is no higher priority thread and the current thread has not exhausted its quantum, the current thread is allowed to continue to run

Sleep(N) will generally wake up in N to N + 1 ms

Normally there is a 1 ms timer tick A tick interrupt can cause a reschedule When a system tick occurs the kernel checks for a reschedule or No-Op The kernel will make sure one of the highest priority run-able threads is executing If there is no higher priority thread and the current thread has not exhausted its quantum, the current thread is allowed to continue to run

Sleep(N) will generally wake up in N to N + 1 ms When the CPU has nothing to do (no threads can run) the system tick is reset to when the next scheduled event is to occur An interrupt can re-set the system tick back to 1 ms

Windows CE 6.0 memory model was re-architected GWES, Networking, File System, and Driver Manager were all moved into the kernel’s address space All threads from a kernel mode driver run in the kernel process context They have full access to the kernel space Security checks are not required to run or call APIs

Windows CE 6.0 memory model was re-architected GWES, Networking, File System, and Driver Manager were all moved into the kernel’s address space All threads from a kernel mode driver run in the kernel process context They have full access to the kernel space Security checks are not required to run or call APIs

Kernel mode APIs The exact same API interface is used, but without the overhead of a system privilege level change or argument marshalling These resolve to quick calls through a vector table

Windows CE 6.0 memory model was re-architected GWES, Networking, File System, and Driver Manager were all moved into the kernel’s address space All threads from a kernel mode driver run in the kernel process context They have full access to the kernel space Security checks are not required to run or call APIs

Kernel mode APIs The exact same API interface is used, but without the overhead of a system privilege level change or argument marshalling These resolve to quick calls through a vector table

Kernel mode drivers are ideal for real-time work

Software-based real-time measurement tool Code in kernel is minimally instrumented to provide the data

Software-based real-time measurement tool Code in kernel is minimally instrumented to provide the data

Measures both ISR and IST latencies ISR latency From IRQ to ISR

IST latency From the end of the ISR to the start of the IST

Software-based real-time measurement tool Code in kernel is minimally instrumented to provide the data

Measures both ISR and IST latencies ISR latency From IRQ to ISR

IST latency From the end of the ISR to the start of the IST

Enabled for all sample platforms

Software-based real-time measurement tool Code in kernel is minimally instrumented to provide the data

Measures both ISR and IST latencies ISR latency From IRQ to ISR

IST latency From the end of the ISR to the start of the IST

Enabled for all sample platforms Can be used to test response under varying system load

This does scheduler performance timing tests

This does scheduler performance timing tests Tool enables you to determine how long it takes to perform basic kernel tasks such as— Acquire or release a critical section Wait or signal an event Create a semaphore or mutex Yield a thread Call system APIs

This does scheduler performance timing tests Tool enables you to determine how long it takes to perform basic kernel tasks such as— Acquire or release a critical section Wait or signal an event Create a semaphore or mutex Yield a thread Call system APIs

These things can be good to know in scenarios when more than one time-critical situation can be occurring

Shows interaction between processes, threads, and interrupts

Shows interaction between processes, threads, and interrupts Shows events that affect scheduling and system performance

Shows interaction between processes, threads, and interrupts Shows events that affect scheduling and system performance Shows thread state and migration

We want to: Take an existing BSP Track the occurrence of an external event via a hardware interrupt. Show when CE initially responds to the external event through an ISR Show when CE responds to that even in an IST.

We want to: Take an existing BSP Track the occurrence of an external event via a hardware interrupt. Show when CE initially responds to the external event through an ISR Show when CE responds to that even in an IST.

To do this, we will: Watch the MMC card-detect signal. Have an interrupt triggered on card insertion. Turn on an LED when the ISR starts Turn off the same LED when the IST starts. Watch the LED signal relative to the MMC card-detect signal to see how long it takes for CE to respond.

We want to: Take an existing BSP Track the occurrence of an external event via a hardware interrupt. Show when CE initially responds to the external event through an ISR Show when CE responds to that even in an IST.

To do this, we will: Watch the MMC card-detect signal. Have an interrupt triggered on card insertion. Turn on an LED when the ISR starts Turn off the same LED when the IST starts. Watch the LED signal relative to the MMC card-detect signal to see how long it takes for CE to respond.

To watch the signals we’ll use an oscilliscope.

IRQ

CPU

SOFTWARE

IRQ

CPU

IST

SOFTWARE

IRQ

CPU

IST

SOFTWARE

IRQ

CPU

IST

SOFTWARE

IRQ CPU

IST

SOFTWARE

IRQ CPU

ISR

IST

SOFTWARE

IRQ IRQ CPU

ISR

IST

SOFTWARE

IRQ CPU

ISR

IST

SOFTWARE

IRQ CPU

ISR

IST

SOFTWARE

CPU

IST

SOFTWARE

Scheduler CPU

IST

SOFTWARE

Scheduler CPU

IST

SOFTWARE

CPU

IST

SOFTWARE

CPU

IST

SOFTWARE

CPU

IST

SOFTWARE

Events we will see on the scope: Card Insertion ISR Response (turn on LED) IST Response (turn off LED)

Events we will see on the scope: Card Insertion ISR Response (turn on LED) IST Response (turn off LED)

By using a timing scale we can see how responsive our system is.

Events we will see on the scope: Card Insertion ISR Response (turn on LED) IST Response (turn off LED)

By using a timing scale we can see how responsive our system is. System used for test: PXA27X (Marvell/Intel) on PhyCore270 Board 100Mhz MMC CD and LED via GPIO

Start with the desired deadline and acceptable jitter

Start with the desired deadline and acceptable jitter Understand the hardware What is slow? What can be blocked by external events?

Start with the desired deadline and acceptable jitter Understand the hardware What is slow? What can be blocked by external events?

Understand the OS Know how critical sections and other synchronization methods work Know what system calls can be blocked on shared resources

Start with the desired deadline and acceptable jitter Understand the hardware What is slow? What can be blocked by external events?

Understand the OS Know how critical sections and other synchronization methods work Know what system calls can be blocked on shared resources

Separate Real-Time threads from UI threads

Start with the desired deadline and acceptable jitter Understand the hardware What is slow? What can be blocked by external events?

Understand the OS Know how critical sections and other synchronization methods work Know what system calls can be blocked on shared resources

Separate Real-Time threads from UI threads Pre-allocate critical resources

Start with the desired deadline and acceptable jitter Understand the hardware What is slow? What can be blocked by external events?

Understand the OS Know how critical sections and other synchronization methods work Know what system calls can be blocked on shared resources

Separate Real-Time threads from UI threads Pre-allocate critical resources Use buffers to communicate between time-critical system components and applications/UI

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.