B

Optimizing Performance

2

2

2

Performance is affected by the shielding characteristics of the system and the distribution of the involved processes among the CPUs. This chapter explains these concepts and how you can optimize the performance of your simulation.

Disabling Nonessential Software

Nonessential server software must be disabled on the real-time host in order to maximize real-time performance and minimize the impact of background applications when running simulation tests.

The X graphical environment seriously degrades real-time performance and should not be run on the system. Edit the /etc/inittab file and replace the default setup level with the following line to start the system at run level 3 (multiuser, no graphics):

id:3:initdefault:

Turn off the following services on the system by entering the following commands as root:

# chkconfig crond off

# chkconfig atd off

# chkconfig smb off

# chkconfig sendmail off

# chkconfig nfs off

# chkconfig vsftpd off

# chkconfig xfs off

# chkconfig vncserver off

# chkconfig rcim_clocksync off

CPU Shielding

Shielding allows a subset of processors to be dedicated to tasks that require the most deterministic performance. The processors should be shielded to prevent the OS from assigning processes to them unless explicitly requested by SIMulation Workbench.

The CPUs to be shielded depend on your system hardware configuration. In general, to achieve optimum performance follow these recommendations:

   In a non-hyperthreaded environment, shield (shield –a) all logical CPUs except CPU 0 (“logical CPUs” are shown in the first column of the cpu(1) command output). See shield(1) for more information. If you have 4 physical CPUs the command would be:

# shield -a1,2,3

      To permanently shield those processrs, add the command to /etc/rc.d/rc.local.

   In a hyperthreaded environment, shield (shield –a) all logical CPUs except CPU 0 and its sibling. Do not bind real-time synchronous tasks to the CPU 0 sibling. Use the cpu command to find which CPU is CPU 0’s sibling (it will have the same physical id).

# cpu

log id

(phys id) state shielding

--------- ----- ---------

0 (0)     up    none

1 (3)     up    none

2 (0)     up    none

3 (3)     up    none

   Bind the RCIM interrupt to the same logical CPU as the one that will execute the scheduler process (usually the first shielded logical CPU, in most cases CPU 1).

For a complete description of shielding, see the shield(1) man page and the RedHawk Linux User’s Guide.

CPU Assignment

Proper binding of SimWB processes to shielded CPUs could be essential in achieving the required performance. The information displayed by the Real-Time Viewer under the Scheduler Info tab should help to analyze the load of the system and adjust CPU assignments.

Be aware that the displayed execution times do not include kernel times; that is, process context switch times and times spent in system calls (scheduler fbswait(), semaphore operations used to step through the simulation cycle, and any system calls that script or user programs may have).

Concurrent’s NightStar RT tools can be used if more in-depth analysis is needed.

To assign CPUs to the tasks that perform I/O, use the I/O Tasks form. See I/O Tasks... for details.

CPUs are assigned to the models used in tests in the Test form. See Model Settings for details.

Scheduling Parameters

In addition to assigning CPUs, the I/O Tasks and Test forms can be used to set the scheduling parameters the O/S uses to schedule processes when more than one process needs to run on a given CPU.

The choices for scheduling policy are:

Other

The default scheduling policy, sometimes referred to as the interactive scheduling policy. It is intended for programs that do not have real-time requirements. There is only a single priority value, zero, for this policy.

The actual scheduling priority is dependent on the “nice” value (see nice(1)and nice(2) man pages) and a value maintained and adjusted by the kernel based on the CPU utilization of the process.

This scheduling policy is not recommended for SimWB tasks.

Round Robin

The round robin scheduling policy. This policy has priorities from 1 to 99, with 99 being the most urgent. All processes running with this scheduling policy are more urgent than any with the Other scheduling policy. The scheduling priority is never adjusted by the kernel.

A process will be preempted by any other process with a more urgent priority that becomes runnable on the same CPU. A CPU-bound process that exceeds its time slot (quantum) can also be preempted by another process with the same policy and priority on the same CPU.

The quantum size can be selected using the “nice” system call or by the “run” command on the real-time host. The available quantum values may be seen by running the “run --quantum=list” command on the real-time host.

Use caution when selecting priorities in the 80-99 range as some system kernel worker threads run in that range. If they are starved of CPU time, the system may become unresponsive.

Use of this scheduling policy requires the CAP_SYS_NICE privilege.

FIFO

The First-In-First-Out scheduling policy. This policy is identical to the Round Robin policy except there are no quantums. A CPU-bound process will not be preempted by a process of the same priority urgency. See “Round Robin” above for more details.

Evaluating SimWB Performance Capability

In the following, we are going to look at the autotrans test in the Real-Time Viewer under a variety of settings made in the I/O Tasks, Tests, and Test Sessions dialogs. See I/O Tasks..., Model Settings, Test Sessions..., and Scheduler Info Tab (Upper Right Pane).

Non-Real-Time Performancert_viewer_sched_other.png

Under these worst case settings, it doesn’t take long (just a few seconds) for nondeterminism to become very visible due to the interactive scheduler giving the SimWB tasks low priority and other tasks running on the same CPU. Make the following settings to run the test under these conditions:

I/O Tasks

logtxout_1

CPU: 1, Sched Policy: Other.

logtxin_1:

CPU: 1, Sched Policy: Other

Tests

autotrans_demo_1

CPU: 1, Sched Policy: Other

Test Sessions

Create a new session and run it.

Scheduling

Scheduler: Soft Timer

Real-Time Schedulerrt_viewer_cpu_1_no_shield.png

We get much better determinism using the default schedular with a FIFO schedular policy. This and the following snapshots are all taken after over an hour of running the test.

I/O Tasks

logtxout_1

CPU: 1, Sched Policy: FIFO.

logtxin_1:

CPU: 1, Sched Policy: FIFO

Tests

autotrans_demo_1

CPU: 1, Sched Policy: FIFO

Test Sessions

Create a new session and run it.

Scheduling

Scheduler: Default. Note this requires an RCIM and FBS.

Shieldingrt_viewer_cpu_1_shield.png

Shielding improves real-time determinism by preventing other tasks from interrupting the processor.

Shielding

Run the following command on the real-time host:

# shield -a1

I/O Tasks

logtxout_1

CPU: 1, Sched Policy: FIFO.

logtxin_1:

CPU: 1, Sched Policy: FIFO

Tests

autotrans_demo_1

CPU: 1, Sched Policy: FIFO

Test Sessions

Run the same session as above.

Multiprocessingrt_viewer_cpu_1-4.png

Spreading the tasks across multiple shielded CPUs improves determinism even further.

Shielding

Run the following command on the real-time host:

# shield -a1,2,3,4

I/O Tasks

logtxout_1

CPU: 2, Sched Policy: FIFO.

logtxin_1:

CPU: 3, Sched Policy: FIFO

Tests

autotrans_demo_1

CPU: 4, Sched Policy: FIFO

Test Sessions

Run the same session as above.

Red Line Schedulerrt_viewer_red_line.png

With tasks assigned to their own shielded CPUs, it is possible to have them do busy waits on a semaphore for dispatching. Without system service calls getting in the way, process dispatch is a fast and deterministic as it can be.

Shielding

Run the following command on the real-time host:

# shield -a1,2,3,4

I/O Tasks

logtxout_1

CPU: 2, Sched Policy: FIFO.

logtxin_1:

CPU: 3, Sched Policy: FIFO

Tests

autotrans_demo_1

CPU: 4, Sched Policy: FIFO

Test Sessions

Create a new session and run it.

Scheduling

Scheduler: Red Line. Note this requires an RCIM and FBS.