Applications that use the HP-proprietary thread-independent services (or tis) interface should include the tis.h header file and link against the shared C run-time library ( libc.so ).

A.3 Two-Level Scheduling on Tru64 UNIX Systems

Under Tru64 UNIX Version 4.0 and later, the Threads Library implements a two-level scheduling model. The thread library schedules "user threads" onto kernel execution contexts (often known as "kernel threads" or "virtual processors"), just as Tru64 UNIX schedules processes onto the processors of a multiprocessing machine.

A user thread is executed on a kernel thread until it blocks or exhausts its timeslice quantum. Then, the Threads Library schedules a new user thread to run. While the Threads Library is scheduling user threads onto kernel threads, the Tru64 UNIX kernel is independently scheduling those kernel threads to run on physical processors. The term "two-level scheduling" refers to this relationship.

This division allows most thread scheduling to take place completely in user mode, without the intervention of the kernel. Since a thread context switch does not involve any privileged information, it can be done much more efficiently in user mode.

The key to making the two-level scheduling model work is efficient two-way communication between the Threads Library and the Tru64 UNIX kernel. When a thread blocks in the kernel, the Threads Library scheduler is notified so that it can schedule another thread to take advantage of the idle kernel thread. This mechanism, sometimes referred to as an upcall, is inspired by original research on scheduler activations at the University of Washington. (See Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism by Anderson, Bershad, Lazowska, and Levy; ACM Operating Systems Review Volume 25, Number 5, Proceedings of the Thirteenth ACM Symposium on Operating Systems Principles, October 13-16, 1991).

A.3.1 Use of Kernel Threads

Tru64 UNIX kernel threads are created as they are needed by the application. The number of kernel threads that the Threads Library creates is limited by normal Tru64 UNIX configuration limits regarding user and system thread creation. Normally, however, the Threads Library creates one kernel thread for each actual processor on the system, plus a "manager thread" for bookkeeping operations.

The Threads Library does not delete these kernel threads or let them terminate. Kernel threads not currently needed are retained in an idle state until they are needed again. (These idled kernel threads are deleted by the kernel if they remain idle for a long time.) When the process terminates, all kernel threads in the process are reclaimed by the kernel.

The Threads Library scheduler can schedule any user thread onto any kernel thread. Therefore, a user thread can run on different kernel threads at different times. Normally, this should pose no problem. However, for example, the kernel thread ID as reported by the dbx or Ladebug debuggers (in "native" $threadlevel) can change at any time.

A.3.2 Support for Realtime Scheduling

The Threads Library supports Tru64 UNIX realtime scheduling. This allows you to set the scheduling policy and priority of threads. By default, threads are created using process contention scope. This means that the full range of POSIX.1 scheduling policy and priority is available. However, threads running in process contention scope do not preempt lower-priority threads in another process. For example, a thread in process contention scope with SCHED_FIFO policy and maximum priority 63 will not preempt a thread in another process running with SCHED_FIFO and lower priority.

In contrast, system contention scope means that each thread created by the program has a direct and unique binding to one kernel execution context. A system contention scope thread competes against all threads in the system and will preempt any thread with lower priority. For this reason, the priority range of threads in system contention scope is restricted unless running with root privilege.

Specifically, a thread with SCHED_FIFO policy cannot run at a priority higher than 18 without privilege, since doing so could lock out all other users on the system until the thread blocked. Threads at any other scheduling policy (including SCHED_RR ) can run at priority 19 because they are subject to periodic timeslicing by the system. For more information, see the Tru64 UNIX Realtime Programming Guide.

If your program lacks necessary privileges, attempting to call the following routines for a thread in system contention scope returns the error value [EPERM]:

pthread_attr_setschedpolicy()	( Error returned by pthread_create() at thread creation )
pthread_attr_setschedparam()	( Error returned by pthread_create() at thread creation )
pthread_setschedparam()

Prior to Tru64 UNIX Version 4.0, all threads used only system contention scope. In Tru64 UNIX Version 4.0, all threads created using the pthread interface, by default, have process contention scope.

A.4 Thread Cancelability of System Services

Tru64 UNIX supports the required system cancelation points specified by the POSIX.1 standard and by the Single UNIX Specification, Version 2 (UNIX98).

For legacy multithreaded applications, note that threads created using the cma or d4 interfaces will not be cancelable at any system call. (Here "system call" means any function without the pthread_ prefix.) If system call cancelation is required, you must write code using the pthread interface.

For more information, see Section 2.3.7.

A.4.1 Cancelation Points

accept()

aio_suspend()

close()

connect()

creat()

fcntl() (for cmd F_SETLKW)

fsync()

getmsg()

getpmsg()

lockf()

mq_receive()

mq_send()

msgrcv()

msgsnd()

msync()

nanosleep()

open()

pause()

poll()

pread()

pthread_cond_timedwait()

pthread_cond_wait()

pthread_delay_np()

pthread_join()

pthread_testcancel()

putmsg()

putpmsg()

pwrite()

read()

readv()

recv()

recvfrom()

recvmsg()

select()

sem_wait()

send()

sendmsg()

sendto()

shutdown()

sigpause()

sigsuspend()

sigtimedwait()

sigwait()

sigwaitinfo()

sleep()

system()

tcdrain()

t_close()

t_connect()

t_listen()

t_rcv()

t_rcvconnect()

t_rcvrel()

t_rcvreldata()

t_rcvudata()

t_rcvv()

t_rcvvudata()

t_snd()

t_sndrel()

t_sndreldata()

t_sndudata()

t_sndv

t_sndvudata()

usleep()

wait()

wait3()

waitid()

waitpid()

write()

writev()

A.4.2 Conditional or Future Cancelation Points

closedir()

closelog()

ctermid()

dbm_close()

dbm_delete()

dbm_fetch()

dbm_nextkey()

dbm_open()

dbm_store()

dlclose()

dlopen()

endgrent()

endhostent()

endnetent()

endprotoent()

endpwent()

endservent()

endutxent()

fclose()

fcntl() (for any cmd)

fflush()

fgetc()

fgetpos()

fgets()

fgetwc()

fgetws()

fopen()

fprintf()

fputc()

fputs()

fputwc()

fputws()

fread()

freopen()

fscanf()

fseek()

fseeko()

fsetpos()

ftell()

ftello()

ftw()

fwprintf()

fwrite()

fwscanf()

getc()

getc_unlocked()

getchar()

getchar_unlocked()

getcwd()

getdate()

getgrent()

getgrgid()

getgrgid_r()

getgrnam()

getgrnam_r()

gethostbyaddr()

gethostbyname()

gethostent()

gethostname()

getlogin()

getlogin_r()

getnetbyaddr()

getnetbyname()

getnetent()

getprotobynumber()

getprotobyname()

getpwent()

getpwnam()

getpwnam_r()

getpwuid()

getpwuid_r()

gets()

getservbyname()

getservbyport()

getservent()

getutxent()

getutxid()

getutxline()

getw()

getwc()

getwchar()

getwd()

glob()

iconv_close()

iconv_open()

ioctl()

lseek()

mkstemp()

nftw()

opendir()

openlog()

pclose()

perror()

popen()

printf()

putc()

putc_unlocked()

putchar()

putchar_unlocked()

puts()

pututxline()

putw()

putwc()

putwchar()

readdir()

readdir_r()

remove()

rename()

rewind()

rewinddir()

scanf()

seekdir()

semop()

setgrent()

sethostent()

setnetent()

setprotoent()

setpwent()

setservent()

setutxent()

strerror()

syslog()

tmpfile()

tmpname()

ttyname()

ttyname_r()

ungetc()

ungetwc()

unlink()

vfprintf()

vfwprintf()

vprintf()

vwprintf()

wprintf()

wscanf()

Note that appropriate non-standard functions that do not appear in the preceding list might become cancelation points in the future. Tru64 UNIX will also implement new cancelation points, as specified by future revisions of the relevant formal or consortium standard bodies.

A.5 Using Signals

This section discusses signal handling based on the POSIX.1 standard.

Tru64 UNIX Version 4.0 introduced the full POSIX.1 signal model. In previous versions, "synchronous" signals (those resulting from execution errors, such as SIGSEGV and SIGILL) could have different signal actions for each thread. Prior to Tru64 UNIX Version 3.2, all threads shared a common, processwide signal mask, which meant one thread could not receive a signal while another had the signal blocked.

Under Tru64 UNIX Version 4.0 and later, all signal actions are processwide. That is, when any thread uses sigaction or equivalent to either set a signal handler, or to modify the signal action (for example, to ignore a signal), that action will affect all threads. Each thread has a private signal mask so that it can block signals without affecting the behavior of other threads.

Prior to Tru64 UNIX Version 4.0, asynchronous signals were processed only in the main thread. In Tru64 UNIX Version 4.0, any thread that does not have the signal masked can process the signal.

The POSIX 1003.1 sigwait() service allows any thread to block until one of a specified set of signals is delivered. A thread can wait for any of the asynchronous signals except for SIGKILL and SIGSTOP.

For example, you can create a thread that blocks on a sigwait() routine for SIGINT, rather than handling a Ctrl/C in the normal way. This thread could then cancel other threads to cause the program to shut down the current activities.

Following are two reasons for avoiding signals:

• Signals cannot be used in a modular way in a multithreaded program.
• Signals, used as an asynchronous programming technique, are unnecessary in a multithreaded program.

In a multithreaded program, signal handlers cannot be used in a modular way because there is only one signal handler routine for all of the threads in an application. If two threads install different signal handlers for the signal, all threads will dispatch to the last handler when they receive the signal.

Most applications should avoid using asynchronous programming techniques with threads. For example, techniques that rely on timer and I/O signals are usually more complicated and errorprone than techniques that rely on simply waiting synchronously within a thread. Furthermore, most of the thread services are not supported for use in signal handlers, and most run-time library functions cannot be used reliably inside a signal handler.

Some I/O intensive code may benefit from asynchronous I/O, but these programs will generally be more difficult to write and maintain than "pure" threaded code.

A thread should not wait for a synchronous signal. This is because synchronous signals are the result of an error during the execution of a thread, and if the thread is waiting for a signal, then it is not executing. Therefore, a synchronous signal cannot occur for a particular thread while it is waiting, and the thread will wait forever.

The POSIX.1 standard requires that the thread block the signals for which it will wait before calling sigwait() . For reliable operation, the signals should be blocked in all threads. Otherwise, the signal might be delivered to another thread before the sigwait thread calls sigwait() , or after it has returned with another signal.

A.5.2 Handling Synchronous Signals as Exceptions

For the signals traditionally representing synchronous errors in the program, the Threads Library catches the signal and converts it into an equivalent exception. This exception is then propagated up the call stack in the current thread and can be caught and handled using the normal exception catching mechanisms.

Table A-3 lists Tru64 UNIX signals that are reported as exceptions by default. If any thread declares an action for one of these signals (using sigaction(2) or equivalent), no thread in the process can receive the exception.

Table A-3 Signals Reported as Exceptions

Signal	Exception
SIGILL	pthread_exc_illinstr_e
SIGIOT	pthread_exc_SIGIOT_e
SIGEMT	pthread_exc_SIGEMT_e
SIGFPE	pthread_exc_aritherr_e
SIGBUS	pthread_exc_illaddr_e
SIGSEGV	pthread_exc_illaddr_e
SIGSYS	pthread_exc_SIGSYS_e
SIGPIPE	pthread_exc_SIGPIPE_e

A.6 Thread Stack Guard Areas

When creating a thread based on a thread attributes object, the Threads Library potentially rounds up the value specified in the object's guardsize attribute. The Threads Library does so based on the value of the configurable system variable PAGESIZE (see <sys/mman.h> . The default value of the guardsize attribute in a thread attributes object is the number of bytes equal to the setting of PAGESIZE .

A.7 Thread Stack and Backing Store Allocation

Starting in Version 5.0, for threads that accept the default stack address attribute, the Threads Library allocates a thread's writable stack area from uncommitted virtual memory, then commits predefined increments of the writable stack area to the thread only as it is needed. The stack's corresponding backing store is also reserved incrementally as the stack is committed. In this way, no more backing store is reserved than the stack actually requires.

Because Tru64 UNIX 5.0 does not commit backing store (or physical pages) for stacks until the pages are used by the program, the default stack size has been increased. The previous default of about 24Kb (3 pages) has been increased to 5Mb.

A.8 Dynamic Activation

You cannot use dlopen() to dynamically add libpthread.so to a process when the main program is not "initially threaded". (By "initially threaded" we mean that the libpthread.so library is initialized at program activation time; either because the main program links against libpthread.so directly, or because it links against some other shared library that links against libpthread.so .)

You can determine whether a binary is "initially threaded". Many components of Tru64 UNIX, including libc , decide whether to be "thread safe" at program initialization time. These components cannot adapt if the environment changes later at runtime. That is, these libraries work properly with threads only when the process is initially threaded.

You can safely use dlopen() to load shared libraries, whether or not they depend on libpthread.so , as long as the main program was initially threaded.

This restriction is unlikely to be removed, as the "standards community" generally agrees that dynamically changing the process environment from "nonthreaded" to "threaded" is dangerous (and rarely if ever useful). Few systems implementing POSIX threads even allow the possibility---for example, libc.so on many systems contains "stub" versions of the POSIX functions that are preempted by linking with libpthread.so but will not be preempted by later loading libpthread.so with dlopen() . In other words, calls to pthread_create() might continue to fail with ENOSYS , and pthread_mutex_lock() might continue to succeed without any memory references to the lock.

A.8.1 RTLD_LAZY Symbol Resolution

Because the dynamic loader is not fully thread-safe, the RTLD_LAZY symbol resolution mode is disabled in a threaded process. For most code, the consequences of this are not important. (The dl_open() will take slightly longer to resolve and load all the symbols, and more virtual memory will be used if you don't end up resolving all the symbols anyway.)

However, if a shared library has unresolved external symbol references, it may still be successfully loaded using dlopen() with RTLD_LAZY , as long as the routines that reference the unresolved symbols are not used. If the same dlopen() call uses RTLD_NOW , the dlopen() call will return a failure because of the unresolved references. All calls to dlopen() in a threaded process quietly ignore a value of RTLD_LAZY and behave as if RTLD_NOW were specified. This can sometimes cause confusion because a shared library that works normally in a nonthreaded environment may fail when loaded into a shared process. You can test for these problems by loading your library with lazy binding explicitly disabled, by defining the environment variable LD_BIND_NOW to a non-null value.

A.8.2 Alternate Thread-safe Libraries

When you use cc -pthread or cc -threads (or the same switches for the cxx compiler), the compiler driver will automatically search for "thread-safe" alternate library versions, and replace the specified -l dependencies for you. For example, if /usr/shlib includes both libmylib.so and libmylib_r.so , and you link with cc -pthread -lmylib , the cc driver will link with libmylib_r.so instead of libmylib.so .

However, when you call dlopen() to dynamically load libmylib.so , this replacement does not occur. In a threaded process, you must be careful to request libmylib_r.so explicitly if it exists.