work with solaris contract subsystem
A process contract is the formal definition of the relationship thatexists between a Process A and its monitoring rocess. In case process Aterminates abnormally, the monitoring process will be able to restartit.This is a small exercise that is using the contract subsystem and its associated commmands.
0. Get some information
# man contract
1. SMF and contracts
Check the state of the “syslog” service :
# svcs system-log
STATE STIME FMRI
online Dec_10 svc:/system/system-log:default
#
# pgrep -fl syslog
658 /usr/sbin/syslogd
# ps -o pid,comm,ctid | grep syslog
658 /usr/sbin/syslogd 35
# svcs -pv system-log
STATE NSTATE STIME CTID FMRI
online - Dec_10 35 svc:/system/system-log:default
Dec_10 658 syslogd
Allthese commands show that the service is enabled ( syslogd is startedwith pid 658 ) and is monitored by the contract subsystem usingContractID 35.
Another command shows more information about the contract :
#ctstat -i 35
CTID ZONEIDTYPE STATE HOLDEREVENTSQTIME NTIME
35 0 process owned 7 0 - -
Or in verbose mode :
#ctstat -vi 35
CTID ZONEIDTYPE STATE HOLDEREVENTSQTIME NTIME
35 0 process owned 7 0 - -
cookie: 0×20
informative event set: none
critical event set: core signal hwerr empty
fatal event set: none
parameter set: inherit regent
member processes: 658
inherited contracts: none
The HOLDER field indicates that the process holding the contract is having pid 7 :
# ps -fp 7
UID PIDPPID C STIME TTY TIME CMD
root 7 1 0 Dec 10 ? 0:41 /lib/svc/bin/svc.startd
which is the main SMF daemon.Let’s find out more about the contract held by this process. The /proc file system
can help there :
# cd /proc/7/contracts/
# ls
171819222730323435363738394042434650 83
where we find out why ’svc.startd’, also knows as svc://system/svc/restarter:default is called the master
restarter in the SMF framework : it is holding contracts for many many services.
Comingback to the verbose output of the ctstat command, we see that someevents received by the holder of contract 35 are considered critical.Among them, “signal” is an event that indicates the reception of afatal signal from another process. The restarter has the job ofrestarting syslogd if one ‘critical’ event is received.
Let’s check that…
2. The master restarter
Before helping syslogd to die, let’s open another terminal and type
#ctwatch -rv 35
CTID EVID CRIT ACK CTTYPE SUMMARY
which can be used to see all the events related to contract 35.
We can now use svcadm :
#svcadm refresh system-log
Nothing visible, a quick look at the Pid of syslogd tells us that it only reread its config file. We did the
famous “pkill -HUP syslogd” in the SMF way. Let’s try again :
#svcadm restart system-log
#svcs -pv system-log
STATE NSTATE STIME CTID FMRI
online - 10:24:37 89 svc:/system/system-log:default
10:24:37 2173 syslogd
while the other “ctwatch” terminal shows :
#ctwatch -rv 35
CTID EVID CRIT ACK CTTYPE SUMMARY
35 33 crit noprocesscontract empty
Whathappend ? The syslogd process was terminated by svcadm. The contractbeing linked to the process is then also terminated and is ‘empty’.While starting another syslogd, another contract, number 89, wascreated.
Now in the “ctwatch” terminal, let’s type
#ctwatch -rv 89
CTID EVID CRIT ACK CTTYPE SUMMARY
while in the other window, we kill the syslog daemon (only in the global zone) :
#pkill -9 -z 0 syslogd
We observe in the “ctwatch terminal” :
#ctwatch -rv 89
CTID EVID CRIT ACK CTTYPE SUMMARY
89 34 crit noprocessprocess 2173 received a fatal signal
signal: 9 (SIGKILL)
sender pid: 2187
sender ctid: 86
89 35 crit noprocesscontract empty
Whichshows that the contract subsystem was notified that the syslogd processreceived a signal. We even may know who sent it. In our case, it is the“kill” command, which terminated already. The result of the signal isthat
contract is now empty, ended.
But :
# svcs -pv system-log
STATE NSTATE STIME CTID FMRI
online - 10:27:04 90 svc:/system/system-log:default
10:27:04 2193 syslogd
shows that the master restarter has done its job. The system-log service is still online because “svc.startd” has
instructed “init” to fork and exec a new version of syslogd to keep the service running.
3. The contract file system
Everyoneknows the /proc filesystem, used to provide information to systemadministrators about the running processes in a nice well-knownfile-based manner. Commands like “ps”, “pfile”, “pgrep”,… get theirinformation by opening and reading files in /proc which are actually aninterface to the process structures maintained by the kernel.
Thesame is true for ctfs, the contract file system. All the contractcommands get their input from the kernel through another pseudofilesystem mounted on /system/contract.
Example :
# truss -t open ctstat -i 35
open(”/var/ld/ld.config”, O_RDONLY) Err#2 ENOENT
open(”/lib/libcontract.so.1″, O_RDONLY) = 3
open(”/lib/libuutil.so.1″, O_RDONLY) = 3
open(”/lib/libc.so.1″, O_RDONLY) = 3
open(”/lib/libnvpair.so.1″, O_RDONLY) = 3
open(”/lib/libnsl.so.1″, O_RDONLY) = 3
open(”/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1″, O_RDONLY) = 3
open(”/usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3″, O_RDONLY) = 3
CTID ZONEIDTYPE STATE HOLDEREVENTSQTIME NTIME
open64(”/system/contract/all/35/status”, O_RDONLY) = 3
35 0 process owned 7 0 - -
4. Monitor and restart ANY application
Thecommand ‘ctrun’ can be used to create a contract for any application.It will then monitor the application process for all the events thatyou specify and restart the application if some fatal event occurs.Check it out!
#ctrun -r 0 -o noorphan -f signal /usr/openwin/bin/xclock &
Myxclock will be restarted any number of times ( -r 0 ), ctrun will makesure that all processes get killed before restarting ( -o noorphan )and we monitor the ’signal’ type of fatal event.
You get a contractid and so on.
Then just kill the clock and see it nicely reappear…
1What is the contract file system?
From contract(4): " The /system/contract file system acts as the primaryinterfacetothe contract subsystem."
From learning solaris: "A process contract is the formal definition of the relationship that exists between a Process A and its monitoring process."
In my own words : "The contract file system sets up a framework in the kernel that enables real time monitoring of processes and enables us to act on changes in process states immediately."
2How to use the contract file system>
The contract file system is primarily used by the SMF(5) framework when services are started. The service restarter uses contracts to keep track of the processes in a contract.
It is also possible for applications and users to create contracts on the fly. The following commands are used by users to work with and manage contracts:
[*]ctstat
[*]ctrun
[*]ctwatch
[*]/system/contract
3Examine your system
The contract framework is implemented as a pseudo file system.At system boot the contract file system is mounted at/system/boot.
www2.petervg.nl # df -n /system/contract/system/contract : ctfs www2.petervg.nl # grep ctfs /etc/vfstabctfs - /system/contract ctfs - no -www2.petervg.nl # ls /system/contract//all22260223626063 6064 6065 6066 6073 6078 6079 6083 6090 6091 6093 6094 6095 6096 6097 6098
4Create our own contract
With the ctrun command we can run a command and place it in a new contract.
4.1New process without a new contract
www2.petervg.nl # ptree -c $$14536 zsched22362] 3168/usr/lib/ssh/sshd 3171/usr/lib/ssh/sshd 3173-bash 3265sleep 555 3279ptree -c 3173www2.petervg.nl # sleep 555 & 3280www2.petervg.nl # ptree -c $$14536 zsched22362] 3168/usr/lib/ssh/sshd 3171/usr/lib/ssh/sshd 3173-bash 3280sleep 555 3281ptree -c 3173
As we can see, the sleep 555 commandis running in the same contract (22362) as the parent bash shell.
4.2New command with a new contract
www2.petervg.nl # ptree -c $$14536 zschedcontract 22362] 3168/usr/lib/ssh/sshd 3171/usr/lib/ssh/sshd 3173-bash 3300ptree -c 3173www2.petervg.nl # ctrun sleep 666 & 3301www2.petervg.nl # ptree -c $$14536 zsched22362] 3168/usr/lib/ssh/sshd 3171/usr/lib/ssh/sshd 3173-bash 3301ctrun sleep 666 22366] 3302sleep 666 3303ptree -c 3173
And in this example we see that the sleep 666command is actually run under a new contract (22366).
But does this contract automagically restart the sleep process whenit is killed? The answer is no. By a contract will not restart theprocesses in a contract.
5Checking the status of a contract
When a contract is started we can use the ctstat command to get informationabout the contract. The ctwatch command can then be usedto monitor the use of the contract over time.
www2.petervg.nl # ps -o ctid,pid,ppid,args | grep sleep2236235773173 ctrun sleep 5552238535783577 sleep 5552236235993591 grep sleep
With the ps -o ctid command it ispossible to identify the contract a particular command is runningin.
www2.petervg.nl # ptree -c $$14536 zsched 3168/usr/lib/ssh/sshd 3171/usr/lib/ssh/sshd 3173-bash 3591ksh 4323ctrun sleep 666 4324sleep 666 4336ptree -c 3591www2.petervg.nl # ctstat -i 22449CTID ZONEIDTYPE STATE HOLDEREVENTSQTIME NTIME 22449 12 process owned 4323 0 - -
In the previous output we see that the contract under investigationis contract 22449 which is running in zone 12. The current contractis a process contract which is held by process 4323. This meansthat process 4323 created the contract and started the process inthis contract.
With the -v option we can obtain more verbose information abouta specific contract.
www2.petervg.nl # ctstat -v -i 22452CTID ZONEIDTYPE STATE HOLDEREVENTSQTIME NTIME 22452 12 process owned 4408 0 - - cookie: 0 informative event set: core critical event set: hwerr empty fatal event set: hwerr parameter set: none member processes: 4409 inherited contracts: none
页:
[1]