Mandalika's scratchpad [ Work blog @Oracle | Stock Market Notes | My Music Compositions ]

Old Posts: 09.04  10.04  11.04  12.04  01.05  02.05  03.05  04.05  05.05  06.05  07.05  08.05  09.05  10.05  11.05  12.05  01.06  02.06  03.06  04.06  05.06  06.06  07.06  08.06  09.06  10.06  11.06  12.06  01.07  02.07  03.07  04.07  05.07  06.07  08.07  09.07  10.07  11.07  12.07  01.08  02.08  03.08  04.08  05.08  06.08  07.08  08.08  09.08  10.08  11.08  12.08  01.09  02.09  03.09  04.09  05.09  06.09  07.09  08.09  09.09  10.09  11.09  12.09  01.10  02.10  03.10  04.10  05.10  06.10  07.10  08.10  09.10  10.10  11.10  12.10  01.11  02.11  03.11  04.11  05.11  07.11  08.11  09.11  10.11  11.11  12.11  01.12  02.12  03.12  04.12  05.12  06.12  07.12  08.12  09.12  10.12  11.12  12.12  01.13  02.13  03.13  04.13  05.13  06.13  07.13  08.13  09.13  10.13  11.13  12.13  01.14  02.14  03.14 

Thursday, March 27, 2008
Accessing the Command Line Args of a Shell Script from an Embedded AWK script

Let's start with the example of the skeleton of a shell script running on Sun Solaris.

if [ $# -lt 2 ]; then
echo "Usage: "
pmap -x $PIDS | grep total | awk 'BEGIN { FS = " " } {print $1,$2,$3,$4,$5} {rss+=$4}
{private+=$5} END {print "Total Private mem: "private/1024" M Total RSS mem: "rss/1024" M Total
Shared mem: " (rss-private)/1024 "M **** For 10000 user load: per user memory footprint is:
"((private/1024)+(((rss-private)/1024)/NR))/10000" MB/user"}'

Notice the highlighted hard coded number for the number of users. The goal is to read the number of users value from the command line rather than editing the script every time there is a change in the number of users.

Assuming there is a shell script with an embedded AWK script like the one shown in the above example, how the awk script can access the command line arguments supplied to the top level shell script?

It is not possible to use $1, $2 etc., because $1 and $2 will hold the string tokens returned by the awk output. For example, if awk returned "total Kb 472648 242936 57768", then $1 will hold the string "total" and $2 will have "Kb".

The solution/trick is to pass the required command line arguments from the main shell script to the awk script with the -v option of awk; and then to access the arguments supplied for the awk script either by directly referencing the argument name or by using ARGC & the ARGV array notation (just like C/C++).

/usr/bin/awk on Sun Solaris doesn't accept the -v option. Use /usr/bin/nawk or /usr/xpg4/bin/awk on Solaris.

Now with all the above information, the last line of code can be modified as shown below to get rid of the hard coded numbers:
pmap -x $PIDS | grep total | nawk -v Arg1=$2 'BEGIN { FS = " " } {print $1,$2,$3,$4,$5} {rss+=$4} {private+=$5} 
END {print "Total Private mem: "private/1024" M Total RSS mem: "rss/1024" M Total Shared mem: "
(rss-private)/1024 "M **** For ", Arg1, " user load: per user memory footprint is:
"((private/1024)+(((rss-private)/1024)/NR))/Arg1" MB/user"}'

Technorati Tags:
| | | |

Saturday, March 22, 2008
Oracle 10g: Upgrading to a Higher Patchset (eg., to

This blog post outlines the steps involved in patching an existing Oracle 10g RDBMS environment to a higher patch set with an example showing the steps for upgrading from to For detailed instructions, check the README page that is associated with the patch{set}. Use these instructions at your own risk.

| |

Sunday, March 16, 2008
Is Oracle's PeopleSoft Really a Multi-Threaded Application?

Perhaps the answer to this question is irrelevant to many of the PeopleSoft end users - but it is really important for the administrators to know what kind of application they are dealing with.

Here is a snapshot of the process statistics (prstat output) for the PeopleSoft application server processes running on a Solaris 10 system:

10864 psft 353M 235M sleep 60 10 0:13:42 1.5% PSAPPSRV/11
10855 psft 353M 235M sleep 2 10 0:13:55 1.5% PSAPPSRV/11
10846 psft 353M 235M sleep 3 10 0:14:04 1.5% PSAPPSRV/11
10870 psft 353M 235M sleep 0 10 0:13:50 1.5% PSAPPSRV/11
10873 psft 353M 235M sleep 1 10 0:13:57 1.4% PSAPPSRV/11
10852 psft 353M 235M sleep 0 10 0:13:57 1.4% PSAPPSRV/11
10858 psft 353M 235M sleep 60 10 0:13:47 1.3% PSAPPSRV/11
10849 psft 349M 231M cpu0 20 10 0:13:55 1.3% PSAPPSRV/11
10867 psft 353M 235M sleep 60 10 0:13:53 1.3% PSAPPSRV/11
10861 psft 349M 231M sleep 60 10 0:13:56 1.2% PSAPPSRV/11

Notice the number of LWPs (represented by NLWP in the snapshot) that are associated with each of those PSAPPSRV processes. Just by looking at the above snapshot, one may under the impression that the PSAPPSRV (PeopleSoft Application Server process) is a multi-threaded process because it appears there are 11 worker threads actively processing the user requests.

To dig a little deeper, Solaris provides the ability to check the statistics for each of the light-weight processes (LWPs) that are associated with a process (here I'm assuming that the intended audience can differentiate a process from a light-weight process). With the help of -L option of the prstat, Solaris reports the statistics for each LWP in a given process. Let's have a close look at those stats.

10864 psft 353M 235M cpu32 0 10 0:01:37 1.5% PSAPPSRV/1
10864 psft 353M 235M sleep 59 0 0:00:00 0.0% PSAPPSRV/11
10864 psft 353M 235M sleep 29 10 0:00:00 0.0% PSAPPSRV/10
10864 psft 353M 235M sleep 28 10 0:00:00 0.0% PSAPPSRV/9
10864 psft 353M 235M sleep 28 10 0:00:00 0.0% PSAPPSRV/8
10864 psft 353M 235M sleep 59 0 0:00:00 0.0% PSAPPSRV/7
10864 psft 353M 235M sleep 59 0 0:00:00 0.0% PSAPPSRV/6
10864 psft 353M 235M sleep 51 2 0:00:00 0.0% PSAPPSRV/5
10864 psft 353M 235M sleep 59 0 0:00:00 0.0% PSAPPSRV/4
10864 psft 353M 235M sleep 29 10 0:00:00 0.0% PSAPPSRV/3
10864 psft 353M 235M sleep 59 0 0:00:00 0.0% PSAPPSRV/2

Notice the PID in the first column. It confirms that the above snapshot is the process stats breakdown by LWPs for a given process. Now check the output under the TIME column. That column represents the cumulative execution time for the process -- LWP, in this case. Except for the LWP #1, the exec time for rest of the LWPs is zero i.e., even though the PSAPPSRV process spawned 10 more LWPs, in reality they are not doing any work at all. When I tried to find the reason {from my counterpart at Oracle Corporation} for the creation of multiple LWPs, I was told that the multiple LWPs are a side effect of loading JRE(s) into the process address space during the run-time. Also it appears the PeopleSoft application server processes (PSAPPSRV) can process only one user request (transaction) at a time. It is the limitation of the PeopleSoft Enterprise by design.

So the bottomline is: PeopleSoft application server processes (PSAPPSRV) are not multi-threaded even though they appear to be multi-threaded from the operating system perspective.

Before we conclude, make sure you understand that the discussion in this blog post applies only to the PeopleSoft application server processes, PSAPPSRV. You cannot generalize it to the whole PeopleSoft Enterprise. For example, application engine processes (PSAESRV) that run under the control of the Process Scheduler are actually multi-threaded processes. However expanding the discussion around Process Scheduler/Application Engine is beyond the scope of this blog post.

Sanjay Goyal
Technorati Tags:
| |

Thursday, March 06, 2008
Windows Vista/HP Pavilion dv9000: Fixing Y! Messenger's Webcam Not Connected Error

If the built-in webcam of HP Pavilion dv9000 is not functioning properly on Windows Vista with Yahoo! Messenger, one of the following steps may help fix the issue (Courtesy: HP Customer Support).

Step #1:

If the issue persists even after these steps or if there is no entry found in "Device Manager", then download and install the updated webcam driver from the following link:

Step #2 (Recommended):

Technorati Tags:
| | | | | | | |

Monday, March 03, 2008
PeopleSoft: Fixing "msgget: No space left on device" Error on Solaris 10

When high number of application server processes are configured in a single PeopleSoft domain or in multiple domains cumulative, it is very likely that the PeopleSoft server domain boot process may fail with errors like:
Booting server processes ...
exec PSSAMSRV -A -- -C psappsrv.cfg -D CS90SPV -S PSSAMSRV :
113954.ben15!PSSAMSRV.29746.1.0: LIBTUX_CAT:681: ERROR: Failure to create message queue
113954.ben15!PSSAMSRV.29746.1.0: LIBTUX_CAT:248: ERROR: System init function failed, Uunixerr = :
msgget: No space left on device

113954.ben15!tmboot.29708.1.-2: CMDTUX_CAT:825: ERROR: Process PSSAMSRV at ben15 failed with /T
tperrno (TPEOS - operating system error)

In this particular example, the PeopleSoft Enterprise is running on a Solaris 10 system. Fortunately the error message is very clear in this case; and the failure is related to the message queues. During the domain boot up process, there is a call to msgget() to create a message queue. If the call to msgget() succeeds, it returns a non-negative integer that serves as the identifier for the newly created message queue. However in case of a failure, it returns -1 and sets the error number to EACCES, EEXIST, ENOENT or ENOSPC depending on the underlying reason.

From the above error messages it clear that the msgget() failed with the errno set to ENOSPC (No space left on device). Man page of msgget(2) has the following explanation for ENOSPC error code on Solaris:
The msgget() function will fail if:
ENOSPC A message queue identifier is to be created but
the system-imposed limit on the maximum number of
allowed message queue identifiers system wide
would be exceeded. See NOTES.


The system-imposed limit on the number of message queue
identifiers is maintained on a per-project basis using the
project.max-msg-ids resource control.

Good stuff. It has enough clues to suspect the configured number for the message queue identifiers.

Prior to the release of Solaris 10, the /etc/system System V IPC tunable, msgsys:msginfo_msgmni, could be configured to control the maximum number of message queues that can be created. The default value on pre-Solaris 10 systems is 50.

With the release of Solaris 10, majority of the System V IPC tunables were obsoleted and equivalent resource controls were created for the remaining tunables, to reduce the administrative overhead. On Solaris 10 and later versions, System V IPC can be tuned on a per project basis using the newly introduced resource controls.

On Solaris 10, The resource control, project.max-msg-ids, replaced the old /etc/system tunable, msginfo_msgmni. And the default value has been raised to 128.

Now back to the failure in PeopleSoft environment. Let's first check the current value configured for project.max-msg-ids.

Alternatively run the command ipcs -q to check the number of active message queues. Note that the project with id '3' is configured to create a maximum of 128 (default) message queues. In any case, the number of active message queues from the ipcs -q output may almost match with the configured value for the project.max-msg-ids.

Since it appears the configured PeopleSoft domain(s) needs more than 128 message queues in order to bring up all the application server processes that constitute the PeopleSoft Enterprise, the solution is to increase the value for the resource control, project.max-msg-ids, to any value beyond 128. For the sake of simplicity, let's increase it to 256 (2 * default value, that is). Again prctl utility can be used to set the new value for the resource control.

That's all there is. With the above change, the PeopleSoft Enterprise should boot up at least with no Failure to create message queue .. msgget: No space left on device errors.

Before we conclude, note that the above mentioned solution is not persistent across multiple operating system reboots. To make it persistent, create a new project with projadd command. The man page for projadd(1M) has an example showing the creation of a project.
Technorati Tags:
| | | |


This page is powered by Blogger. Isn't yours?