Sun Studio 11: Asynchronous Profile Feedback Data Collection

Sun released Studio 11 compiler collection, couple of weeks back, and giving it away for free, for everyone (Studio 10 is also freely downloadable from Sun downloads web site; but the only requirement is that the user must register at OpenSolaris web site)

About asynchronous profile collection

Asynchronous profile feedback data collection is one of the new features in this release. However the data collection part is totally transparent to the end user, and hence this feature was neither documented nor highlighted anywhere. In simple words, this new feature doesn't require any changes to the way the feedback data was collected; but increases the probability of getting a good profile from multi-threaded applications.

Prior to this release, the profiler thread has to wait until the shared library finalization and for the process to call exit(), before writing all the feedback data to feedbin file. In a way it mandates the process to be exited, to get the feedback data. Also there is no guarantee that all the applications (esp. multi-threaded apps) are/will be designed to terminate gracefully. So, some processes may not call exit() at all. In those cases, getting usable feedback data is very unlikely.

To alleviate the problems described above, we need some mechanism to collect the feedback data from a running process without requiring it terminate gracefully. Fortunately Studio 11 has some desirable enhancements; and due to these, the chances of getting a good profile from many single/multi-threaded applications is high, irrespective of how they exit.

Undocumented environment variables

When the collect binaries are built with Studio 11, the profiler thread occasionally (I don't have the exact time in sec) writes the feedback data to feedbin file {on disk}. The time interval between periodic profile snapshots can be controlled by the undocumented env variable SUN_PROFDATA_ASYNC_INTERVAL, whose value is interpreted as the duration, in seconds. If SUN_PROFDATA_ASYNC_INTERVAL has been set to a positive integer value n at startup of an application, the profiler thread collects periodic profile data every n seconds, and subsequently updates the corresponding feedbin.

When data for a snapshot is collected, the profiler updates a single profile directory whose name is of the form:
<procname>.<hostname>.<pid>[.profile]

where:
<procname> is the name of the process being profiled
<hostname> is the host name of the machine executing the profiled process
<pid> is the process id of the profiled process

.profile is appended to the name of the profile directory unless is specified using the value of the {documented} environment variable SUN_PROFDATA.

The collected profile data can be used in use phase of PFO, by specifying the compiler option:
-xprofile=use:<procname>.<hostname>.<pid>

Note:

1. The directory name can be renamed at your will, before specifying it with -xprofile=use option
   
   
2. The profiler thread collects profile snapshots only for the process in which it was initiated. So, forked processes will not inherit the profiler thread (or simply profiler)
   
   
3. When the application is built with -xprofile=collect, the object prof_lib.o is linked into profiled shared libraries, and prof_tsd.o is linked into profiled executables. These objects provide the run-time support for profile feedback data collection. The linking of these object files is transparent to us. To check this, specify -# flag of C compiler, or -v option of C++ compiler, on compile line.

Multiple profile snapshots per proces

Studio 11 also enables the collection of profile data more than once per process. If the env. variable SUN_PROFDATA_ASYNC_SEQUENCE is defined and set to an integer value, num_snapshots >= 1, the profiler generates a sequence of distinct profile snapshots whose names are of the form:
<procname>.<hostname>.<pid>.<n>[.profile]

where:
<n> is a positive integer in the range [1..num_snapshots].

Subsequent profile snapshots are applied to update the <procname>.<hostname>.<pid>[.profile] directory for the remaining life time of the process.

The time sequence of profile snapshots generated by setting SUN_PROFDATA_ASYNC_SEQUENCE may be used to determine how long profile data should be collected from a given application in order to obtain good performance with -xprofile=use.

Example

Let's assume that the program mymtserver is compiled with -xprofile=collect. The profile data collection can be done as follows:

% %uname -n
Govinda
% setenv SUN_PROFDATA_ASYNC_INTERVAL 30 
% setenv SUN_PROFDATA_ASYNC_SEQUENCE 3 
% setenv SUN_PROFDATA_VERBOSE
% setenv SUN_PROFDATA_DIR /tmp/mymtserver
% ./mymtserver & 
[1] 1234

This will collect a snapshot of profile data from process 1234 every 30 seconds for as long as it continues executing. The first 3 snapshots will be saved in their own feedback directories:
/tmp/mymtserver/mymtserver.Govinda.1234.1.profile, /tmp/mymtserver/mymtserver.Govinda.1234.2.profile and /tmp/mymtserver/mymtserver.Govinda.1234.3.profile.

Then the subsequent snapshots will update the feedback directory: /tmp/mymtserver/mymtserver.Govinda.1234.profile.

Note:
To get any warning messages during profile data collection, set the env. variable, SUN_PROFDATA_VERBOSE

Often the default values are good enough to get the feedback data; and we may not need any of these env. variables mentioned here (in the example). Perhaps that's the main reason for these to remain undocumented. Nevertheless they provide more control over the data collection, where we need.

Async. profile collection with Studio 9 & 10

Even though this feature was first integrated into Studio 11, it was backported to Studio 9 & 10, and released as common C/C++ patches. So, to have this feature Studio 9 & 10 must have the following patches installed:
Studio  9: 115983-06 or later
Studio 10: 117832-06 (most likely -- not released yet) or later

Related posts:

1. Sun Studio C/C++: Profile Feedback Optimization
   
2. Sun Studio C/C++: Profile Feedback Optimization II

Credit/Acknowledgements:
Chris Aoki, Sun Microsystems
___________________
Technorati tags: Sun Studio