Sun Studio C/C++: PFO - single feedbin for all executables

(Note: the words "feedbin" and "feedback" are used interchangeably throughout this blog post)

It is a known fact that by default compiler arranges to create one profile feedback file for each profiled executable. A profiled binary contains compiler inserted instrumentation code, to collect run-time data (like block execution counts) from it. Compiler inserts instrumentation code, when the original code is compiled with -xprofile=collect compiler option.

The default behavior is good enough for small programs or applications with very few executables. However for very large real world applications with tens of executables, having too many feedback files poses slight inconvenience in use phase, where these feedback files are specified on compile line with -xprofile=use:<path_to_feedback_file> option, to generate highly optimized binaries. For example, if the application consists of twenty executables, we need to have twenty -xprofile=use options on the compile line, like:

% cc -xO4 -xprofile=use:feedback1 -xprofile=use:feedback2 ... -xprofile=use:feedback19 -xprofile=use:feedback20 -o optimizedbin application.c

There are two major inconveniences with this approach:

1. If make file grabs all compiler flags from environment variables like CFLAGS, it may not be possible to specify all -xprofile=use options in a single CFLAGS, due to the underlying shell restrictions on the number of characters per variable.
   
   
2. The compile line may become too long, and look ugly with too many -xprofile=use flags.

To get around these inconveniences, it is suggested to use compiler supported environment variables SUN_PROFDATA_DIR and SUN_PROFDATA in profile data collection phase, to request the profiler to write all the profile feedback data from different executables in to a single feedbin file, instead of creating one for each executable. If these environment variables are set, the profiler writes the feedback data in to the file pointed by SUN_PROFDATA, under directory SUN_PROFDATA_DIR ie., the feedback data from all executables will be written in to SUN_PROFDATA_DIR/SUN_PROFDATA.

The following example illustrates the default behavior, as well as the behavior with SUN_PROFDATA and SUN_PROFDATA_DIR environment variables.

eg.,

%cat a.c
#include <stdio.h>

int main()
{
    printf("In a.c\n");
    return (0);
}

%cat b.c
#include <stdio.h>

int main()
{
    printf("In b.c\n");
    return 0;
}

% cc -xO2 -xprofile=collect -o a a.c
% cc -xO2 -xprofile=collect -o b b.c

Default behavior

% setenv SUN_PROFDATA_DIR /tmp/default

% ./a
In a.c

% ./b
In b.c

% ls /tmp/default
a.profile/  b.profile/

Observe that there are two profiles, one for each executable. a.profile holds the feedback data for the executable "a"; and b.profile holds the feedback data for the executable "b".

To use the feedback data, the programs have to be compiled as follows:

% cc -xO2 -xprofile=use:/tmp/default/a -o a a.c
% cc -xO2 -xprofile=use:/tmp/default/b -o b b.c

Requesting for a single feedback file to hold all profile data

% setenv SUN_PROFDATA_DIR /tmp/consolidate
% setenv SUN_PROFDATA singlefeedbin.profile

These variables inform the profiler to write all feedback data from different profiled binaries into a single feedbin file: /tmp/consolidate/singlefeedbin.profile.

%mkdir /tmp/consolidate

%./a
In a.c

%./b
In b.c

% ls /tmp/consolidate
singlefeedbin.profile/

Observe that the singlefeedbin.profile's feedbin holds the feedback data for both executables "a" and "b". If there are more profiled executables, shared libraries, all the feedback data will be appended to this feedbin file.

To use this profile, simply:

% cc -xO2 -xprofile=use:/tmp/consolidate/singlefeedbin -o a a.c
% cc -xO2 -xprofile=use:/tmp/consolidate/singlefeedbin -o b b.c

Related blog posts:

1. Sun Studio C/C++: Profile Feedback Optimization (PFO): part 1 | part 2
   
2. Sun Studio 11: Asynchronous Profile Feedback Data Collection

Acknowledgements:
Chris Aoki, Sun Microsystems
___________________
Technorati tags: Sun Studio