I'm very glad to be one of the major contributors of this benchmarking effort. As Sun hasn't published any competitive benchmarking numbers running Siebel on Sun platform for long, we started off with a single goal "must win" and put Sun on the top of UNIX vendors list with very well balanced price/performance ratio. P/P is what really the customer cares about i.e., customer needs superior performance with low cost.
We spent nearly two and half months (project duration: 3 months) on fixing various issues ranging from simple configuration issues to memory leaks in Siebel components & core dumps to scalability issues. Finally on 6th of september (labor day) the big day has arrived; able to simulate 10000 concurrent virtual users hitting the same oracle database (running on a "dual core" UltraSPARC IV machine, E2900) at the same time with an average think time of 30 sec and with an average response time of about 0.5 sec. UltraSPARC IV rocked by handling 662 concurrent users at app tier & 4902 users at database tier. Hewlett-Packard's (holding the top spot by then) numbers are 385 & 1786 at app & db tiers respectively. HPs numbers are not even comparable to Sun's numbers and lagging by 72% & 174% at both app & db tiers. Needless to say Sun Microsystems
moved comfortably to the top spot by winning this benchmark. 'Way To Go' Sun!!
Obviously my manager 'George Drapeau' was very excited with the success of the project and discussed the effort in his blog at: http://blogs.sun.com/roller/page/drapeau/20041003#sun_and_siebel_kick_some
Sun Microsystems released a press note with the result of the benchmark on 10th October:http://www.sun.com/smi/Press/sunflash/2004-10/sunflash.20041005.1.html
Siebel systems published the certified results in their web site:http://www.siebel.com/crm/performance-benchmark.shtm
Topology with all tiers:http://blogs.sun.com/roller/resources/drapeau/pspptopology.jpg
Screenshots of load generator & the resource monitoring: