Mandalika's scratchpad [ Work blog @Oracle | My Music Compositions ]

Old Posts: 09.04  10.04  11.04  12.04  01.05  02.05  03.05  04.05  05.05  06.05  07.05  08.05  09.05  10.05  11.05  12.05  01.06  02.06  03.06  04.06  05.06  06.06  07.06  08.06  09.06  10.06  11.06  12.06  01.07  02.07  03.07  04.07  05.07  06.07  08.07  09.07  10.07  11.07  12.07  01.08  02.08  03.08  04.08  05.08  06.08  07.08  08.08  09.08  10.08  11.08  12.08  01.09  02.09  03.09  04.09  05.09  06.09  07.09  08.09  09.09  10.09  11.09  12.09  01.10  02.10  03.10  04.10  05.10  06.10  07.10  08.10  09.10  10.10  11.10  12.10  01.11  02.11  03.11  04.11  05.11  07.11  08.11  09.11  10.11  11.11  12.11  01.12  02.12  03.12  04.12  05.12  06.12  07.12  08.12  09.12  10.12  11.12  12.12  01.13  02.13  03.13  04.13  05.13  06.13  07.13  08.13  09.13  10.13  11.13  12.13  01.14  02.14  03.14  04.14  05.14  06.14  07.14  09.14  10.14  11.14  12.14  01.15  02.15  03.15  04.15  06.15  09.15  12.15  01.16  03.16  04.16  05.16  06.16  07.16  08.16  09.16  12.16  01.17  02.17  03.17  04.17  06.17  07.17  08.17  09.17  10.17  12.17  01.18  02.18  03.18  04.18  05.18  06.18  07.18  08.18  09.18  11.18  12.18  01.19  02.19  05.19  06.19  08.19  10.19  11.19  05.20  10.20  11.20  12.20  09.21  11.21  12.22 


Tuesday, September 28, 2004
 
Solaris: Virtual Memory System

VM system manages the mapping of physical to users processes & kernel i.e., VM system manages system's memory on behalf of kernel and processes

Responsibilities of VM system:

(1) manages virtual-to-physical mapping of memory

(2) presents a simple memory programming model to applications so that application developers need not know how the underlying memory hardware is arranged <- memory is abstracted; user applications deal with virtual addresses and virtual address spaces

(3) allows processes to see linear ranges of bytes in their address space regardless of the physical layout or fragmentation of real memory

(4) efficient allocation of physical memory to processes & kernel subsystems

VM system uses slower storage medium (disk) to store data that doesn't fit within the physical memory (RAM) of the system, thus accomodating programs larger than the size of physical memory

(5) keeps most frequently used portions of memory in the RAM, to make the application run faster; manages swapping of memory between primary and secondary storage to optimize performance

(6) handles requirements of shared images between multiple users & processes

Quick Notes:

Address space:
linear range of memory

Every process will have an address space; each address space isbroken into several segments that represent mapping of the executable, heap, shared libraries & a program stack
Each segment is divided into equal sized pieces of VM known as pages. Hardware MMU (Memory Management Unit) does the mapping of VM pages to physical memory

Demand Paging:
VM system implements demand paging; pages of memory are allocated on demand

Page Fault:
MMU raises an event to tell the kernel that an access has occured to an area of memory that doesn't have the physical memory mapped to it. Heap of a process is also allocated in a similar way.

Initially only VM space (address space) is allocated to the process. When the memory is first referenced, a page fault occurs and the memory is allocated one page at a time

Most of the kernel's memory is not pageable; i.e., it is allocated from physical memory which cannot be stolen by page scanner

Each page of physical memory is associated with a file and an offset; the file and offset identify the backing store for the page

Anonymous memory:
pages used for regular process heap and stack; swapfs takes care of that. Anon memory doesn't have a vnode attached to it

Dirty page:
A page that has had its contents modified

Heap:
scratch memory aka temporary processing space for a process

Hardware Memory Management Unit (MMU) maps pages into physical memory by using a platform specific set of translation tables called Translation Lookaside Buffer (TLB) & Translation Software Buffer (TSB)


Sunday, September 26, 2004
 
Linux: Installing Source RPM (SRPM) package

RPM stands for RedHat Package Manager. RPM is a system for installing and managing software & most common software package manager used for Linux distributions. Because it allows you to distribute software already compiled, a user can install the software with a single command.

There are two flavors of RPMs: binary & source code. Source code are designated by ending with -src.rpm rather than just .rpm

Installing source rpms (SRPM):
Unlike ordinary packages, they can't be uninstalled using RPM. SRPM packages are not under the control of the RPM database. All RPM does is copy the files contained in the package to the hard disk. SRPMs are not listed in the RPM database and not marked as installed in YaST's package selection forms.

A source RPM (SRPM) package typically contains a gzipped tar archive with the source files, and an RPM spec file

Govinda:/users/techno/downloads # ls -l xmms*src*
-rw------- 1 techno users 4032244 2004-09-28 12:43 xmms-1.2.10-1.src.rpm

Listing the contents of a source rpm:
Govinda:/users/techno/downloads # rpm -qpl xmms-1.2.10-1.src.rpm
xmms-1.2.10.tar.gz
xmms.spec

Source rpm can be installed just like any other rpm with "-ivvh" flags
Govinda:/users/techno/downloads # rpm -ivvh xmms-1.2.10-1.src.rpm
D: counting packages to install
D: found 1 packages
D: looking for packages to download
D: retrieved 0 packages
D: New Header signature
D: Signature size: 180
D: Signature pad : 4
D: sigsize : 184
D: Header + Archive: 4031964
D: expected size : 4031964
D: found 1 source and 0 binary packages
D: New Header signature
D: Signature size: 180
D: Signature pad : 4
D: sigsize : 184
D: Header + Archive: 4031964
D: expected size : 4031964
D: installing a source package
D: sources in: /usr/src/packages/SOURCES
D: spec file in: /usr/src/packages/SPECS
D: file: xmms-1.2.10.tar.gz action: unknown
D: file: xmms.spec action: unknown
xmms ##################################################
GZDIO: 494 reads, 4041876 total bytes in 0.020 secs


Verbose output shows that the sources are under: /usr/src/packages/SOURCES & the specification (SPEC) file is under: /usr/src/packages/SPECS
Govinda:/users/techno/downloads # ls -l /usr/src/packages/SOURCES/xmms*
-rw-rw-r-- 1 root root 4034832 2004-02-23 13:54 /usr/src/packages/SOURCES/xmms-1.2.10.tar.gz
Govinda:/users/techno/downloads # ls -l /usr/src/packages/SPECS/xmms*
-rw-rw-r-- 1 root root 6665 2004-02-23 13:57 /usr/src/packages/SPECS/xmms.spec

The RPM build tree
-----------------------
To use RPM as a tool for building software packages, we must follow some rules in terms of locations for the source code used to build this software. Expect two different kinds of input to build a package:

(1) Source
Of course we need source code to build the package (assume that we got the source without any customizations). Usually these sources are distributed as compresse tar file (tgz). RPM can handle other formats as well

(2) Spec file
The spec file is the heart of RPM's package building process. It's like a make file, as it contains information on how to build the package. But it's settled on a much higher level and contains all meta information about the package. .spec is the extension for SPEC files

Note:
It is not uncommon to have a patch included in the source files

RPM directories for building packages
----------------------------------------------------------------------------------
Directory Contents
----------------------------------------------------------------------------------
SPECS Spec files
SOURCES Source and patch files
BUILD Used to build (unpack and compile) the software package
RPMS Contains the build RPM files (sorted by architecture)
SRPMS Contains the build SRPM files


RPM Build Stages
---------------------------------------------------------------------
Flag Stage
---------------------------------------------------------------------
p prep (unpack sources and apply patches)
l list check (do some cursory checks on %files)
c compile (prep and compile)
i install (prep, compile, install)
b binary package (prep, compile, install, package)
a bin/src package (prep, compile, install, package)

Building the Package
-------------------------
After the SRPM is installed, we can continue with the RPM build stages. RPM builds packages in 4 stages:

(1) Preparation
The prep-stage is where the actual work building the binary package starts. In this stage the sources are unpacked and the patches to the sources are applied (if any). If the package builder had forseen any additional preparations which have to be done prior to compiling the sources, they will be executed in this stage

(2) Compilation
After the sources are prepared, the compilation process can take place. In this stage all source files are compiled and linked to the resulting binaries, libraries, etc.

(3) Installation
This stage installs the new software package on the system. At this point the build process changes your actual system. It puts the binaries, libraries, configuration files, etc. in the places where they belong for the new software package. No entry is made in the RPM database reflecting this installation.

Quick Note:
Installing binaries from a source package doesn't include that the package is under the control of the RPM database

(4) Packaging
The last step is to create the RPM and the SRPM for the new software package. You can turn off this stage when you just want to compile and install the package. Better create the new SRPM, if sources were modified ange the behavior of the binaries

RPM Build Options
------------------------------------------------------------------------------------
Option Meaning
------------------------------------------------------------------------------------
-bstage-spec or
-tstage-tarball build package (see table 34-2 for stage spec)
--short-circuit skip straight to specified stage (only for c,i)
--clean remove build tree when done
--rmsource remove sources and spec file when done
--buildrootdir use dir as the build root
--buildarcharch build the packages for architecture arch
--buildosos build the packages for operating system os
--test do not execute any stages
--timechecksecs set the time check to secs seconds (0 disables)
--rebuildsrpm install source package, build binary package and remove
spec file, sources, patches, and icons.
--rmsourcespec remove sources and spec file
--recompilesrpm like --rebuild, but don't build any package
------------------------------------------------------------------------------------


To start the build process use the command rpm -b with the spec file as argument. Running rpm -ba will go through all stages and result in a RPM package, a SRPM package, and the software installed in your system (but not entered in the RPM database). If you don't want to go through the whole build process and want to stop at a certain stage, you can give the stage name as additional argument to rpm

If you are only interested in looking at the source code, you can stop at prep stage. The sources are installed, unpacked, and all changes SuSE made to them are applied

Govinda:/usr/src/packages/SPECS # rpm -bp xmms.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.69504
+ umask 022
+ cd /usr/src/packages/BUILD
+ cd /usr/src/packages/BUILD
+ rm -rf xmms-1.2.10
+ /bin/gzip -dc /usr/src/packages/SOURCES/xmms-1.2.10.tar.gz
+ tar -xf -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd xmms-1.2.10
++ /usr/bin/id -u
+ '[' 0 = 0 ']'
+ /bin/chown -Rhf root .
++ /usr/bin/id -u
+ '[' 0 = 0 ']'
+ /bin/chgrp -Rhf root .
+ /bin/chmod -Rf a+rX,g-w,o-w .
+ exit 0


First the tar files gets uncompressed and unpacked. If you look into the directory /usr/src/packages/BUILD after you execute the prep stage, you'll see that there is a subdirectory, xmms-1.2.10 containing the source files for the xmms package

Govinda:/usr/src/packages/BUILD/xmms-1.2.10 # ls -l
total 1841
drwxr-xr-x 12 root root 1232 2004-02-23 13:53 .
drwxrwxrwt 3 root root 80 2004-09-28 13:22 ..
-rw-r--r-- 1 root root 9127 2004-01-11 10:22 ABOUT-NLS
-rw-r--r-- 1 root root 2110 2004-01-16 16:41 AUTHORS
-rw-r--r-- 1 root root 17992 2003-05-19 14:22 COPYING
-rw-r--r-- 1 root root 212169 2004-02-23 13:53 ChangeLog
drwxr-xr-x 5 root root 200 2004-02-23 13:53 Effect
-rw-r--r-- 1 root root 6679 2000-12-22 15:09 FAQ
drwxr-xr-x 5 root root 192 2004-02-23 13:53 General
-rw-r--r-- 1 root root 9236 2003-05-19 14:22 INSTALL
drwxr-xr-x 8 root root 256 2004-02-23 13:53 Input
-rw-r--r-- 1 root root 238 2004-01-16 16:11 Makefile.am
-rw-r--r-- 1 root root 24209 2004-02-23 13:44 Makefile.in
-rw-r--r-- 1 root root 8395 2004-02-23 13:49 NEWS
drwxr-xr-x 8 root root 264 2004-02-23 13:53 Output
-rw-r--r-- 1 root root 40371 2004-01-16 16:41 README
....
....


To continue with the example, we'll go on to the compilation stage. You could either start over and let RPM do the preparation again, or use the prepared sources and jump right into the compilation. The default for RPM is to start over at the beginning. The command rpm -bc will have the source and patch files (if any) do the preparation work and then compile the source code. Since we have gone through the preparation already, we will short-circuit RPM and only do the compilation:

Govinda:/usr/src/packages/SPECS # rpm -bc --short-circuit xmms.spec
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.13688
+ umask 022
+ cd /usr/src/packages/BUILD
+ cd xmms-1.2.10
+ unset LINGUAS
+ CFLAGS=-O2 -march=i486 -mcpu=i686
+ export CFLAGS
+ CXXFLAGS=-O2 -march=i486 -mcpu=i686
+ export CXXFLAGS
+ FFLAGS=-O2 -march=i486 -mcpu=i686
+ export FFLAGS
+ '[' -f configure.in ']'
+ libtoolize --copy --force
You should update your `aclocal.m4' by running aclocal.
+ ./configure i386-pc-linux --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info
configure: WARNING: you should use --build, --host, --target
checking build system type... i386-pc-linux
checking host system type... i386-pc-linux
...
...
make[3]: Entering directory `/usr/src/packages/BUILD/xmms-1.2.10/libxmms'
if /bin/sh ./libtool --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I. -I/usr/include/gtk-1.2 -I/usr/include/glib-1.2 -I/usr/lib/glib/include -D_REENTRANT -I/usr/X11R6/include -I../intl -I.. -O2 -march=i486 -mcpu=i686 -Wall -Wpointer-arith -MT configfile.lo -MD -MP -MF ".deps/configfile.Tpo" -c -o configfile.lo `test -f 'configfile.c' || echo './'`configfile.c; then mv -f ".deps/configfile.Tpo" ".deps/configfile.Plo"; else rm -f ".deps/configfile.Tpo"; exit 1; fi
gcc -DHAVE_CONFIG_H -I. -I. -I. -I/usr/include/gtk-1.2 -I/usr/include/glib-1.2 -I/usr/lib/glib/include -D_REENTRANT -I/usr/X11R6/include -I../intl -I.. -O2 -march=i486 -mcpu=i686 -Wall -Wpointer-arith -MT configfile.lo -MD -MP -MF .deps/configfile.Tpo -c configfile.c -o configfile.o
...
...

After this step the binaries are ready to be installed in your system. You can copy them manually if you are only interested in special parts of the package, or let RPM install the entire set. This is done with rpm -bi. Again, this command by default starts at the very beginning. But we can use the --short-circuit switch again to skip the first two stages:

# rpm -bi --short-circuit xmms.spec
Executing: %install
+ umask 022
+ cd /usr/src/packages/BUILD
+ cd rxvt-2.4.7
+ make install
./autoconf/mkinstalldirs /usr/X11R6/bin
./autoconf/mkinstalldirs /usr/X11R6/man/man1
make[1]: Entering directory `/usr/src/packages/BUILD/xmms-1.2.10/src'
....
....


Besides the missing entry in the RPM database, you now have the same status as you would have with the installed binary package.

The last stage is to create the binary and the source packages. You do this with rpm -ba. Unfortunately there is no way to skip the other stages for this option. So you have to go through the whole process again. The only choice you have is to skip building of the source package. If you want to do this use rpm -bb instead of rpm -ba. In order to have a complete example we will create the binary and the source package:

# rpm -bb xmms.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.3103
+ umask 022
+ cd /usr/src/packages/BUILD
+ cd /usr/src/packages/BUILD
+ rm -rf xmms-1.2.10
+ /bin/gzip -dc /usr/src/packages/SOURCES/xmms-1.2.10.tar.gz

...
...
Wrote: /usr/src/packages/SRPMS/xmms-1.2.10.src.rpm
Wrote: /usr/src/packages/RPMS/i386/xmms-1.2.10.i386.rpm

The build process is completed. RPM tells you where it put the source and the binary package

For the impatient
=============
Method# 1:

step (1) Install source rpm
rpm -ivh <application>.src.rpm
This decompresses the source files into /usr/src directory under which the source and spec files (building parameters) are unloaded into the ./SOURCES and ./SPECS directories, respectively.

step (2) cd into /usr/src/SPECS directory and run the following commands in order to build the source according the spec files:
rpm -bb <application>.spec
This compiles the source and packages the binaries into tidy RPM packages for you.

step (3) Install the RPM that you just built:
cd /usr/src/RPMS
rpm -Uvh <application>.i386.rpm
Method# 2:
To compile and install a source RPM package, do the following:
# rpm --rebuild <application>.src.rpm

(lots of output)

Wrote: /usr/src/packages/RPMS/i386/<application>.i386.rpm

+ exit 0

If all goes well, you can skip everything below. If not, you may want to cd /usr/src/packages/BUILD/<application>-Version, read on below, and try to figure out how to compile the application


Saturday, September 25, 2004
 
Linux: Installing Dynamic Fonts

Make sure xfs (X Font Server) is installed

1) Copy all true type/dynamic fonts to /usr/X11R6/lib/X11/fonts/truetype directory
2) Restart X Font Server as root user
/etc/init.d/xfs restart

3) Restart web browser

Alternate method:
1) Copy all true type/dynamic fonts to $HOME/.fonts
2) Restart X Font Server as root
/etc/init.d/xfs restart
3) Restart web browser

Let the browser show the fonts being used in the web page i.e., do not override the default font settings by selecting "Always use my fonts" option



Wednesday, September 22, 2004
 
UNIX/C: Program that prints Itself

Source:
% cat printme.c
#include <stdio.h>

char *s="char *s=%c%s%c;%cmain(){printf(s,34,s,34,10,10);}%c";

main() {
printf(s,34,s,34,10,10);
}

Compilation line:
% CC -o printme printme.c

Output:
% ./printme
char *s="char *s=%c%s%c;%cmain(){printf(s,34,s,34,10,10);}%c";
main(){printf(s,34,s,34,10,10);}



Tuesday, September 21, 2004
 
SPARC: Position Independent Code (PIC)

The code within a dynamic executable is usually tied to a fixed address in memory; but position-independent code can be loaded anywhere in the address space of a process. Because the code is not tied to a specific address, it will execute correctly without page modification at a different address in each process that uses it. This code creates programs that require the smallest amount of page modification at runtime.

If a shared object is built from code that is not position-independent, the text segment will usually require a large number of relocations to be performed at runtime. Although the runtime linker is equipped to handle this, the system overhead this creates can cause serious performance degradation. The compiler can generate position-independent code with -Kpic option. Ideally, any frequently accessed data items benefit from using the -Kpic model

-Kpic Vs -KPIC
----------------
Both -Kpic & -KPIC affect references to global offset table entries. The global offset table is an array of pointers, the size of whose entries are constant for 32–bit (4 bytes) and 64–bit (8–bytes).

The code sequence to make reference to an entry under -Kpic is something like:
        ld    [%l7 + j], %o0    ! load &j into %o0
Where %l7 is the precomputed value of the symbol _GLOBAL_OFFSET_TABLE_ of the object making the reference

This code sequence provides a 13–bit displacement constant for the global offset table entry, and thus provides for 2048 unique entries (2^11) for 32–bit objects, and 1024 unique entries (2^10) for 64–bit objects. If an object is built that requires more than the available number of entries, the link-editor produces a fatal error:
$ CC -Kpic -G -o libhidden.so hidden.o extras.o
ld: fatal: too many symbols require `small' PIC references:

have 2050, maximum 2048 -- recompile some modules -K PIC.
To overcome this error condition, some or all of the input relocatable objects have to be compiled with the -KPIC option

Source:
Linker & Libraries Guide, Sun Microsystems Inc.,



Monday, September 20, 2004
 
Solaris/S1S9: Scope of Symbols IV - hidden scope

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

The answer is to hide all the symbols within the module and export only those that are needed to external modules (objects). It can be achieved with the "hidden" value to -xldscope flag ie., -xldscope=hidden

An exert is from S1S9 documentation regarding hidden scope:
The symbol has hidden linker scoping. Hidden linker scoping is more restrictive than symbolic and global linker scoping. All references within a dynamic module bind to a definition within that module. The symbol will not be visible outside of the module
Time to check that with an example. Let's compile libhidden library with -xldscope=hidden
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -xldscope=hidden -c hidden.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -G -xldscope=hidden -o libhidden.so hidden.o

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -s -C libhidden.so

Symbol Table: .dynsym
index value size type bind oth ver shndx name
[0] 0x00000000 0x00000000 NOTY LOCL D 0 UNDEF
[1] 0x000006c8 0x00000000 SECT LOCL D 0 .rodata
[2] 0x000107a8 0x00000000 SECT LOCL D 0 .data
[3] 0x000107ac 0x00000000 OBJT GLOB D 0 .data _edata
[4] 0x00000000 0x00000000 OBJT GLOB D 0 ABS _PROCEDURE_LINKAGE_TABLE_
[5] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF void __Cimpl::cplus_init()
[6] 0x000006d2 0x00000000 OBJT GLOB D 0 .rodata _etext
[7] 0x000106d4 0x00000000 OBJT GLOB D 0 .dynamic _DYNAMIC
[8] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF void __Cimpl::cplus_fini()
[9] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF _ex_register
[10] 0x00000634 0x00000090 FUNC GLOB D 0 .fini _fini
[11] 0x00000538 0x000000fc FUNC GLOB D 0 .init _init
[12] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF _ex_deregister
[13] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF void __Crun::do_exit_code_in_range(void*,void*)
[14] 0x000107ac 0x00000000 OBJT GLOB D 0 .bss _end
[15] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF _get_exit_frame_monitor
[16] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF atexit

Symbol Table: .symtab
index value size type bind oth ver shndx name
[25] 0x000107a8 0x00000004 OBJT LOCL H 0 .data age
[26] 0x000004d0 0x00000020 FUNC LOCL H 0 .text char firstchar(char*)
[27] 0x00000490 0x0000002c FUNC LOCL H 0 .text char*lastname(char*)
[28] 0x00000500 0x00000038 FUNC LOCL H 0 .text int agefunc()
[30] 0x00000450 0x0000002c FUNC LOCL H 0 .text int addtentomyage(int)

Dynamic symbol table has no entries for the interfaces of the module libhidden.so; this is because all the symbols were marked as local (LOCL, ie., local to module) and hidden (H, ie., not visible outside the module) - it can be seen from the static symbol table (.symtab)

Now if we compile & link our driver program with libhidden.so, it should fail as it can't access the symbols from the module libhidden.so

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/% CC -o driver -lgeneric -lhidden driver.c
Undefined first referenced
symbol in file
int agefunc() driver.o
char firstchar(char*) driver.o
char*lastname(char*) driver.o
ld: fatal: Symbol referencing errors. No output written to driver

Great! Works as expected

Now let's see how to export the symbols that are necessary to compile & run the driver program. How do we know what all symbols we need to export (or make visible to external modules) to make the driver program work?

Simple; we just need to carefully look at the error message that was thrown by link-editor (ld) while linking driver's object file with libhidden.so to create the "driver" executable

The error says:
Undefined first referenced
symbol in file
int agefunc() driver.o
char firstchar(char*) driver.o
char*lastname(char*) driver.o
ld: fatal: Symbol referencing errors. No output written to driver

i.e., ld is not able to access agefunc(), firstchar(), lastname() interfaces from libhidden.so. Let's export those symbols.

__declspec(dllexport) specifier can be used to mark the symbol to be ready for exported. Similarly __declspec(dllimport) specifier can be used to ask the compiler driver to import that particular symbol and make it available. Compiler driver (CC) checks the symbol table for the symbol and if it finds the symbol as an interpreter (P) symbol, it just makes it available to the caller

Modified hidden.h with __declspec specifiers
-------------------------------------------------------------------
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat hidden.h
#ifdef S1S9_EXPORT
__declspec(dllexport) char *lastname(char *);
__declspec(dllexport) int agefunc();
__declspec(dllexport) char firstchar(char *);
__declspec(dllexport) int addtentomyage(int);
#else
__declspec(dllimport) char *lastname(char *);
__declspec(dllimport) int agefunc();
__declspec(dllimport) char firstchar(char *);
__declspec(dllimport) int addtentomyage(int);
#endif

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -xldscope=hidden -c hidden.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -G -xldscope=hidden -DS1S9_EXPORT -o libhidden.so hidden.o
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/% elfdump -s -C libhidden.so
Symbol Table: .dynsym
index value size type bind oth ver shndx name
[9] 0x000005d0 0x00000020 FUNC GLOB D 0 .text char firstchar(char*)
[16] 0x00000590 0x0000002c FUNC GLOB D 0 .text char*lastname(char*)
[17] 0x00000550 0x0000002c FUNC GLOB D 0 .text int addtentomyage(int)
[19] 0x00000600 0x00000038 FUNC GLOB D 0 .text int agefunc()

Symbol Table: .symtab
index value size type bind oth ver shndx name
[25] 0x000108a8 0x00000004 OBJT LOCL H 0 .data age

Observe that because of the __declspec specifiers and preprocessor symbol S1S9_EXPORT, all the interfaces were marked as GLOB & D (global & defined); but still the data variable "age" is hidden as it was not exported (we didn't export it as driver program is not accessing the variable "age" directly)

Finally run the driver program

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -o driver -lhidden driver.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver
lname = mandalika
age = 35
first char = c
It just works!! Kudos to Sun ONE Studio compiler team for introducing this new feature (actually this feature was partially supported in S1S8). With this functionality the developers can export only the symbols that are needed by external modules and keep the rest in local module; hence the application performs better during run-time with less number of i-cache misses


 
Solaris/S1S9: Scope of Symbols - Adv/Disadv of GLOB scope

Advantages of global scope:

1) Easy to use interposing libraries
2) No worries about the application during run-time; if the run-time linker finds the right set of symbols, the application works just fine
3) No need for special mechanisms like linker map files or special compiler flags to make the symbols global in scope

Disadvantages of global scope:
1) Chance of namespace collisions with 3rd party libraries
2) Run-time performance will be a little slow because of the i-cache miss% & #page faults
3) Chance of accidental/malicious interposition of global symbols
4) Can't restrict functionality available to external objects
5) size of the application binary will be big compared to the binaries with reduced symbol scopes
6) Application symbols will be exposed through static/dynamic symbol tables

More disadvantages than advantages, huh! How to overcome the disadvantages of the default scope?



Friday, September 17, 2004
 
Solaris/S1S9: Scope of Symbols :: III symbolic scope contd.,

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

The alternative to link-editor's -Bsymbolic is the compiler driver's -xldscope=symbolic flag. Sun ONE Studio 8 (aka S1S8) introducted a new flag -xldscope={global|symbolic|hidden} for specifying the appropriate linker scoping within the source program

"global" will be assumed as the value for -xldscope if none were specified. The interpretation for the value "global" from Sun's official documentation is as follows:

Symbol definitions have global linker scoping and is the least restrictive linker scoping. All references to the symbol bind to the definition in the first dynamic load module that defines the symbol. This linker scoping is the current linker scoping for extern symbols

The value "symbolic" will be interpreted as follows:

Symbol definitions have symbolic linker scoping and is more restrictive than global linker scoping. All references to the symbol from within the dynamic load module being linked bind to the symbol defined within the module. Outside of the module, the symbol appears as though it were global. This linker scoping corresponds to the linker option -Bsymbolic. Although you cannot use -Bsymbolic with C++ libraries, you can use the __symbolic specifier without causing problems

Time to re-compile & build our libhidden with the new compiler flag -xldscope=symbolic

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -xldscope=symbolic -c hidden.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -G -xldscope=symbolic -o libhidden.so hidden.o
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -o driver -lgeneric -lhidden driver.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver
lname = mandalika
age = 35
first char = c
It works!!

Simple; ain't it?

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -C -s libhidden.so
Symbol Table: .dynsym
index value size type bind oth ver shndx name
[3] 0x00010870 0x00000004 OBJT GLOB P 0 .data age
[12] 0x00000598 0x00000020 FUNC GLOB P 0 .text char firstchar(char*)
[15] 0x00000558 0x0000002c FUNC GLOB P 0 .text char*lastname(char*)
[19] 0x00000518 0x0000002c FUNC GLOB P 0 .text int addtentomyage(int)
[20] 0x000005c8 0x00000038 FUNC GLOB P 0 .text int agefunc()
All the symbols of libhidden are still "global" (GLOB) & Protected (P)

What are the advantages & disadvantages of having all symbols in default (global) scope?

(to be continued ..)


Thursday, September 16, 2004
 
Solaris/S1S9: Scope of Symbols - III symbolic scope

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

The default left-to-right dependency ordering can be overridden by using the link-editor (ld) flag: -Bsymbolic

-Bsymbolic works in dynamic mode only i.e., while creating dynamic objects like shared objects. This flag informs the link-editor to bind all references to global symbols to their definitions, if available within the object.

This is the flag to be used in our example to get the correct result, as we want to bind the call to addtentomyage() to the definition available within hidden.c

Let's create libhidden again with -Bsymbolic flag

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -G -Bsymbolic -o libhidden.so hidden.o

Now let's build & run the driver program again:
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -o driver -lgeneric -lhidden driver.c

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver
lname = mandalika
age = 35
first char = c
Cool; it works!! - got the desired result

Let's examine the symbol table for the scope

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -C -s libhidden.so

Symbol Table: .dynsym
index value size type bind oth ver shndx name
[3] 0x00010878 0x00000004 OBJT GLOB P 0 .data age
[12] 0x00000598 0x00000020 FUNC GLOB P 0 .text char firstchar(char*)
[15] 0x00000558 0x0000002c FUNC GLOB P 0 .text char*lastname(char*)
[19] 0x00000518 0x0000002c FUNC GLOB P 0 .text int addtentomyage(int)
[20] 0x000005c8 0x00000038 FUNC GLOB P 0 .text int agefunc()

All the symbols were still global (GLOB), but now marked as "Protected" (P)

Advantage:
No worries about name collisions

Disadvantage:
-Bsymbolic was intended for specialized dynamic objects and is not recommended for general use. That means, there is no support for applications that were built with -Bsymbolic. But still we need the functionality to avoid name collisions. Is there any alternate way to achieve the functionality of -Bsymbolic?

(to be continued ..)


Wednesday, September 15, 2004
 
Solaris/S1S9: Scope of Symbols: Default scope contd.,

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

Lets add one more local method called "addtentomyage()" in hidden.h

hidden.h
---------
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat hidden.c
#include
#include "hidden.h"

int age = 25;

int addtentomyage(int age)
{
return ((int) age + 10);
}

char *lastname(char *firstname)
{
return ((char *)"mandalika");
}

char firstchar(char *strings)
{
return (*strings);
}

int agefunc()
{
int finalage = 0;

finalage = addtentomyage(age);
return (finalage);
}


Create libhidden.so and run the driver program

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/% CC -c hidden.c ; CC -G -o libhidden.so hidden.o ; CC -lhidden -o driver driver.c

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver

lname = mandalika
age = 35
first char = c
Cool; got the expected result

Now lets introduce one more library called "libgeneric" with a buggy implementation for addtentomyage()

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat generic.c
#include "hidden.h"

int addtentomyage(int age)
{
return ((int) age + 100);
}
Compile & create a new library called "libgeneric"

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -c generic.c ; CC -G -o libgeneric.so generic.o

Now link the new library (libgeneric) with the driver executable as follows (observe that libgeneric is placed before libhidden in the compile line):

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -o driver -lgeneric -lhidden driver.c

Run the driver program and observe the result:
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver

lname = mandalika
age = 125
first char = c
That's not good; the expected age was 35, but got 125! What's wrong?

Even though there is an implementation available inside the libhidden module itself, linker picked up the implementation from libgeneric by blindly following the left to right rule (ref: Linker & Libraries guide by Sun Microsystems). And in general references to global symbols within shared objects are not bound until runtime, even if definitions are available, so that definitions of the same symbol in an executable or other shared object can override the object's own definition

The tool "ldd" will be useful for dumping the dependency ordering
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ldd driver
libgeneric.so => /sunbuild1/giri/testcases/symbol-hiding/libgeneric.so
libhidden.so => /sunbuild1/giri/testcases/symbol-hiding/libhidden.so
libCstd.so.1 => /usr/lib/libCstd.so.1
libCrun.so.1 => /usr/lib/libCrun.so.1
libm.so.1 => /usr/lib/libm.so.1
libw.so.1 => /usr/lib/libw.so.1
libc.so.1 => /usr/lib/libc.so.1
libdl.so.1 => /usr/lib/libdl.so.1
/usr/lib/cpu/sparcv8plus/libCstd_isa.so.1
/usr/platform/SUNW,Ultra-Enterprise/lib/libc_psr.so.1
Advantage:
The application can take advantage of interposing libraries i.e., if needed, the application can enjoy an alternate implementation for the same set of interfaces if available

Disadvantage:
Name collisions (just like the one shown in the example); due to the collision the application may break (our example shows that broken part - expected value: 35, but the returned value is: 125)

How to override this default behavior and make it pick the right implementation?

(to be continued ..)


Tuesday, September 14, 2004
 
Solaris/S1S9: Scope of Symbols - II :: Default scope

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

Default behavior
----------------

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -c hidden.c

scope of symbols inside object "hidden.o":

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -C -s hidden.o
Symbol Table: .symtab
index value size type bind oth ver shndx name
[3] 0x00000080 0x0000002c FUNC GLOB D 0 .text int agefunc()
[4] 0x00000050 0x00000020 FUNC GLOB D 0 .text char firstchar(char*)
[5] 0x00000010 0x0000002c FUNC GLOB D 0 .text char*lastname(char*)
[6] 0x00000000 0x00000004 OBJT GLOB D 0 .data age


All symbols are "global" & "defined" (GLOB & D)

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ls -l
total 10
-rw-r--r-- 1 build engr 281 Sep 14 16:03 driver.c
-rw-r--r-- 1 build engr 223 Sep 14 16:00 hidden.c
-rw-r--r-- 1 build engr 63 Sep 14 15:58 hidden.h
-rw-r--r-- 1 build engr 1468 Sep 14 16:00 hidden.o


bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -G -o libhidden.so hidden.o
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ls -l
total 22
-rw-r--r-- 1 build engr 281 Sep 14 16:03 driver.c
-rw-r--r-- 1 build engr 223 Sep 14 16:00 hidden.c
-rw-r--r-- 1 build engr 63 Sep 14 15:58 hidden.h
-rw-r--r-- 1 build engr 1468 Sep 14 16:00 hidden.o
-rwxr-xr-x 1 build engr 5256 Sep 14 16:05 libhidden.so

scope of symbols inside object "libhidden.so"

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -C -s libhidden.so
Symbol Table: .symtab
index value size type bind oth ver shndx name
[1] 0x00000000 0x00000000 FILE LOCL D 0 ABS libhidden.so
[40] 0x00010828 0x00000004 OBJT GLOB D 0 .data age
[46] 0x00000558 0x00000020 FUNC GLOB D 0 .text char firstchar(char*)
[54] 0x00000518 0x0000002c FUNC GLOB D 0 .text char*lastname(char*)
[56] 0x00000588 0x0000002c FUNC GLOB D 0 .text int agefunc()


Global & Defined

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ls -l
total 22
-rw-r--r-- 1 build engr 281 Sep 14 16:03 driver.c
-rw-r--r-- 1 build engr 223 Sep 14 16:00 hidden.c
-rw-r--r-- 1 build engr 63 Sep 14 15:58 hidden.h
-rw-r--r-- 1 build engr 1468 Sep 14 16:00 hidden.o
-rwxr-xr-x 1 build engr 5256 Sep 14 16:05 libhidden.so

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%setenv LD_LIBRARY_PATH /sunbuild1/giri/testcases/symbol-hiding:$LD_LIBRARY_PATH

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -lhidden -o driver driver.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ls -l
total 148
-rwxr-xr-x 1 build engr 63680 Sep 14 16:11 driver
-rw-r--r-- 1 build engr 297 Sep 14 16:11 driver.c
-rw-r--r-- 1 build engr 223 Sep 14 16:00 hidden.c
-rw-r--r-- 1 build engr 63 Sep 14 15:58 hidden.h
-rw-r--r-- 1 build engr 1468 Sep 14 16:00 hidden.o
-rwxr-xr-x 1 build engr 5256 Sep 14 16:05 libhidden.so

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%./driver
lname = mandalika
age = 25
first char = c

It just *Works*

scope of symbols inside executable program "driver":

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%elfdump -C -s driver
Symbol Table: .symtab
index value size type bind oth ver shndx name
[90] 0x00028f6c 0x00000000 FUNC GLOB D 0 UNDEF char firstchar(char*)
[122] 0x00028f60 0x00000000 FUNC GLOB D 0 UNDEF int agefunc()
[197] 0x00028f48 0x00000000 FUNC GLOB D 0 UNDEF char*lastname(char*)

i.e., the default symbol scope on Solaris (UNIX® infact) is "Global" & "Defined"


 
Solaris/S1S9: Scope of Symbols - I :: Source files

[Updated: 04/07/2006] Much accurate information is available in a better format at:
Reducing Symbol Scope with Sun Studio C/C++
__________________

bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%which CC
/sunbuild3/S1S9_0609/SUNWspro/bin/CC
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%CC -V
CC: Sun C++ 5.6 d-iteam 2004/06/09
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%ls -l
total 8
-rw-r--r-- 1 build engr 215 Sep 14 15:59 hidden.c
-rw-r--r-- 1 build engr 63 Sep 14 15:58 hidden.h
-rw-r--r-- 1 build engr 281 Sep 14 16:03 driver.c
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat hidden.h
char *lastname(char *);
int agefunc();
char firstchar(char *);
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat hidden.c
#include
#include "hidden.h"

int age = 25;

char *lastname(char *firstname)
{
return ((char *)"mandalika");
} //lastname

char firstchar(char *strings)
{
return (*strings);
} // firstchar

int agefunc()
{
age = 25;
return (age);
} // agefunc
bpte4500s001:/sunbuild1/giri/testcases/symbol-hiding/%cat driver.c
#include
#include "hidden.h"

int main()
{
char *lname, first;
int age = 0;

lname = lastname((char *)"Giri");
printf("\nlname = %s", lname);

age = agefunc();
printf("\nage = %d", age);

first = firstchar((char *)"chakry");
printf("\nfirst char = %c\n", first);

return (0);
} // main()



Thursday, September 09, 2004
 
Solaris & UltraSPARC IV: Displaying processor information

"psrinfo" displays information about processors and virtual processors (USIV supports virtual processors; Each physical processor may support multiple virtual processors & each virtual processor may support multiple virtual processors)

govinda@/users/techno> psrinfo
0 on-line since 09/03/2004 17:17:07
1 no-intr since 09/05/2004 14:11:56
2 no-intr since 09/05/2004 14:11:56
3 no-intr since 09/05/2004 14:11:56
512 on-line since 09/03/2004 17:17:14
513 no-intr since 09/05/2004 14:11:56
514 no-intr since 09/05/2004 14:11:56
515 no-intr since 09/05/2004 14:11:56

"psrinfo -p" display the number of physical processors in a system; shows additional information about each physical processor with "-v" flag. -v stands for verbose mode; with -v flag, "psrinfo" displays information about processor type, floating point unit type and clock speed. If any of this information cannot be determined, psrinfo displays unknown.

govinda@/users/techno> psrinfo -p
4

govinda@/users/techno> psrinfo -pv
The UltraSPARC-IV physical processor has 2 virtual processors (0, 512)
The UltraSPARC-IV physical processor has 2 virtual processors (1, 513)
The UltraSPARC-IV physical processor has 2 virtual processors (2, 514)
The UltraSPARC-IV physical processor has 2 virtual processors (3, 515)

govinda@/users/techno> psrinfo -v
Status of virtual processor 0 as of: 09/09/2004 20:57:59
on-line since 09/03/2004 17:17:07.
The sparcv9 processor operates at 1200 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 09/09/2004 20:57:59
no-intr since 09/05/2004 14:11:56.
The sparcv9 processor operates at 1200 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 09/09/2004 20:57:59
no-intr since 09/05/2004 14:11:56.
The sparcv9 processor operates at 1200 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 09/09/2004 20:57:59
no-intr since 09/05/2004 14:11:56.
The sparcv9 processor operates at 1200 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 512 as of: 09/09/2004 20:57:59
on-line since 09/03/2004 17:17:14.
The sparcv9 processor operates at 1200 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 513 as of: 09/09/2004 20:57:59

Reference:
Solaris man page of "psrinfo"



Tuesday, September 07, 2004
 
JDS: Installing Sun Java Desktop System 2.0

This document will guide you through the process of installing JDS 2.0 on a PC from integrated CDROM images

Requirements

In order to complete the installation you will require the following:


 
Solaris: malloc Vs mtmalloc

Performance of Single Vs Multi-threaded application

Memory allocation performance in single and multithreaded environments is an important aspect of any application. Some allocators, such as malloc on Solaris Operating Environment work best with single-threaded applications, but degrades performance with a multi-threaded application. As memory is being allocated concurrently in multiple threads, all the threads must wait in a queue while malloc() handles one request at a time. With a few extra threads, this can slow down performance, causing a problem known as heap contention i.e., all the threads compete for access to the same heap. If the application is making a considerably high number of calls to malloc() that will be an indication of heap contention

One solution to alleviate the problem is to use multiple heaps. (heap is nothing but a block(s) of memory that a process can allocate or free at run time). Every process will have its own heap i.e., heap is private to the process. If there are multiple heaps, the threads do not need to wait for a heap's lock; instead each thread can use the one allocated to the CPU it is running on

Heap Usage

When an application executes, the operating system creates a virtual address space for the new process. This address space contains will have the stack, application's executable code, data, and the heap. The heap is where dynamic memory allocation occurs. Various system calls are used to manage a process's heap; brk() and sbrk() system calls can be used to resize a process's data segment to accommodate requests for more memory allocations

malloc library

malloc on Solaris performs well only in single-threaded applications. Its performance is unsuitable for multithreaded applications, because its performance worsens as threads are added. This is due to the fact that malloc maintains a single heap, and thus only one thread can allocate/deallocate memory at any given time. Therefore, with the addition of more threads, we find more threads waiting, and the wait time grows longer, resulting in increasingly slow execution times. However, since only one memory operation is going on at a time, the overall heap usage is low. While low heap usage is good, the slow performance of malloc in multithreaded applications still should not justify using it in such applications.

mtmalloc library

mtmalloc library of Solaris was designed to address the problems faced by multi-threaded applications with single heap malloc library. Multi-threaded applications scale well by taking advantage of having concurrent access to heap by multiple threads. However memory footprint is a big tradeoff in using mtmalloc. Heap usage for mtmalloc was much higher compared to the traditional malloc. This is most likely because mtmalloc is a power of two allocator, i.e., it rounds the size of requested blocks up to the next power of two. For example, if a request is made to allocate 17 bytes, the actual size of the allocated block would be rounded up to 32 bytes. As these requests add up, it can lead to internal fragmentation, so that the "extra" memory that is allocated is wasted

<more about mtmalloc>
<example with pmap -x output showing heap allocation from malloc, mtmalloc libraries>


Monday, September 06, 2004
 
Solaris: Recovering from a Runtime Linker Failure

ld.so.1 Failure: Cannot execute /usr/lib/ld.so.1

Imagine a situation where you are playing with the runtime linker (ld.so.1) and by accident you overwrite the default runtime linker with another linker that is incompatible or buggy. All of a sudden all dynamically linked applications stop working, including the user and system utility commands, and you see this error message:

Cannot execute /usr/lib/ld.so.1, Killed

Do not panic and do not close the terminal window in which you are working. A couple of simple steps will bring the system back to the normal state. You can use statically linked mv and cp binaries and a backup copy of ld.so.1 to recover from the runtime linker failure. The Solaris Operating System keeps statically linked utilities cp, mv, ln, rcp, and tar under /usr/sbin/static. The replica of ld.so.1 can be found under /etc/lib.

The recovery steps are as follows:
1) /usr/sbin/static/mv /usr/lib/ld.so.1 /usr/lib/ld.so.1.buggy
2) /usr/sbin/static/cp /etc/lib/ld.so.1 /usr/lib

Here is a sample scenario (with the problem and the resolution):
# pwd
/data/sunperf
# ls
ld.so.1.buggy oracle.csh myapp* tools/
# cd /usr/lib
# mv /data/sunperf/ld.so.1.buggy ld.so.1
# ls
ls: Cannot execute /usr/lib/ld.so.1
Killed
# date
date: Cannot execute /usr/lib/ld.so.1
Killed
# file /usr/sbin/static/cp
file: Cannot execute /usr/lib/ld.so.1
Killed
# which file
which: Cannot execute /usr/lib/ld.so.1
Killed
# /data/sunperf/myapp
/data/sunperf/myapp: Cannot execute /usr/lib/ld.so.1
Killed
# rm ld.so.1
rm: Cannot execute /usr/lib/ld.so.1
Killed
# /usr/sbin/static/mv ld.so.1 ld.so.1.buggy
# ls
ls: Cannot find /usr/lib/ld.so.1
Killed
# /usr/sbin/static/cp /etc/lib/ld.so.1 ld.so.1
# ls
0@0.so.1 libl.so.1 llib-lcmd
32 libm.a llib-lcmd.ln
64 libmail.a llib-lcpc
abi libmail.so llib-lcpc.ln
accept libmail.so.1 llib-lcrypt
.........
(output truncated)
# date
Fri Mar 19 12:30:15 PST 2004
...
The following threads on Experts Exchange address the same problem: http://www.experts-exchange.com/Operating_Systems/Solaris/Q_209 10530.html
http://www.experts-exchange.com/Operating_Systems/Solaris/Q_101 18801.html

Additional Notes :
Static binaries cp, mv, ln, rcp, and tar are part of the SUNWsutl package. Make sure that you have installed this package on your box running the Solaris OS:

$ pkginfo -x SUNWsutl
SUNWsutl Static Utilities
(sparc) 11.8.0,REV=2000.01.08.18.12

The package SUNWsutl is available on both the SPARC and x86 Platform Editions of the Solaris OS, and these recovery instructions apply for both platforms.




 
Linux: Finding out the amount of free & used memory

The command free can be used to display the total amount of free and used physical and swap memory in the system,as well as the shared memory and buffers used by the kernel

techno/work> which free
/usr/bin/free

/users/techno> free
total used free shared buffers cached
Mem: 384888 314220 70668 0 19740 57504
-/+ buffers/cache: 236976 147912
Swap: 524280 81232 443048

Interpreting the output of free:
All the numbers are reported in 1024-byte blocks. Here, we see a system with 384,888 blocks (about 384 MB) of physical RAM, with 314,220 (about 306 MB) currently in use. The "shared" column lists the amount of physical memory shared between multiple processes. Here, we see that about 0 MB of pages are being shared (not a good sign; memory is not being utilized well). The "buffers" column shows the amount of memory being used by the kernel buffer cache. The buffer cache is used to speed up disk operations, by allowing disk reads and writes to be serviced directly from memory. The buffer cache size will increase or decrease as memory usage on the system changes; this memory is reclaimed if it is needed by applications. Therefore, although we see that 306 MB of system memory is in use, not all (but most) of it is being used by application programs. The "cache" column indicates how many memory pages the kernel has cached for faster access later. Since the memory used for buffers and cache can easily be reclaimed for use by applications, the second line (-/+ buffers/cache) provides an indication of the memory actually used by applications (the "used" column) or available to applications (the "free" column). The sum of the memory used by buffers and cache reported in the first line is subtracted from the total used memory and added to the total free memory to give the two figures on the second line. In the third line, we see the total amount of swap, 524,280 blocks (about 511 MB). In this case, only very little of the swap is being used; there is plenty of physical RAM available. If additional applications were started, larger parts of the buffer cache memory would be used to host them. Swap space is generally used as a last resort when the system can't reclaim physical memory in other ways. Note that the amount of swap reported by free is somewhat less than the total size of your swap partitions and files. This is because several blocks of each swap area must be used to store a map of how each page in the swap area is being utilized. This overhead should be rather small; only a few kilobytes per swap area.

References:
1) Man page of free
2) O`Reilley's Running Linux


Sunday, September 05, 2004
 
UNIX®: How to build a Shared Library (*.so file)

It's very easy to build a Shared Library on Solaris (and *nix in general). However, the Solaris link-editor (/usr/ccs/bin/ld)has quite a few options. It can sometimes be very confusing to determine which options to use. Some of the general rules are outlined here:

Let the compiler do the work

First - do not invoke the link-editor (/usr/ccs/bin/ld) directly, instead run it via your compiler driver (cc/gcc/CC/g++). The compiler driver will include many magic files (crt*.o, values-xa.o, ...) which are important to the construction of your shared library. If you invoke the link-editor directly you will likely miss those files and the benefits they give you. The first thing that will fail if you drop the comiler supplied files is that your _init and _fini routines will not run when the shared library is loaded.

Use PIC code

Compile code destined for the shared library as Position Independent Code. You do this by using the cc -Kpic or gcc -fpic options, depending upon which compiler you are using. PIC code is the most flexible and efficient code for a shared library because it permits the final library to be loaded at any address in memory without having to be modified at run-time. This permits the sharing of the library among all processes which have loaded it on a given system - hence the name Shared Library. If you include non-PIC code in a shared library, it will still run but you loose the share ability of the library. You can enforce that all code in the shared object is PIC by adding the -z text option to your link line.

List your dependencies

Always list all of your libraries dependencies (what you link against) when building the shared library. By listing them at link-edit time the built library will have recorded in it what libraries it needs at run-time. You can enforce this at build time by adding the -z defs option to your link line.

Include the proper RunPaths

By default the run-time linker (ld.so.1) will always search /lib & /usr/lib (or /lib/64 & /usr/lib/64 for 64bit processes) to find a shared library at run-time. If your library is referencing objects not in those directories you must also include a RunPath to find the dependencies. This is done by adding a -R {path} for each directory containing a shared libraries being referenced by the current object being built. You should not depend upon LD_LIBRARY_PATH being set at run-time to find shared libraries. Applications and dependencies can be built to include a runpath using ld(1), and the -R option. This path is used to search for the dependencies of the object in which the runpath is recorded. If the dependencies are not in a constant location, use the $ORIGIN token as part of the pathname

Example

Following is a example link demonstrating all of the options that were recommended above

% cc -Kpic bar.c -G -o bar.so -z text -z defs -L /tmp/play/lib -R /tmp/play/lib -lfoo -lc
% ldd -r bar.so
libfoo.so.1 => /tmp/play/lib/libfoo.so.1
libc.so.1 => /lib/libc.so.1
/platform/SUNW,Sun-Blade-1000/lib/libc_psr.so.1
libm.so.2 => /lib/libm.so.2
%

Note that you can use ldd to verify that your shared library is self contained and that it has the proper RunPath's defined. If either of those hadn't been true then ldd would have given appropriate error messages

Acknowledgements:
Michael Walker & Rod Evans of Sun Microsystems




2004-2019 

This page is powered by Blogger. Isn't yours?