Pages

Monday, September 06, 2004

Solaris: Recovering from a Runtime Linker Failure

ld.so.1 Failure: Cannot execute /usr/lib/ld.so.1

Imagine a situation where you are playing with the runtime linker (ld.so.1) and by accident you overwrite the default runtime linker with another linker that is incompatible or buggy. All of a sudden all dynamically linked applications stop working, including the user and system utility commands, and you see this error message:

Cannot execute /usr/lib/ld.so.1, Killed

Do not panic and do not close the terminal window in which you are working. A couple of simple steps will bring the system back to the normal state. You can use statically linked mv and cp binaries and a backup copy of ld.so.1 to recover from the runtime linker failure. The Solaris Operating System keeps statically linked utilities cp, mv, ln, rcp, and tar under /usr/sbin/static. The replica of ld.so.1 can be found under /etc/lib.

The recovery steps are as follows:
1) /usr/sbin/static/mv /usr/lib/ld.so.1 /usr/lib/ld.so.1.buggy
2) /usr/sbin/static/cp /etc/lib/ld.so.1 /usr/lib

Here is a sample scenario (with the problem and the resolution):
# pwd
/data/sunperf
# ls
ld.so.1.buggy oracle.csh myapp* tools/
# cd /usr/lib
# mv /data/sunperf/ld.so.1.buggy ld.so.1
# ls
ls: Cannot execute /usr/lib/ld.so.1
Killed
# date
date: Cannot execute /usr/lib/ld.so.1
Killed
# file /usr/sbin/static/cp
file: Cannot execute /usr/lib/ld.so.1
Killed
# which file
which: Cannot execute /usr/lib/ld.so.1
Killed
# /data/sunperf/myapp
/data/sunperf/myapp: Cannot execute /usr/lib/ld.so.1
Killed
# rm ld.so.1
rm: Cannot execute /usr/lib/ld.so.1
Killed
# /usr/sbin/static/mv ld.so.1 ld.so.1.buggy
# ls
ls: Cannot find /usr/lib/ld.so.1
Killed
# /usr/sbin/static/cp /etc/lib/ld.so.1 ld.so.1
# ls
0@0.so.1 libl.so.1 llib-lcmd
32 libm.a llib-lcmd.ln
64 libmail.a llib-lcpc
abi libmail.so llib-lcpc.ln
accept libmail.so.1 llib-lcrypt
.........
(output truncated)
# date
Fri Mar 19 12:30:15 PST 2004
...
The following threads on Experts Exchange address the same problem: http://www.experts-exchange.com/Operating_Systems/Solaris/Q_209 10530.html
http://www.experts-exchange.com/Operating_Systems/Solaris/Q_101 18801.html

Additional Notes :
Static binaries cp, mv, ln, rcp, and tar are part of the SUNWsutl package. Make sure that you have installed this package on your box running the Solaris OS:

$ pkginfo -x SUNWsutl
SUNWsutl Static Utilities
(sparc) 11.8.0,REV=2000.01.08.18.12

The package SUNWsutl is available on both the SPARC and x86 Platform Editions of the Solaris OS, and these recovery instructions apply for both platforms.

2 comments:

  1. soalris - cannot find ld.so.1 HELP
    I have a situation...
    my problem started when I renamed ld.so.1 to ld.so.1.orig. Per sun.com, execute ../static/mv to recover, however, SUNWsutl Static Utilities is not installed on my solaris box. Also, I got kicked out of my terminal session and I cannot even connect.
    recommendation/suggestion/help would be very appreciated.

    ReplyDelete
  2. Try to boot from Solaris bootable CD-ROM, into single user mode. The steps are as follows:

    1) Put the CD-ROM into CD drive and reboot the system.

    2) From the boot solaris menu, select CD-ROM drive and then type "b -s" at the prompt.

    The other way is the type "boot cdrom -s" at "ok" prompt.

    3) The above step takes you to single user mode with a functional but limited Solaris access from CD-ROM.

    4) To access your file system(s), you need to mount them. So, atleast mount the root file system (/).

    eg.,
    If your root file system is c0t0d0s0, mount it as follows:

    mount /dev/dsk/c0t0d0s0 /mnt

    Note:
    If you don't know the partition name where / is installed, have a look at /mnt/etc/vfstab file.

    5)Then copy back the original run-time linker (ld.so.1). So, you have to do the following:

    cp /mnt/usr/lib/ld.so.1.orig /mnt/usr/lib/ld.so.1.

    6) Once you are done, simply reboot.

    ReplyDelete