Category Archives: Uncategorized

Java performance on Niagara processors ( UltraSPARC T1/T2/T3 )

This problem known as much time as Niagara processor but still a lot of Java applications suffer from it
Some JVM parameters vary depending on the machine type. For example JVM will set different garbage collectors for different types of machines.
At start JVM detects that Niagara system has a lot of “cpu’s” and so this is a server and if garbage collector is not set it will use parallel gc. And this garbage collector creates a number of gc threads, which is comparable to number of “cpu’s”. ( looks like correct formula for today is : (ncpus <= 8) ? ncpus : 3 + ((ncpus * 5) / 8) ) So you will have really a lot gc threads per JVM at your Niaraga system. And it isn't really a big problem until you use just a few number of JVM's. But if your application use really a lot of JVM's or, for example, you use your Niagara server to consolidate some Java applications this overhead will be significant. Last time I saw this problem at T5440 with just 74 JVM's. And it led to periodical hangs of the whole server. Yes, this is a bug. But there are a lot of Java applications at Niagara servers which use rather old JVM right now. And some of them experiencing performance problems... Solution is really simple. You can set number of gc threads by your hands :

-XX:ParallelGCThreads=N 

Where 4 will be a good N for most of applications.
This can not only solve performance problems with some apps but also free some servers resources.

What processes are swapped? (w vmstat field)

There are some fields in vmstat output, which are hard to interpret. One of them is “w” I think. Yes, from man page we know, that it is “the number of swapped out lightweight processes (LWPs) that are waiting for processing resources to finish.”. So what we ought to do if it is null?
In most cases, it means that some time ago (now?) this server is low on memory. But what can we do if we solve this problem but this value doesn’t change?

bash-3.2# vmstat 5
 kthr      memory            page            disk          faults     cpu
r b w   swap  free  re  mf pi po fr de sr s6 sd sd sd   in   sy   cs us sy id
0 0 84 5041144 290400 1  6  0  0  0 1176 0 0  0  0  7 4231  164  838  0 2 98
0 0 84 5041136 290456 0  0  0  0  0 376 0  0  0  0  0 4228  148  575  0 2 98
0 0 84 5041136 290456 0  0  0  0  0  0  0  0  0  0  0 4232  162  579  0 2 98
0 0 84 5041128 290448 0  0  0  0  0  0  0  0  0  0  0 4227  149  637  0 2 98

First of all we can see lwp’s from what processes are swapped :

echo "::walk thread thr |::print kthread_t t_schedflag|::grep .==0 |::eval p_pidp->pid_id" | mdb -k

Or command name :

echo "::walk thread thr |::print kthread_t t_schedflag|::grep .==0 |::eval p_user.u_comm" | mdb -k

So we know lwps of what processes are still swapped. If this number in vmstat doesn’t grow, in most cases it isn’t a problem, but for a lot of people it is alarm. So to take processes date back to memory from swap we must ask this process to do something, for example sending SIGHUP to it, or just trying to truss it. Truss will need some date about process ant it will go out from swap.

Konsole tab names by script. Ssh wrapper

I have a lot of small scripts, which are very useful for me, but the famous one by my friends is ssh wrapper for Konsole.
At work I usually have a lot of opened ssh sessions to different servers. My work system is linux notebook and Konsole as default terminal emulator. So some years ago I wrote small ssh wrapper, which set name to Konsole tab by last argument to ssh (hostname).

antony@amaliya:$ cat /usr/local/bin/ssh
#!/bin/bash
# Kocsole wrapper around ssh to rename tabs
# anton@pavlenko.net

REAL_SSH=/usr/bin/ssh
DCOP=/opt/kde/bin/dcop

if [ ! -z "$KONSOLE_DCOP_SESSION" ]
then

 # Use the last argument as the title
 for arg in $@; do
 NEW_TITLE="$arg"
 done

 OLD_TITLE=`dcop "$KONSOLE_DCOP_SESSION" sessionName`
 $DCOP "$KONSOLE_DCOP_SESSION" renameSession "$NEW_TITLE"

 function restore_title() {
 $DCOP "$KONSOLE_DCOP_SESSION" renameSession "$OLD_TITLE"
 }

 # If SSH is interrupted (CTRL-C), restore the old title
 trap "restore_title" SIGINT
 $REAL_SSH $*
 restore_title

else
 $REAL_SSH $*
fi

So if you have the same problems – use this wrapper!

resource controls and zones

Nowadays Solaris Zones and resource control became more and more popular. But when you use it output of usual Solaris commands became confusing. For example if you use zone.max-swap rctl swap -l in and other tools in Zone will return info about swap for global zone, without of any restrictions. The same for nlwps resource control and others.
So how can we get real amount of used swap or lwps? And not only from global zone.
kstat can help.
There is zone_caps class in kstat, so kstat -c zone_caps -n swapresv_zone_1 will show all info about zone.max-swap for zone, with id 1.

To list oll used resource controls :

kstat -p caps:::usage

Service to pid and vice versa

In Solaris 10 widely used concept of “service”.
Each service can start some daemons, but information, what service start this or that daemon sometimes isn’t obvious.
In Solaris 10 each process has new field :ctid – contract ID of the process.
so, ps -ef -o pid,ctid,comm will show you contract ID for all processes.
And
svcs -a -o FMRI,CTID – about Contract ID for each service.
Now you can find all processes, for ftp service, for example :

ps -ef -o pid,ctid,comm |grep `svcprop -p restarter/contract svc:/network/ftp:default`

suid script

In solaris ( all other Unix’es too?) most script interpretators will ignore suid bit, and the effective user id is set to the real user id. To avoid this you can use specific options. For example -p for ksh and bash.