Bjoern Olausson

Okay, I finally run all flavours of NAMD 2.7b1 and got some help from Chao Mei.

Actually I wasn't able to get the runtime configuration for the SMP/PXSHM version right (I couldn't assign the processes to nodes so that every process launches x threads on each node [1 Process per Node with 11 Threads would have been the optimum in my case]).

Anyway, I figured out the following.

Use "multicore" as long as you do not use any interconnect.

If you need to use some interconnect, go for ConnectX DDR Infiniband and use the charm (6.1.2) build in IB support (ibverbs).

For me the ibversion didn't work with SSH so I was forced to use RSH which limited me to max 240 Cores, then my system ran out of Ports or into some other limits...
That's why there are only results for > 240 Cores available for the mvapich version which uses SSH for the job distribution.

[EDIT] By adding "++batch 32 ++timeout 300" right after charmrun I am now able to submit the jobs to >240 Cores[/EDIT]

I couldn't benchmark against mvapich v1.1 or mvapich 1.4 cause I wasn't able to compile charm against them. I Didn't test any of the net versions cross nodes, cause when you can use Infiniband, ethernet gets uninteresting :-)

My setup:

Nodes: 24 Nodes each containing 'Dual Six-Core AMD Opteron 2427@2,2GHz' == 288 Cores
Interconnect: Mellanox MT26418 (ConnectX IB DDR, PCIe 2.0 5GT/s)
Charm 6.1.2
Namd 2.7b1
Intel ICC 11.0.083
MVAPICH 1.2
SGE 2.6
Scientific Linux SL release 5.3 (Boron)

Attachments:

File	File size	Last Modified
Compiling-NAMD.sh	4 Kb	08/20/09 17:48
ibverb_vs_mvapich.agr	10 Kb	08/20/09 17:34
Linux-x86_64-icc.mpi-linux-x86_64-mvapich1_2.dat	2 Kb	08/20/09 16:41
Linux-x86_64-icc.multicore-linux64-icc.dat	0 Kb	08/20/09 16:41
Linux-x86_64-icc.net-linux-x86_64-ibverbs-icc.dat	2 Kb	08/20/09 16:42
Linux-x86_64-icc.net-linux-x86_64-icc.dat	0 Kb	08/20/09 16:42
nextcore_allq.sh	6 Kb	08/20/09 20:49
single-node.agr	12 Kb	08/20/09 17:34