<div dir="ltr"><div><div>Hello dear Support and Users, <br><br>We are having an issue with Molpro using the SGE grid scheduler. The annoying thing is that it used to work fine and we upgraded our system and it stopped working, but not straight away. We have not re-compiled MolPro with <br>-auto-ga-openmpi-sge<br>(we compiled it like this before) should we? <br><br>The issue is as follows: <br><br>when we launch molpro through SGE the executable (or wrapper) cannot start the MPI copies of the executable <br><br>/share/apps/MOLPRO_MPP/parallel.x /share/apps/MOLPRO_MPP/bin/molpro.exe  -v SAMPL5_051.in<br>compute-0-32.local: Connection refused<br>tmp = /home/gkamath/pdir//share/apps/MOLPRO_MPP/bin/molpro.exe.p<br> Creating: host=compute-0-32.local, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=59106<br>  1: interrupt(1)<br><br>connection is refused. (the full output is below). <br><br> # PARALLEL mode<br> nodelist=4<br> first   =4<br> second  =<br> third   =<br> HOSTFILE_FORMAT: $user $hostname 1 $exe $working_dir<br><br>gkamath compute-0-32.local 1 /share/apps/MOLPRO_MPP/bin/molpro.exe /home/gkamath/sample<br><br> export LD_LIBRARY_PATH=':/opt/gridengine/lib/linux-x64:/opt/openmpi/lib:/opt/python/lib'<br> export AIXTHREAD_SCOPE='s'<br> export MOLPRO_PREFIX='/share/apps/MOLPRO_MPP'<br> export MP_NODES='0'<br> export MP_PROCS='1'<br>        MP_TASKS_PER_NODE=''<br> export MOLPRO_NOARG='1'<br> export MOLPRO_OPTIONS=' -v SAMPL5_051.in'<br> export MOLPRO_OPTIONS_FILE='/tmp/7115.1.qmbm.q/molpro_options.7461'<br>        MPI_MAX_CLUSTER_SIZE=''<br> export PROCGRP='/tmp/7115.1.qmbm.q/procgrp.7461'<br> export RT_GRQ='ON'<br>        TCGRSH=''<br> export TMPDIR='/tmp/7115.1.qmbm.q'<br> export XLSMPOPTS='parthds=1'<br>/share/apps/MOLPRO_MPP/parallel.x /share/apps/MOLPRO_MPP/bin/molpro.exe  -v SAMPL5_051.in<br>compute-0-32.local: Connection refused<br>tmp = /home/gkamath/pdir//share/apps/MOLPRO_MPP/bin/molpro.exe.p<br> Creating: host=compute-0-32.local, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=59106<br>  1: interrupt(1)<br><br><br><br><br>However (!!) when we go to the queue via qrsh and submit the molpro command by hand: <br>/share/apps/MOLPRO_MPP/bin/molpro -v -n 4 SAMPL5_051.in<br><br>we are scheduled by SGE as before, placed on an computational node and Molpro runs fine. <br><br></div><div>For eg:<br><br>gkamath@cluster01 sample]$ /share/apps/MOLPRO_MPP/bin/molpro -v -n 4 SAMPL5_051.in<br> # PARALLEL mode<br> nodelist=4<br> first   =4<br> second  =<br> third   =<br> HOSTFILE_FORMAT: $user $hostname 1 $exe $working_dir<br><br>gkamath <a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a> 1 /share/apps/MOLPRO_MPP/bin/molpro.exe /home/gkamath/sample<br>gkamath <a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a> 1 /share/apps/MOLPRO_MPP/bin/molpro.exe /home/gkamath/sample<br>gkamath <a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a> 1 /share/apps/MOLPRO_MPP/bin/molpro.exe /home/gkamath/sample<br>gkamath <a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a> 1 /share/apps/MOLPRO_MPP/bin/molpro.exe /home/gkamath/sample<br><br> export LD_LIBRARY_PATH=':/opt/gridengine/lib/linux-x64:/opt/openmpi/lib:/opt/python/lib'<br> export AIXTHREAD_SCOPE='s'<br> export MOLPRO_PREFIX='/share/apps/MOLPRO_MPP'<br> export MP_NODES='0'<br> export MP_PROCS='4'<br>        MP_TASKS_PER_NODE=''<br> export MOLPRO_NOARG='1'<br> export MOLPRO_OPTIONS=' -v SAMPL5_051.in'<br> export MOLPRO_OPTIONS_FILE='/tmp/molpro_options.29879'<br>        MPI_MAX_CLUSTER_SIZE=''<br> export PROCGRP='/tmp/procgrp.29879'<br> export RT_GRQ='ON'<br>        TCGRSH=''<br>        TMPDIR=''<br> export XLSMPOPTS='parthds=1'<br>/share/apps/MOLPRO_MPP/parallel.x /share/apps/MOLPRO_MPP/bin/molpro.exe  -v SAMPL5_051.in<br>tmp = /home/gkamath/pdir//share/apps/MOLPRO_MPP/bin/molpro.exe.p<br> Creating: host=<a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a>, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=55604<br> Creating: host=<a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a>, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=58287<br> Creating: host=<a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a>, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=42902<br> Creating: host=<a href="http://cluster01.interxinc.com">cluster01.interxinc.com</a>, user=gkamath,<br>           file=/share/apps/MOLPRO_MPP/bin/molpro.exe, port=34881<br> token read from /share/apps/MOLPRO_MPP/lib//.token<br> input from /home/gkamath/sample/SAMPL5_051.in<br> output to /home/gkamath/sample/SAMPL5_051.out<br> XML stream to /home/gkamath/sample/SAMPL5_051.xml<br> Move existing /home/gkamath/sample/SAMPL5_051.xml to <br> /home/gkamath/sample/SAMPL5_051.xml_1<br> Move existing /home/gkamath/sample/SAMPL5_051.out to <br> /home/gkamath/sample/SAMPL5_051.out_1<br><br> f2003 hello world                                                           <br><br></div><div><br>We are using OpenMPI. I am attaching the environment variables env_sge during the full SGE submit, and env_qrsh for the qrsh method. <br><br>our parallel environment is: <br><br>pe_name            orte<br>slots              9999<br>user_lists         NONE<br>xuser_lists        NONE<br>start_proc_args    /bin/true<br>stop_proc_args     /bin/true<br>allocation_rule    $fill_up<br>control_slaves     TRUE<br>job_is_first_task  TRUE<br>urgency_slots      min<br>accounting_summary TRUE<br><br>Additionally, when we submit a simple hello MPI job for these slots everything works exactly as it should, the job gets placed and executed. <br><br>We are a little lost, it would be great if you could help us out. We are using MolPro 2012 (I don't know which). Thank you in advance.<br><br></div>We really appreciate suggestions and help. <br><br></div>Ganesh Kamath<br><br><br>Certain other details:<br> SHA1      : 2c68d29c09da70e1723824271fadde4bcd5f07a0<br> ARCHNAME  : Linux/x86_64<br> FC        : /opt/intel/compilerpro-12.0.2.137/bin/intel64/ifort<br> FCVERSION : 12.0.2<br> BLASLIB   :<br> id        : interx<br><br><br><br><br></div>