[molpro-user] possible bug: moldpro not releasing deleted scratch files

[Andy May]

Jörg,

When Molpro opens a scratch file it immediately calls unlink() for that 
file. This means the file is scheduled for deletion, but postponed until 
either all processes close the file (i.e. when Molpro's finished with 
them), or when the processes linked to them terminate. So the reason 
these files are not actually deleted until you kill the process is 
likely because they are still being used. My guess is that your 
calculation really does need this amount of disk space.

The reason we use unlink() instead if putting in a call to delete the 
file is to cover the case of the program crashing, or the user killing 
the job. In both of these cases without the call to unlink() the files 
would remain as permanent files on the disk since the file delete code 
would have never been reached. Of course that's no problem on clusters 
when the batch scheduler has been set up to clean scratch, but in most 
other cases the scratch files would otherwise have to be removed manually.

Best wishes,

Andy

On 20/02/13 22:18, Jörg Saßmannshausen wrote:
> Dear all,
>
> I have come across something highly anoying in Molpro: When I am doing a
> rather large LCCSD(T)/cc-pVTZ calculation, the program does not release the
> deleted files and thus is filling up the scratch space:
>
> df /scr/
> Dateisystem           Size  Used Avail Use% Eingehängt auf
> /dev/md0              932G  932G   88K 100% /scr
>
> but:
> du -sh /scr/molpro/
> 35G     /scr/molpro/
>
> The program crashes as the scratch is full but one process is still running.
> When I am then doing a lsof (output attached), I see a number of these files:
>
> molpro.ex  2452       sassy   19u      REG    9,0          73  134219798
> /scr/molpro/init_job5000002452 (deleted)
>
> Once I kill the program in the queue, it is releasing the deleted files and I
> get my space back.
>
> $ df /scr/
> Dateisystem           Size  Used Avail Use% Eingehängt auf
> /dev/md0              932G   39G  893G   5% /scr
>
> This is not what I expect.  I would have thought once the file is deleted it is
> released to the file system can remove it. As that is not happening, the
> scratch of 1 TB is filling up and my calculation crashes :-(
>
> I am using that version of Molpro.
>   SHA1      : 2c68d29c09da70e1723824271fadde4bcd5f07a0
>   ARCHNAME  : Linux/x86_64
>   FC        : /opt/intel/compilerpro-12.0.2.137/bin/intel64/ifort
>   FCVERSION : 12.0.2
>
> Is that a bug or is there something I am doing wrong here. I am using xfs for
> scratch.
>
> All the best from London!
>
> Jörg
>
>
>
> _______________________________________________
> Molpro-user mailing list
> Molpro-user at molpro.net
> http://www.molpro.net/mailman/listinfo/molpro-user
>