NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [ldm-users] 20200423: Re: Efficiency of splitting pqacts

  • To: "Herzmann, Daryl E [AGRON]" <akrherz@xxxxxxxxxxx>
  • Subject: Re: [ldm-users] 20200423: Re: Efficiency of splitting pqacts
  • From: Steven Emmerson <emmerson@xxxxxxxx>
  • Date: Thu, 23 Apr 2020 17:51:43 -0600
Hi Daryl,

On Thu, Apr 23, 2020 at 1:37 PM Herzmann, Daryl E [AGRON] <
akrherz@xxxxxxxxxxx> wrote:

> I am sure Unidata will correct my ignorance / incorrect details, but my
> understanding is that an individual pqact process can only do 32 "things"
> at one time, or there's 32 slots available for work.
>

The "things" are file descriptors and the Unix standard only *guarantees* 32
of them. Modern O/S provide much more.


> Now, the above depends on the action.  If you run `PIPE -close`, the slot


The "slot" is a file descriptor.


> can be used for another product even with the PIPEd process still
> running...


The PIPE-d process might still be running, but the file descriptor will
have been closed -- so, yes, that descriptor can be reused.


>   This type of action can lead LDM to DOSing the server it is on as it
> will fire off as many PIPE'd processes that it can.


If the decoders take a long time to process the data-product, then, yes,
you can wind up with many of them. This is not a problem with pqact(1) *per
se*, but with inefficient decoders (pure Python, for example, is about
three orders of magnitude slower than C).

Regards,
Steve Emmerson
  • 2020 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: