[ldm-users] 20230703: Re: How to control LDM batching RPC requests every 30 seconds?

• To: ldm-users@xxxxxxxxxxxxxxxx
• Subject: [ldm-users] 20230703: Re: How to control LDM batching RPC requests every 30 seconds?
• From: Tom Yoksas <yoksas@xxxxxxxx>
• Date: Mon, 3 Jul 2023 09:28:47 -0600

Hi Daryl,

On 7/3/23 09:05, Herzmann, Daryl E [AGRON] wrote:

I'm attempting to get straight in my head some relay latency between a LDM 
server and client.  Both are running the current release, 6.14.5.  My simple 
test is as follows, on the LDM upstream

     pqinsert something every two seconds

on the LDM downstream, monitor receipt of these inserted products via `ldmadmin 
watch` and also putting the LDM client connection into debug mode to see what 
the RPC is doing.  What I observe appears to LDM batching the RPC calls every 
30 seconds instead of a more timely relay of inserted products to downstreams.

Your observation is mostly correct in the following sense:

If 'pqinsert' processes are run outside of the LDM process group, then
the LDM routines that send product to downstreams will not be informed
that there is a new product to send, but they will check every 30
seconts..  The default action for an upstreamLDM is to send products
when notified that there are products to send, or wait 30 seconds and
then check to see if there are new products to send.  The net effect of
this for 'pqinsert' processes run outside of the LDM process group is
latencies that will range from 0 to 30 seconds with an average around 15
seconds.

There are two solutions for the "30 second delay":

- run the 'pqinsert' instances a part of the LDM process group

  I.e., run them from an EXEC entry in the LDM configuration file.

- have the insertion process (e.g., BASH, Python, etc. script) send a
  CONT signal to the _negative_ of the process ID of the lead LDM
  server running on the machine (the process ID for the lead server is
  stored in the ~ldm/ldmd.pid file)

re:

My typical naive and likely wrong understanding of LDM's code is that this is 
controlled by:

src/registry/globals.c
/*
  * Time to sleep in pq_suspend() and before retrying connects.
  */
unsigned int            interval = 30;

So my questions:

1) Is this latency a function of my usage of `pqinsert` on the server side 
instead of some proper LDM forked child process?

2) Are there side effects / significant performance concerns of setting the 
`interval` value above to some lower threshold?  I am thinking about setting it 
to 1 or 5 seconds.  Do I need to be careful to change this on all my LDM 
servers and clients in my local topology?

3) Is there some other setting that controls this batching of RPC from the 
server to the client?

Please let us know if the explanation above does not answer this
question.

Cheers,

Tom
--
+----------------------------------------------------------------------+
* Tom Yoksas                                      UCAR Unidata Program *
* (303) 497-8642 (last resort)                           P.O. Box 3000 *
* yoksas@xxxxxxxx                                    Boulder, CO 80307 *
* Unidata WWW Service                     http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+

• Follow-Ups:
  • Re: [ldm-users] 20230703: Re: How to control LDM batching RPC requests every 30 seconds?
    • From: Jared P Bostic

• References:
  • [ldm-users] How to control LDM batching RPC requests every 30 seconds?
    • From: Herzmann, Daryl E [AGRON]