Re: [thredds] How to speed up gbx9 file generation

• To: 唐健 <tangjian@xxxxxxxxxx>
• Subject: Re: [thredds] How to speed up gbx9 file generation
• From: Sean Arms <sarms@xxxxxxxx>
• Date: Wed, 6 Jul 2016 15:27:37 -0600

Greetings Jian,

The performance of the TDM will be limited to the performance of your I/O
system. While running more than one TDM against a single TDS won't work due
to the way the TDM finds collections that need scanned, you can run a
single TDM in a multithreaded mode. In your TDM run script, you can set the
number of threads to use by setting the flag

-nthreads=<number of threads>

By default, the TDM runs with one thread.

Cheers,

Sean


On Wed, Jul 6, 2016 at 12:40 AM, 唐健 <tangjian@xxxxxxxxxx> wrote:

> HI everyone
>
> We run our thredds server and TDM on a virtual workstation with 2.0GHz
> E78850*2, and 32GB memory, windows server 2008.
> The script run TDM is like this
> "C:/Program Files/Java/jdk1.7.0_55/bin/java" -Xmx10g
> -Dtds.content.root.path="C:\Program Files\Apache Software Foundation\Tomcat
> 7.0\content" -jar tdm-4.5.jar -tds "http://thredds.cma.gov.cn/"; -cred ..
> In operation it only take like 0.5GB memory.
>
> My problem is almost all global model and meso-scale model output comes in
> at the most busy time around 3-4 pm local time.
> and there are a lot of files. it takes quite long time for tdm to build
> indexes.
> for example, when one model output is all there, tdm is busy building
> indexes for another model, and it cannot generate gbx9 and ncx2 file for
> this model.
> Is there anything I can do about it?
> Maybe run multi-TDM?
> or put one TDM in charge of every single model output?
>
> or can I use any other program to scan dirs and generate gbx9 for each
> file once it comes in ?
>
> If anyone has any suggestion, please let me know.
> Many thanks
>
>
> Cheers
> Jian
>
>
> --
> Dr. Jian TANG
> Central Meteorological Office
> National Meteorological Center of CMA
>
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>

• Follow-Ups:
  • Re: [thredds] How to speed up gbx9 file generation
    • From: 唐健

• References:
  • [thredds] How to speed up gbx9 file generation
    • From: 唐健