Re: [thredds] Add non time dependent variable to FMRC

• To: "Antonio S. Cofiño" <cofinoa@xxxxxxxxx>
• Subject: Re: [thredds] Add non time dependent variable to FMRC
• From: marceloandrioni@xxxxxxxxxxxxxxxx
• Date: Tue, 3 Apr 2018 15:07:36 -0300
• Sensitivity:

Hello Antonio, I did as you suggested and using
<protoDataset choice="First"/>
inside the featureCollection and appending the non time dependent variables 
(ncks -A) to the first file in the collection produced the desired result. Now 
I can see all the non time dependent variables inside the "Best" collection.

Thank you very much.


-----"Antonio S. Cofiño" <cofinoa@xxxxxxxxx> escreveu: -----
Para: marceloandrioni@xxxxxxxxxxxxxxxx, thredds@xxxxxxxxxxxxxxxx
De: "Antonio S. Cofiño" <cofinoa@xxxxxxxxx>
Data: 03/04/2018 11:37 AM
Assunto: Re: [thredds] Add non time dependent variable to FMRC

                   Hi, 
       
     
     
On 03/04/18 14:51, marceloandrioni@xxxxxxxxxxxxxxxx         wrote:
                   
Hello, I have a FMRC aggregation of several yearly files with the usual 
variables (ssh,t,s,u,v), e.g.

<featureCollection name="mercator" featureType="FMRC" harvest="true" 
path="mercator">
  <collection spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/>
  <update startup="true"/>
  <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
  </netcdf>
  <fmrcConfig regularize="false" datasetTypes="Best Files" />
</featureCollection>

I have a different file with non time dependent variables (bathymetry, sea land 
mask, etc) and would like to have these variables also appear in the FMRC. I 
tried just renaming the file to mercator_1970.nc hoping that the <collection 
spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/> would catch it, 
but it didn't work. My guess is that doesn't work because there is no time 
dimension in this file and time is necessary for FMRC aggregation.
          No. It's because the prototype dataset been chosen by the       
featureCollection doesn't contain those variables. By default the       
featureCollection choose the penultimate dataset/file to build-up       the 
resulting dataset with all dataset/files in the collection.
                 
Besides appending the non time dependent variables to every single file with
ncks -A bathymetry_mask.nc mercator_YYYY.nc
is there a way using ncml to join all the files (with and without time 
dimensions) in the FMRC?          You can change the default prototype dataset 
by: 
          
<featureCollection name="mercator" featureType="FMRC" harvest="true" 
path="mercator">
  <collection spec="/home/opendap/datasets/mercator/**/mercator_#yyyy#\.nc"/>
  <update startup="true"/>
  
  <protoDataset choice="First"/>
  <fmrcConfig regularize="false" datasetTypes="Best Files" />
</featureCollection>

and append the missing variables to the first dataset/file in the collection, 
i.e.
ncks -A bathymetry_mask.nc mercator_1970.nc

Then they should be "appear" in the resulting dataset. From the doc [1] you can 
read
....
The choice of the protoDataset matters when the datasets are not homogenous:

    Global and variable attributes are taken from the prototype dataset.
    If a variable appears in the prototype dataset, it will appear in the 
feature collection dataset. If it doesnt appear in other datasets, it will have 
missing data for those times.
    If a variable does not appears in the prototype dataset, it will not appear 
in the feature collection dataset, even if it appears in other datasets.
....


            
Also, I am adamant in using FMRC because my tests showed that is a lot faster 
to retrieve data than using aggregation joinNew or joinExisting.          In my 
case I'm reluctant to use FMRC because you have less       control on building 
the resulting dataset. With pure ncml and       aggregations you have better 
control on resulting dataset and good       performance if you control cache 
and definition of the coordinate       values. 
     
     Regards
     
     Antonio
     
     [1]       
https://www.unidata.ucar.edu/software/thredds/v4.6/tds/reference/collections/FeatureCollections.html#elements
       
     
     
     
--
Antonio S. Cofiño
Dep. de Matemática Aplicada y 
        Ciencias de la Computación
Universidad de Cantabria
http://www.meteo.unican.es           
 
"O emitente desta mensagem é responsável por seu conteúdo e endereçamento. Cabe 
ao destinatário cuidar quanto ao tratamento adequado. Sem a devida autorização, 
a divulgação, a reprodução, a distribuição ou qualquer outra ação em 
desconformidade com as normas internas do Sistema Petrobras são proibidas e 
passíveis de sanção disciplinar, cível e criminal."
 
"The sender of this message is responsible for its content and addressing. The 
receiver shall take proper care of it. Without due authorization, the 
publication, reproduction, distribution or the performance of  any other action 
not conforming to Petrobras System internal policies and procedures is 
forbidden and liable to disciplinary, civil or criminal sanctions."
 
"El emisor de este mensaje es responsable por su contenido y direccionamiento. 
Cabe al destinatario darle el tratamiento adecuado. Sin la debida autorización, 
su divulgación, reproducción, distribución o cualquier otra acción no conforme 
a las normas internas del Sistema Petrobras están prohibidas y serán pasibles 
de sanción disciplinaria, civil y penal."

• References:
  • Re: [thredds] Add non time dependent variable to FMRC
    • From: Antonio S . Cofiño
  • [thredds] Add non time dependent variable to FMRC
    • From: marceloandrioni