NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

Re: [netcdfgroup] nccopy and redirect - based authorization

  • To: Antonio S. Cofiño <cofinoa@xxxxxxxxx>
  • Subject: Re: [netcdfgroup] nccopy and redirect - based authorization
  • From: "ashwin .D" <winash12@xxxxxxxxx>
  • Date: Fri, 25 Aug 2017 09:29:35 +0530
Dear Antonio,
                      Thank you so much for the effort in doing this. I am
extremely grateful given that they are so few who are actually using this
and you are somebody who has always encouraged me to use nccopy for over a
year now ! I am using netcdf-c-4.5-rc1 i.e. the latest netCDF version. I
added this line to my .dodsrc file -

HTTP.CREDENTIALS.USERPASSWORD=winash12@xxxxxxxxx:aaaaa

and I still get the same error . Yes I totally agree about the instructions
on the UCAR site. I have an open ticket with the support netcdf and I just
hope they are able to figure this one out.

I am enclosing the logs turned on with HTTP.VERBOSE. As you can see the

* Server auth using Basic with user 'k,ŷ~'

it should say winash12@xxxxxxxxx. So it is still not picking the username
from .netrc or the .dodsrc file.

Best wishes,
Ashwin.

On Thu, Aug 24, 2017 at 5:46 PM, Antonio S. Cofiño <cofinoa@xxxxxxxxx>
wrote:

> Dear Ashwin,
>
> I have tested and it's working depending on the netcdf library version
> you are using.
>
> With the version 4.1.3 (Ubuntu 14.04 repository package) the only way it's
> embedding the credentials in the URI:
>
> $ ncdump -h "https://YourUsernameUriEncoded:YourPassword@xxxxxxxxxxxx/
> thredds/dodsC/files/g/ds083.2/grib1/2000/2000.02/fnl_20000201_06_00.grib1"
>
>
> For netcdf library version 4.3.3.1, compiled from source, the embedded
> credentials on URI it's working.
>
> If you create a .dodsrc on your working directory or $HOME with your
> credentials
>
> $ cat .dodsrc
>
> HTTP.CREDENTIALS.USERPASSWORD=YourUsername:YourPassword
>
> $ ncdump -h "https://rda.ucar.edu/thredds/dodsC/files/g/ds083.2/grib1/
> 2000/2000.02/fnl_20000201_06_00.grib1"
> <https://rda.ucar.edu/thredds/dodsC/files/g/ds083.2/grib1/2000/2000.02/fnl_20000201_06_00.grib1>
>
> and refering the .netrc file from your .dodsrc, both can be at your
> working or $HOME directory
>
> $ cat .netrc
>
> machine rda.ucar.edu login YourUsername password YourPassword
>
> $ cat .dodsrc
>
> HTTP.NETRC=.netrc
> $ ncdump -h "https://rda.ucar.edu/thredds/dodsC/files/g/ds083.2/grib1/
> 2000/2000.02/fnl_20000201_06_00.grib1"
> <https://rda.ucar.edu/thredds/dodsC/files/g/ds083.2/grib1/2000/2000.02/fnl_20000201_06_00.grib1>
>
> I hope this helps.
>
> Antonio
>
> P.S.: The instructions on the UCAR blog about configuring authentication
> are worthless with respect to the cookie parameteres. This is because UCAR
> authentication service it's not using them.
>
> On 22/08/17 19:41, ashwin .D wrote:
>
> >Dear Ashwin,
>
> >Attached you can find a working example for the ESGF opendap endpoints.
>
> >You can put it on your working directory (the HOME also it's valid)
>
>
> >If the opendap endpoint it's not using SSL security you can ignore the 
> >HTTP.SSL.*
> entries.
>
> >If you are having troubles you can enable the HTTP.VERBOSE
>
> >Regards
>
> >Antonio
>
> Dear Antonio,
>
>              Many thanks for giving me your .dodsrc file. I added a couple of 
> the lines from your .dodsrc file and tried it out and
>
> and I still get the same error. Have you tried downloading from the UCAR site 
> recently ?
>
> Best regards,
>
> Ashwin.
>
>
> On Mon, Aug 21, 2017 at 8:12 AM, ashwin .D <winash12@xxxxxxxxx> wrote:
>
>> Hello,
>>          Has anybody been able to use nccopy with redirect-based
>> authorization and download a netCDF file ? I am referring to this -
>> http://ncarrda.blogspot.in/2017/07/accessing-rda-opendap-end
>> points-with.html and I get a authorization failed error even though I
>> have an appropriate .netrc file as well as dodsrc file. Using standard
>> python and netCDF python I am able to read http request and response of
>> metadata pages on RDA - it is only nccopy that is failing. Does anybody
>> have a working example of nccopy with this redirect-based authorization ?
>>
>> Best,
>> Ashwin.
>>
>
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing listnetcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/
>
>
>

Attachment: logs
Description: Binary data

#!/usr/bin/python3.5

import shlex

import subprocess

import re

from datetime import date,datetime,timedelta

from urllib.request import urlopen

import math

url = 
"https://rda.ucar.edu/thredds/dodsC/aggregations/g/ds083.2/1/TP.grb2?lat[0:1:180],lon[0:1:359],time1[2459:1:2459],isobaric3[0:1:30],Temperature_isobaric[2459:1:2459][0:1:30][0:1:180][0:1:359]";


outputFileName = "bbb.nc"

command_list = shlex.split('nccopy -k 4 '+url+' '+ outputFileName)

subprocess.check_call(command_list)