NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

Re: [netcdfgroup] NetCDF on Databricks

  • To: Shannon Sheppard <dbashep@xxxxxxxxx>
  • Subject: Re: [netcdfgroup] NetCDF on Databricks
  • From: "H. Joe Lee" <hyoklee@xxxxxxxxxxxx>
  • Date: Fri, 23 Sep 2022 10:07:11 -0500
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=hdfgroup.org;
Hi, Shannon!

I hope the following example can help you if you haven't seen it already:

https://hdfeos.org/examples/spark.php

Also, please check the following URL if you deal with netCDF-4/HDF5:

https://github.com/HDFGroup/hdf5-spark-connector

- Joe
---
Build the Metaverse w/ HDF.



On Thu, Sep 22, 2022 at 12:54 PM Shannon Sheppard via netcdfgroup <
netcdfgroup@xxxxxxxxxxxxxxxx> wrote:

> Hello Gang,
>
> Is it possible to run NetCDF on Databricks or Spark?  If there is it does
> not appear to be obvious.  There does not appear to be a package for
> processing these fils on Spark, at least not Spark 3.0+
>
> Shep
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> https://www.unidata.ucar.edu/mailing_lists/
>