NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Chris, > I think there is a key different use-case. I tlooks like spack was > designed with super computers in mind, which means, as you say, a very > particular configuration, so collections of binary packages aren't very > useful. > > conda, on the other hand ,was designed with "standard" systems in mind -- > pretty easy on Windows and OS-X, and they have taken a least common > denominator approach that works pretty well for Linux. But yeah, the > repositories of binaries are only useful because of this standardization. > This is not true. Supercomputers are just big shared Linux systems. Simultaneously installing multiple versions of the same thing (without a new repo for each one) is more obviously necessary there than on PCs. But it's still important on PCs as well. People are increasingly finding Environment Modules to be a useful tool for PCs. Spack is useful on any computer where you need control of your configuration and need more than one version of something installed --- or where the version YOU need installed is different from the version the version the autobuilder authors need installed. This is almost always the case when you're dealing with significant software stacks. Maybe you need to use a little-known library that's incompatible with the latest version of NetCDF. Going through a set of 50 Conda packages and fixing the version numbers and MD5 codes to get your particular software stack working is quite tedious. Easybuild works like Conda --- one version, one package --- and the result has been an unmanageable proliferation of recipes for each package, many of them differing only slightly. Before I tried Spack, I tried EasyBuild, it was a nightmare. I thought of abuot-rewriting EasyBuild recipes too. To fix EasyBuild, you'd want a system that produces a specific repo of recipes from a repo of multi-version "proto-recipes." That is, you will want a system that contains Spack's version-wrangling stuff. Now that Spack manages my versions, I'd hate to do it by hand again. In a Conda repo, how do you even know that you haven't specified incompatible versions of some library deep down? (Eg: Package A requires NetCDF 1 and Package B requires NetCDF 2). Spack roots out and reports/fixes these kinds of problems. Before using Spack, I did look at Conda, at least as described by Continuum Analytics. Could never find description of how to get it to do the things you say it can do; for example, build everything from source. The first step in the Conda docs was to download a binary Python, which would not match my build. The reality of software is that even if it "can" do something, it only CAN do that if users can find documentation describing how. > I'm going to extend that -- building python extensions with conda is VERY > doable -- it really helps make it much easier (than not using conda or > spack...). And you can build conda packages that you can then distribute to > users that may not have the "chops" to do the building themselves. > Typing "spack install" or "conda install" does not require many chops. Doesn't matter whether under the hood it's installing binaries or building from source. I have Spack-based builds that install smoothly without modification on a moderate variety of Linux systems (i.e. those I've tested on). > IIUC, the big difference, and use case for spack is that it makes all this > doable on a hihgly specialized system. > I'm running CentOS 7, is that highly specialized? No... the use case for Spack is if any of the following are true: 1. You need simultaneously more than one version of stuff, without having to rebuild entire separate software repos. 2. You want a system that will be flexible in terms of which versions of stuff you use, allowing you to selectively bypass problematic versions of particular packages for your build. You want the system to easily update to new versions as they come out. 3. You don't want to manage and check version compatibility in your software DAG by hand. 4. You want a portable, trouble-free way to build your software stack on a wide variety of other systems, allowing others to install your stuff with less than the 2 weeks of build time required manually. 5. You want the auto-builder to help find dependencies and set up builds for the projects YOU create, as well as for standard projects that come in the repo. 6. You want to distribute your software in a way that doesn't assume it will be the top level; that allows others to build on top of it if they like. 7. You're a supercomputer admin, and you want to get away from hand-building packages and hand-editing module files every time someone requests a new package be installed. -- Elizabeth
netcdfgroup archives: