Re: [netcdfgroup] Fw: Profiling NetCDF4-python

• To: Nick Papior <nickpapior@xxxxxxxxx>
• Subject: Re: [netcdfgroup] Fw: Profiling NetCDF4-python
• From: Matthew Jones <M.Jones3@xxxxxxxxxxxxxxxxx>
• Date: Mon, 9 May 2016 10:50:30 +0000
• Authentication-results: unidata.ucar.edu; dkim=none (message not signed) header.d=none;unidata.ucar.edu; dmarc=none action=none header.from=pgr.reading.ac.uk;
• Spamdiagnosticmetadata: NSPM
• Spamdiagnosticoutput: 1:23

Hi


Thanks for the reply.


kcachegrind uses cProfile, so not overly useful for what I want to do


Thanks

Matt


________________________________
From: Nick Papior <nickpapior@xxxxxxxxx>
Sent: 09 May 2016 11:46
To: Matthew Jones
Cc: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: Re: [netcdfgroup] Fw: Profiling NetCDF4-python

Have you tried:
https://docs.python.org/2/library/profile.html
26.4. The Python Profilers - Python 2.7.11 
documentation<https://docs.python.org/2/library/profile.html>
docs.python.org
26.4.1. Introduction to the profilers¶ cProfile and profile provide 
deterministic profiling of Python programs. A profile is a set of statistics 
that describes how ...



It usually works quite well for simple profiling.

2016-05-09 12:35 GMT+02:00 Matthew Jones 
<M.Jones3@xxxxxxxxxxxxxxxxx<mailto:M.Jones3@xxxxxxxxxxxxxxxxx>>:

Hi


I am trying to profile my NetCDF4-python code to see how to improve read 
performance, which I have identified as the bottleneck. I have done this using 
kcachegrind and line_profiler.


I want to delve a bit deeper into the library to see where the performance is 
dropping. I can't work out how to use line_profiler with the @profile decorator 
in the variable class. I also can't work out how to get kcachegrind to delve 
into the library because the slicing syntax var[...] is not an explicit 
function call, and I can't get var.__getitem__() to do the same thing.


Does anyone have any ideas how I could get more information from profiling the 
netcdf4 library?


Many thanks

Matt



----------------------------------------
Matthew Jones
PhD Student
Atmosphere, Oceans and Climate
Department of Meteorology,
University of Reading

Room 288, ESSC, Harry Pitt Building,
3 Earley Gate, Reading, RG6 6AL, UK


https://www.linkedin.com/pub/matthew-jones/8b/b81/25a
http://www.met.reading.ac.uk/users/users/1887

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/



--
Kind regards Nick

• References:
  • [netcdfgroup] Fw: Profiling NetCDF4-python
    • From: Matthew Jones
  • Re: [netcdfgroup] Fw: Profiling NetCDF4-python
    • From: Nick Papior