NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [ldm-users] high memory

Jeff,

On Mon, 25 Oct 2010, Jeff Lake - Admin wrote:


Art ..
-sh-3.2$ free
           total       used       free     shared    buffers     cached
Mem:      12299204   12071528     227676          0     423980   10159520
-/+ buffers/cache:    1488028   10811176
Swap:      2096472        184    2096288
its the 12299204 12071528 that scares me makes it look like I have zero room left for anything else
It looks scary, but it's not. As long as you are having no performance issues, this is normally what to expect. It's the 10811176 that's important... that says there's roughly 10G of memory available for use by other programs should demand increase on the system. In that case, the size of the buffers and I/O cache (the 423980 10159520) would automatically reduce to create some actual free memory for the additional demands on the system. At the same time, you would see the second line in the free column (10811176) become smaller. Have you observed any performance issues (e.g. disk thrashing) or is it just the memory usage that caught your eye? 

                                Art


-Jeff
Caching the data going to disk actually improves performance because I/O can be scheduled in larger chunks and be ordered more efficiently to disk. Also, the cached data is immediately available for processes to use without going back to the disk to get it. Linux uses most of the memory it doesn't need for other running processes to cache its disk I/O so you'll rarely see free memory available on a busy system. The amount of cache being used can be seen using "free", e.g.: 

$ free
             total       used       free     shared    buffers     cached
Mem:       8182268    8135656      46612          0       6888    6748420
-/+ buffers/cache:    1380348    6801920
Swap:      8385920     140580    8245340
The important number to look at is the second line of the "free" column... this is the actual free memory plus memory being used by buffers/cache (46612+6888+6748420=6801920), most of which could be made available if needed by the system to stuff more processes into the system. So, you could consider this to be effectively free memory... what does your system show for the second line of the free column? 

                              Art


Gilbert,
what does BillyBob know about a real operating system ;}
system loads are below or just flirting with 1, only time the loads bounce over 1 is when scour runs 


-Jeff



On Sun, 24 Oct 2010, Steve Emmerson wrote:


Jeff,

How do you know the LDM is using the memory?

On a Linux system, the operating-system will consume as much memory as
it can without interfering with user processes.


For the first few years of running Linux, I had wondered the same thing.
Look at your load average, for starters. If you're below 1, chances are it's all good. 12 GB of RAM is fine, and you'll never use that much. Bill Gates told me so! ;-)
I have found that running 1.2 GB queue out of /dev/shm for my LDM, even with level 2 data coming in, works fine. It is on my NOAAport ingest server, whose load average most of the time is flat out zero. It does jump up to about .50 when NOAAPort is closer to it's current 10 mb/sec max. 

*******************************************************************************
Gilbert Sebenste ********
(My opinions only!) ****** Staff Meteorologist, Northern Illinois University **** E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx *** 
web: http://weather.admin.niu.edu                                      **
*******************************************************************************

_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 

_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 

Arthur A. Person
Research Assistant, System Administrator
Penn State Department of Meteorology
email:  person@xxxxxxxxxxxxx, phone:  814-863-1563





Arthur A. Person
Research Assistant, System Administrator
Penn State Department of Meteorology
email:  person@xxxxxxxxxxxxx, phone:  814-863-1563
  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: