Exadata – Cellsrv Memory Usage

Have you ever wondered how much memory is used by Storage Indexes? Or how memory is free from the allocated one? Or how much it is used and what is it actually used for?
Well, now there is a way to check that…

CellCLI> alter cell events="immediate cellsrv.cellsrv_dump('memsummary',0)";
Dump sequence #71 has been written to /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/log/diag/asm/cell/cell01/trace/svtrc_30850_35.trc
Cell cell01 successfully altered

CellCLI>

And the generated file content:

------------ CELLSRV MEMORY USAGE SUMMARY -------------
2013-10-28 21:56:36.917421*: Dump sequence #71:
Max memory for the cellsrv: 6289408 KB
Total OS memory allocated by cellsrv: 5238709 KB
Total permanent OS memory (flash, cache, storidx etc): 4922117 KB
No mem threshold failures: 0
Memory threshold flags: 0

Cellsrv memory usage by type:
Memory type:          sga: 311297 KB
Memory type:          pga: 5295 KB
Memory type:        cache: 3818264 KB
Memory type:      storidx: 3392 KB
Memory type:        flash: 358893 KB
Memory type:  heapsummary: 17600 KB
Memory type:     codetext: 76800 KB
Memory type:       malloc: 32768 KB
Memory type:        stack: 614400 KB
Total: 5238709 KB

-------------------- SGA statistics -------------------
Total OS memory allocated to SGA: 311297 KB
Total memory in SGA: 311296 KB (kgh stat)
Total client allocated memory in SGA: 306400 KB (cellsrv stat)
Total client allocated memory in SGA: 307476 KB (kgh stat)
SGA low memory threshold: 153600 KB
SGA low memory threshold failures: 0
Total memory allocated for dynamic buffers: 0 bytes

--------------- Network memory statistics -------------
Total memory in network heap: 81933 KB (kgh stat)
Total client allocated memory in network heap: 2747 KB (kgh stat)

--------------------- PGA statistics ------------------
Total OS memory allocated to PGA: 5295 KB (cellsrv stat)
Total client allocated memory in PGA: 964 KB (cellsrv stat)

Printing PGA usage per thread:
Thread ID 000 total mem: 8264 bytes, alloc mem: 3400 bytes (kgh stat)
Thread ID 001 total mem: 8264 bytes, alloc mem: 3400 bytes (kgh stat)
Thread ID 002 total mem: 106952 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 003 total mem: 115320 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 004 total mem: 8264 bytes, alloc mem: 3400 bytes (kgh stat)
Thread ID 005 total mem: 8264 bytes, alloc mem: 3400 bytes (kgh stat)
Thread ID 006 total mem: 108104 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 007 total mem: 106952 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 008 total mem: 8264 bytes, alloc mem: 3400 bytes (kgh stat)
.
<output omitted >
.
Thread ID 056 total mem: 91080 bytes, alloc mem: 3400 bytes (kgh stat)
Thread ID 057 total mem: 108104 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 058 total mem: 106952 bytes, alloc mem: 20424 bytes (kgh stat)
Thread ID 059 total mem: 106952 bytes, alloc mem: 20424 bytes (kgh stat)

Total PGA memory for all threads: 5282 KB, Allocated mem: 930 KB (kgh stat)

-------------- Smart IO memory statistics -------------

Dbid: 2201331730 Sqlid: 32qpyw4pyagtj Filter Mem: 650 KB Other Pred Mem: 14 KB Total SmartIO Mem: 665 KB
Dbid: 2201331730 Sqlid: 32qpyw4pyagtj Filter Mem: 672 KB Other Pred Mem: 16 KB Total SmartIO Mem: 689 KB

Number of predicate disks: 2
Filter Mem: 1323 KB, Other Pred Mem: 31 KB, Total SmartIO Mem: 1354 KB

-------------------------------------------------------

From here we can see a nice summary about our SGA and PGA, but of course we retrieve more detailed information.

SGA details:

CellCLI> alter cell events="immediate cellsrv.cellsrv_dump('sgaheapsummary',0)";
HEAP SUMMARY - Sorted by hwmsz (default; change to level 2 for cursz)
bt:               Indicates if all the heap memory alloc occured during cellsrv boot-time
hwmsz:            High-water-mark allocation size over cellsrv
cursz:            Current allocation size
hwm#allo:         High-water-mark number of allocations over cellsrv
cur#allo:         Current number of allocations over cellsrv lifetime
#evictions:       Total number of records that have to be evicted as a result of space exhaustion
#records dropped: Total number of records that have to be dropped as corresponding comment can't be found

Mon Oct 28 22:55:04 2013 printing heap global stats... #evictions: 0 #records dropped:1
SGA heap free - cellsrv statistics: 6.07M
Name                                     : bt      hwmsz      cursz  hwm#allo    cur#allo
SGA HEAP                                 : n     300.25M    297.93M     137856     137727
--------------------------------------------------------------------------------------------------------------
FlashCacheCtx                            : y      97.28M     97.28M       2004       2004
SUBHEAP Networ                           : y      80.02M     80.02M         80         80
RemoteSendPort Fixed Size                : y      33.14M     33.14M       1498       1498
RemoteOpenInfo Fixed Size                : y      21.98M     21.98M     119994     119994
reqHandleBucket::netdir                  : n      10.78M     10.78M          1          1
requesthandles:remotereceiveport         : y       5.41M      5.41M         18         18
hashtable:buckets                        : y       4.12M      4.12M         14         14
groupreids:fm                            : y       4.01M      4.01M          1          1
Thread IO Lat Stats                      : y       3.75M      3.75M        120        120
FASTMETRICS_CHUNK                        : y       3.02M      3.02M          3          3
cache replacementQ_8k                    : y       2.90M      2.90M         26         26
msghdr:remotereceiveport                 : y       1.52M      1.52M         18         18
cache replacementQ_32k                   : y       1.45M      1.45M         26         26
cache replacementQ_1/2k                  : y       1.45M      1.45M         26         26
cache replacementQ_2k                    : y       1.45M      1.45M         26         26
cache replacementQ_4k                    : y       1.45M      1.45M         26         26
cache replacementQ_64k                   : y       1.45M      1.45M         26         26
cache replacementQ_16k                   : y       1.45M      1.45M         26         26
SUBHEAP PFilt                            : n       1.43M          0        106          0
Setplan parent                           : y       1.35M      1.35M          4          4
remotereceiveport:HT                     : y       1.30M      1.30M         18         18
cellserver:oss                           : y       1.19M      1.19M          2          2
ossp_globals_pga                         : y       1.03M      1.03M         60         60
replyPayload                             : n       1.00M          0          1          0
ResilverKickoff 1MB response buf         : y       1.00M      1.00M          1          1
SchedTraceBuf                            : y       1.00M      1.00M          1          1
fetchMetrics:replybuffer                 : n       1.00M          0          1          0
and so on.

Here we can see even our latency stat size (:;

And PGA details /one of the possible ways to trace the threads/:

CellCLI> alter cell events="immediate cellsrv.cellsrv_dump('pgaheap',0)";
Dump sequence #78 has been written to /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/log/diag/asm/cell/cell01/trace/svtrc_30850_39.trc
Cell cell01 successfully altered

CellCLI>

Trace output:

******************************************************

Thread ID: 1

******************************************************
HEAP DUMP heap name="OSSP pga heap"  desc=0x7fa5c1553438
 extent sz=0x20c0 alt=0 het=32767 rec=0 flg=2 opc=2
 parent=(nil) owner=(nil) nex=(nil) xsz=0x2058 heap=(nil)
 fl2=0x20, nex=(nil), dsxvers=1, dsxflg=0x0
 dsx first ext=0x3471198
EXTENT 0 addr=0x3471198
  Chunk        0034711a8 sz=       80    perm      "perm           "  alo=80
  Chunk        0034711f8 sz=     8184    perm      "perm           "  alo=3400
Total heap size    =     8264
FREE LISTS:
 Bucket 0 size=56
  Chunk        0034711c8 sz=        0    kghdsx
 Bucket 1 size=88
 Bucket 2 size=152
 Bucket 3 size=168
 Bucket 4 size=280
 Bucket 5 size=432
 Bucket 6 size=536
 Bucket 7 size=1048
 Bucket 8 size=2072
 Bucket 9 size=4120
 Bucket 10 size=8216
 Bucket 11 size=16408
 Bucket 12 size=32792
 Bucket 13 size=65560
 Bucket 14 size=131096
 Bucket 15 size=262168
 Bucket 16 size=524312
 Bucket 17 size=2097176
Total free space   =        0
UNPINNED RECREATABLE CHUNKS (lru first):
PERMANENT CHUNKS:
  Chunk        0034711f8 sz=     8184    perm      "perm           "  alo=3400
  Chunk        0034711a8 sz=       80    perm      "perm           "  alo=80
Permanent space    =     8264
and so on.

Cheers,
Kovachev

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s