GC presentation - Maoni Stephens - PDC 2005

1. CLR: Garbage Collection Inside Out
Maoni Stephens
FUN421
Software Design Engineer
Microsoft Corporation

2. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

3. Generation 1 Generation 0
New Heap Begins with New Generation
Accessible References Keep Objects Alive
Compacts Referenced Objects
Objects Promoted to Older Generation
New Allocations Rebuild New Generation
Basics

4. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

5. Generational GC
Three generations
Most objects die in gen0
Gen1 holds in the in flight data
Gen2 holds the long lived data
Large object heap
Gen0 and Gen1 – ephemeral generations

6. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

7. GC Heap Segments
Unit of VirtualAlloc
When CLR is loaded, initial segments are
allocated
Additional segments reserved as needed
Committed/Decommitted as needed in the
segment
Deleted when not in use
VM hoarding feature in next CLR 1.1 SP
and CLR 2.0 –
STARTUP_HOARD_GC_VM startup flag

8. Generations And Segments
Start from the ephemeral segment
Ephemeral segment becomes a gen2
segment
Newly reserved segment becomes
ephemeral
Many gen2 segments, only one ephemeral

9. Before GC #1
Gen1 Gen0
Before GC #500
Gen2
Gen2
Gen2 Gen1 Gen0
Gen0
Before GC #0
Before GC #2
Gen2 Gen1 Gen0
Before GC #100
Gen2
Gen2 Gen1 Gen0

10. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

11. Allocation
Cheap lock on UP; lock free on MP
Moving a pointer forward
Clearing the memory for new objects
Register for finalization if applicable
Objects allocated together are
close together

12. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

13. Collection
When
Allocation. Not enough space in
gen0.
Induced GC. By calling
System.GC.Collect().
System pressure.

14. Collection
How
Suspend managed threads
GC
Resume managed threads
Two phases of GC
Mark
Compact

15. Roots
Stack
Handle table
Statics
Older generation(s)
Finalizer queue

16. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

17. Large Object Heap
For objects that are 85,000 bytes or larger
Compacting large objects costs a lot
So we only sweep (freelist)
Many LOH segments
Segments are handled similarly to small
object heap segments
Collection happens during gen2 GCs

18. Collection – Cost
GC takes time – “% time in GC” counter
If objects die in gen0 (survival rate is 0) it’s
the ideal situation
The longer the object lives before being
dead, the worse (with exceptions)
Gen0 and gen1 GCs should both be
relatively cheap; gen2 GCs could cost a lot
LOH – different cost model
Temporary large objects could be bad
Should reuse if possible

19. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

20. Concurrent GC
Why do we have concurrent GC
For interactive applications
How it’s done
Trade some CPU and memory for shorter
pause time
Only for gen2
Done on a concurrent GC thread
How do you get concurrent GC
On by default
Can be turned off via hosting or config

21. Server GC
Why do we have server GC – for server apps
that
Have a fairly consistent number of requests
Require high scalibility and high throughput
How it’s done
One thread for each CPU, running at highest priority
One ephemeral segment per CPU
How do you get server GC
Only on via config or hosting
Hosts like ASP.NET and SQLCLR use server GC
In CLR 1.1 it’s in mscorsvr.dll; in CLR 2.0 it’s in
mscorwks.dll

22. WKS GC SVR GC
Where it runs On the user thread
that triggered GC
On the GC threads
running at highest
priority
On a multi proc
machine
One small object
heap + One LOH
N small object
heaps + N LOHs
On a uni proc
machine
WKS GC +
concurrent GC (if
not turned off)
WKS GC +
concurrent GC
OFF

23. Agenda
Basics
Generations
Segments
Allocation
Collection
Large Object Heap
Different Flavors of GC
Pinning
Tools and Resources

24. Pinning
Why do we need pinning
Interop with unmanaged code
How objects get pinned
Use of GCHandle of type
GCHandleType.Pinned
Allowed by language syntax, eg. fixed in c#
Args get pinned by the interop frame
Fragmentation problem

25. After N more GCs
Gen1 P P
After GC X
Gen2 P P
Gen1 start
Before GC X
Gen1
Gen0 start
P P
Gen0 start
Gen2 P P
Fragmentation Problem Caused By Pinning
Gen0 start
Gen1 start

26. Without demotion
Gen0 start
Gen2
Gen1 start
P P
With demotion
Gen0 start
Gen2
Gen1 start
P P

27. Before GC After GC
Gen2
Seg0
Seg1
Gen2
Seg2
Gen2
Seg3
Gen2
Gen1
Eph P Gen0 P
Without Segment Reuse
Gen2
Seg0
Seg1
Gen2
Seg2
Gen2
Seg3
Gen2
Old Eph Gen2 P P
Gen1
New Eph

28. Before GC After GC
Gen2
Seg0
Seg1
Gen2
Seg2
Gen2
Seg3
Gen2
Gen1
Eph P Gen0 P
With Segment Reuse
Gen2
Seg0
Seg1
Gen2
Seg2
Gen2
Gen2 P P
Old Eph
Old Seg3,
New Eph
Gen2 Gen1

29. What The User Code Can Do
To Help
Best patterns
Pinning for a short time is cheap.
Create pinned buffers at the beginning that will stay in
regions that are very compacted
Create pinned buffers that stay together instead of
scattered around
Techniques
Allocate a pinned big buffer (byte array on the LOH),
give out a chunk at a time
Allocate a pool of small buffers and hand one out
when needed

30. void M(byte[] b)
{
// if the buffer is already in gen2 it’s unlikely to move.
if(GC.GetGeneration(b) ==
GC.MaxGeneration ||
// avoid copying if the buffer is too big
b.Length > 8 * 1024)
{
RealM(b);
}
// GetBuffer will allocate one if none is free.
byte[] TempBuffer = Pool.GetBuffer();
RealM(TempBuffer);
CopyBackToUserBuffer(TempBuffer, b);
Pool.Release(TempBuffer);
}

31. Managed Caches
Design patterns
Don’t do it in the finalizer
Use combination of weak and strong references –
could convert strong to weak after certain period
Cache tuning
Hit rate
Cost to add an item
Cost to find an item
Frequency of cleanup
If not implemented carefully could easily cause more
gen2 collections than needed

32. // sweep the cache for dead objects and compacts the list.
static void sweep_WeakRef()
{
// don't bother if no new collection happened since
// we swept last
if (GC.CollectionCount(0) != gen0_count)
gen0_count = GC.CollectionCount (0);
else
return;
lock (list) {
for (int i = 0; i < WeakRef_fill;) {
if (list[i].Target == null) {
// install the last element in the free slot.
WeakRef_fill--;
if (i != WeakRef_fill) {
list[i] = list[WeakRef_fill];
list[WeakRef_fill] = null;
}
}
else
i++;
}
}
}

33. Why We Don’t Allow Many
Customized Settings
Only allow very few settings specified via
hosting API or config
Advantages for GC to do the tuning for you
GC has intimate knowledge of memory when
applications run
GC is tested on many platforms + many
configurations

34. Looking Ahead
Challenges on 64-bit
More common as people start using 64-bit
machines
VM is practically unlimited so physical
memory is the limit
Servers like the SQL Server tend to not
allow paging
Gen2 could take a long time

35. Tools And Resources
http://blogs.msdn.com/maoni/
CLRProfiler
.NET CLR Memory performance counters
The SoS Debugger Extension for
windbg/cdb
vadump
Task Manager

36. © 2005 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.