Mutator transition states should not be counted as part of mutator time

[k-sareen]

Recently I was delving deep into a weird result for instructions retired when using stress factor wherein the mutator instructions was increasing proportional to the number of GCs due to stress factor.

Turns out there are two reasons for this:

1. All code from requesting a GC to block_for_gc until the ScheduleCollection work packet is executed is considered to be part of mutator time (or time.other)
2. GC time is considered to be just before we call resume_mutators, so the work undertaken to wake up and resume mutators is considered to be part of the mutator time.

This means that the instructions retired for the actual mutator work were not actually changing, but it was counting the extra work you need to do to facilitate the triggering of a GC. @steveblackburn and I think there needs to be a third counter for these "transition" states since they are neither real mutator work not real GC/STW work.

[steveblackburn]

Right.

• Kunal's experience makes it obvious that transition states should not be simply attributed to mutator or GC (so attribution to a third state, 'transition' may make sense)
• A more fine-grained attribution may be appropriate during transition (ie count threads separately according to their state)
• Understanding transitions (and their costs) is important to GC performance more broadly, so exposing this information should be useful.

[qinsoon]

In the 'transition' state some threads are actually doing mutator work. Adding a counter for transition can be helpful, but you may still see corner cases (e.g. if you GC frequently, and some mutators can do a lot of work during the transition state if they stop late and wake up early, then you would see mutator time decreases for no reason).

1. All code from requesting a GC to block_for_gc until the ScheduleCollection work packet is executed is considered to be part of mutator time (or time.other)

Some threads are still doing mutator work before they are stopped. Only after all the threads are stopped, we know that mutator phase is really done.

3. GC time is considered to be just before we call resume_mutators, so the work undertaken to wake up and resume mutators is considered to be part of the mutator time.

Some threads that get waken up early will start mutator work.

[qinsoon]

My point is we are counting global time, rather than time spent in each thread. So there will be a period of time where some threads are doing mutator work, and some are not. Currently that period is counted as part of mutator time, and the proposal is to count those as part of transition. It is better as we will know what are pure mutator time, and what are mixed transition time. But some mutator work is actually done during transition time. That is inevitable.