TB: Track permissions on the byte-level

[yoctocell]

This makes the tracking of permissions on the byte-level, like in Stacked Borrows, which makes the tracking of interior mutable data more fine-grained.

cc @RalfJung @JoJoDeveloping

[JoJoDeveloping]

So, how does adding a new node work (and why do we need to call visit_freeze_sensitive three times:

• First, you do a read access at the parent node
• Then, you add the new node. Note that adding a new node is a per-block operation, not per-access
• Then you set the correct permissions for the new node
• Then you reset the SIFA thingy (also per-block)
• Then you call into the data race model

For Stacked Borrows, there are no per-block operations: each stack at each offset its own stack and completely independent from the other ones at different offsets in the same allocation. So there, more (basically everything) can be moved into the per-offset function.

So what do we do for Tree Borrows? One answer is to combine parent reading, permission setting and data race notifying together. This would be somewhat ugly, because you're constructing a node, while also accessing the tree, and this would only be fine because you're careful to not actually add the node into the children array of its parent so that it's not visited when walking the tree.

Should we do this?

[RalfJung]

Ignoring the SIFA thingy since I already forgot how it works, what I think would make sense is to do a single visit_freeze_sensitive where you

• do the parent access
• do the data race access
• prepare a RangeMap<LocationState> of per-location initial states that will later be passed to Tree::new_child

Does that makes sense or is it too ugly as well?

[JoJoDeveloping]

That would work. It has to construct an extra RangeMap but presumably this is cheaper than calling visit_freeze_sensitive thrice.
And it requires moving some logic for retags around quite a bit.

[RalfJung]

presumably this is cheaper than calling visit_freeze_sensitive thrice.

I'm not sure, but that's at least plausible. I also prefer it conceptually.

[RalfJung]

Thanks! This is a lot better.

I don't fully understand the logic around the new perms_map though. It seems like the map is created to cover all the bytes from the beginning of the allocation to the end of the retagged place, which is odd -- it should be either the entire allocation, or just the place. And given that we do not want to actually put anything into the perms map outside the place, I think it should be just the place.

[RalfJung]

I have pushed a commit with various minor cleanups -- please take a look, I hope they make sense. :)

However, in doing so I stumbled upon a question I could not answer.

[JoJoDeveloping]

There's one more nit about the fact that we use LocationState::new_init even if no initial access happens. (This is one of those cases where init means initialized instead of initial, one that should be renamed presumably)

[RalfJung]

Thanks, I think the logic is all good now! Now it's just fine-tuning comments and names.

Also there is one case where I wonder if we want a test to ensure that this is what we are doing -- if there even could be such a test.

@rustbot author

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.

[yoctocell]

@rustbot ready

[RalfJung]

All right, I think this is ready. :)
Please squash the commit history so that each commit makes independent sense. If that's too much work you can also squash everything into a single commit. (./miri squash can help you do that.)

@rustbot author

[RalfJung]

Looking at the diff of what you pushed, you removed one line from the comment for new_child...? What happened there?^^

[RalfJung]

FWIW the comment was outdated, it should say default_perm now.

[yoctocell]

Oops, probably an error I made when trying to fix some merge conflict when splitting the commits

[RalfJung]

Now it looks right. :) Great work, let's land this!

resolved

[RalfJung]

Looks like a timeout, let's try this again...

[RalfJung]

Now that we landed this, I realize... maybe we should put this behind a flag? Or have a flag to turn it off? Some people may wake up soon seeing their CI fail since they were testing with TB and this change affects their code.^^