For example, because CHERI allows ptr-to-int-and-back “transmutations”, you could write code on top of CHERI where you have a custom alloc and all of your pointers end up having enormously powerful capabilities. (Plz correct me if I’m wrong.)
You are wrong :-).
Nothing in CHERI (including assembly) can bypass the capability model and we have proofs of this run against the formal model of the ISA. We allow pointers to be round tripped at intptr_t because not supporting that breaks basically all non-trivial C code, but doing so does not amplify rights. It remains a capability with the same bounds and if your arithmetic takes it out of bounds then that’s fine, but you can’t dereference it while it’s out of bounds (taking it too far out of bounds will make it unrepresentable and unable to be used as a capability) . This is important because we’re building a model that is a foundation for higher-level security abstractions.
If the same memory location is used for a pointer and an integer, the hardware guarantees atomicity with respect to the capability and its tag bit, so any data store (even a single byte) destroys the capability.
If you can’t do these things, you break C idioms that most non-trivial programs depend on. We did a lot of work with formal models of C and surveys of C codebases and programmers to identify a de facto C specification that encapsulated expected C behaviour.
Fil-C, being a language change, allows for the introduction of thin pointer annotations later, and those annotations could be made to be memory-safe. And, those annotations would eliminate substantially all of the overhead of Fil-C in those places where you used them. I think CHERI would have no answer to that, in the sense that Fil-C code that used those annotations wouldn’t have a good story for how to compile to CHERI and still get all of the safety guarantees.
Early versions of CHERI C used a hybrid ABI where some pointers were annotated and others were not. Annotated pointers are capabilities, the rest are integers relative to the default data capability. We found that the friction in adopting the hybrid model was more than an order of magnitude higher than the overhead of recompiling in pure capability mode. We can compile large (multi million line) C and C++ codebases with pure capability mode with small changes (often none), whereas annotating the pointers that want dynamic checks was a lot more work. Our tcpdump changes, for example, to bound packet buffers required around 700 lines of diffs, to build as pure cap required four. Most of the changes for pure-cap mode are easy to upstream because they’re things like using uintptr_t instead of size_t for things that round trip pointers through integer types (which is what C requires anyway). Changes like this have been upstreamed to git, a bunch of KDE projects, and so on.
So, we can do that, but it’s rarely a good idea. You end up with the annotations needing to flow through the code and it becomes a data-flow problem. C programs like to do things like put void* pointers in heterogenous collections and this is very painful if you have two kinds of pointer.
We now recommend that no one use the hybrid ABI because it starts off looking simpler (‘look, I can make these changes incrementally’) but ends up being more work in the end.
Note that this does not always apply to other languages. For example, Java and JavaScript VMs can use 32-bit pointers, left shifted by two or three, as object pointers. They are interpreted as relative to a safe-heap capability and can be bounded based on the dynamic type when passed FFI code. We do the same on a smaller scale with Microvium on CHERIoT: the JS runtime uses a semispace compacting heap and uses 16-bit offsets as JavaScript pointers. When it passes them to C code, it removes store permission and sets the bounds, so C code cannot violate the JavaScript invariants.
For example, because CHERI allows ptr-to-int-and-back “transmutations”, you could write code on top of CHERI where you have a custom alloc and all of your pointers end up having enormously powerful capabilities. (Plz correct me if I’m wrong.)
You are wrong :-).
Nothing in CHERI (including assembly) can bypass the capability model and we have proofs of this run against the formal model of the ISA. We allow pointers to be round tripped at intptr_t because not supporting that breaks basically all non-trivial C code, but doing so does not amplify rights. It remains a capability with the same bounds and if your arithmetic takes it out of bounds then that’s fine, but you can’t dereference it while it’s out of bounds (taking it too far out of bounds will make it unrepresentable and unable to be used as a capability) . This is important because we’re building a model that is a foundation for higher-level security abstractions.
I think you misunderstood me. The point is: if I write a custom malloc in CHERI like so:
mmap something big, now I have a capability with a large lower/upper range
loop over the memory to create a linked free list, now I have a linked list where every next ptr has the same large lower/upper range as the original mmap
implement my_awesome_alloc() as popping off the list. now I’m returning übercapabilities
This isn’t about bypassing the capability model. Nothing is being bypassed here, the capability model is working as intended. But, I now have übercapabilities flowing through the system.
Yes, that’s true, but it’s also unavoidable. The memory allocator is the only thing that knows what the bounds of the object are because it is the thing creating the object model. This is indistinguishable from the case where you create a large object via mmap, construct some internal structure, and hand out interior pointers from a C-language level. You need something (builtins, attributes, and so on) to indicate that the bounds should be reduced. I believe Alex Richardson’s PhD thesis has some discussion about this.
CHERI has no problems with further reducing bounds. A lot of our code is now compiled with sub-object bounds, where taking the address of a struct field gives you a pointer bounded to that specific field. For things using the container-of idiom (most invasive data structures in C), you can explicitly annotate structures or specific fields. This prevents overflow between struct fields (e.g. the Firefox vulnerability from a couple of years ago that used an intra-object overflow to bypass TLS certificate checks).
Fil-C avoids it by forcing folks to use the stdfil allocators.
I get that Fil-C gets there by making trade-offs that you consider unacceptable. But the allocator problem is not unavoidable.
A lot of our code is now compiled with sub-object bounds, where taking the address of a struct field gives you a pointer bounded to that specific field.
I’m curious, how much code can you compile that way?
An earlier project I did that made C safe in a similar way to Fil-C took that approach by default, but in practice, that thing required even more code changes than Fil-C.
In Fil-C, you can zrestrict(ptr, type, count), which gives you a ptr restricted to that type and count, with a runtime check to ensure that this is safe. Interestingly, this operation is like gold for an abstract interpreter, so there will be an incentive to use it aggressively if you want to optimize your code for Fil-C. That’s definitely not as good as a compiler mode that gives it to you by default, though!
This prevents overflow between struct fields (e.g. the Firefox vulnerability from a couple of years ago that used an intra-object overflow to bypass TLS certificate checks).
Yes, and that’s why it’s useful!
Fil-C partly addresses this. If the overflow between struct fields trips the wrong type (like you overflow an int array field and the next thing after it is a ptr, then you’ll trap).
Fil-C avoids it by forcing folks to use the stdfil allocators.
You can only force people to use specific allocators if you can detect things that act like allocators. I’ve seen a lot of code that uses malloc to back another allocator, so you need to spot things that take a void* and then return a void* derived from it. A lot of things that are not allocators also do this.
I’m curious, how much code can you compile that way?
It’s very variable. Almost no systems code unmodified, but quite a bit if you add a few annotations in headers. Most kernels (including Linux and NT) have a set of data structures derived from the 4BSD set, which use the container-of idiom, but you can annotate them elsewhere.
For a lot of userspace applications, it requires no code change, but they typically depend on libraries that do need some annotations. It also depends on the age of the code.
You are wrong :-).
Nothing in CHERI (including assembly) can bypass the capability model and we have proofs of this run against the formal model of the ISA. We allow pointers to be round tripped at intptr_t because not supporting that breaks basically all non-trivial C code, but doing so does not amplify rights. It remains a capability with the same bounds and if your arithmetic takes it out of bounds then that’s fine, but you can’t dereference it while it’s out of bounds (taking it too far out of bounds will make it unrepresentable and unable to be used as a capability) . This is important because we’re building a model that is a foundation for higher-level security abstractions.
If the same memory location is used for a pointer and an integer, the hardware guarantees atomicity with respect to the capability and its tag bit, so any data store (even a single byte) destroys the capability.
If you can’t do these things, you break C idioms that most non-trivial programs depend on. We did a lot of work with formal models of C and surveys of C codebases and programmers to identify a de facto C specification that encapsulated expected C behaviour.
Early versions of CHERI C used a hybrid ABI where some pointers were annotated and others were not. Annotated pointers are capabilities, the rest are integers relative to the default data capability. We found that the friction in adopting the hybrid model was more than an order of magnitude higher than the overhead of recompiling in pure capability mode. We can compile large (multi million line) C and C++ codebases with pure capability mode with small changes (often none), whereas annotating the pointers that want dynamic checks was a lot more work. Our tcpdump changes, for example, to bound packet buffers required around 700 lines of diffs, to build as pure cap required four. Most of the changes for pure-cap mode are easy to upstream because they’re things like using uintptr_t instead of size_t for things that round trip pointers through integer types (which is what C requires anyway). Changes like this have been upstreamed to git, a bunch of KDE projects, and so on.
So, we can do that, but it’s rarely a good idea. You end up with the annotations needing to flow through the code and it becomes a data-flow problem. C programs like to do things like put void* pointers in heterogenous collections and this is very painful if you have two kinds of pointer.
We now recommend that no one use the hybrid ABI because it starts off looking simpler (‘look, I can make these changes incrementally’) but ends up being more work in the end.
Note that this does not always apply to other languages. For example, Java and JavaScript VMs can use 32-bit pointers, left shifted by two or three, as object pointers. They are interpreted as relative to a safe-heap capability and can be bounded based on the dynamic type when passed FFI code. We do the same on a smaller scale with Microvium on CHERIoT: the JS runtime uses a semispace compacting heap and uses 16-bit offsets as JavaScript pointers. When it passes them to C code, it removes store permission and sets the bounds, so C code cannot violate the JavaScript invariants.
score 6 · canonical
I think you misunderstood me. The point is: if I write a custom malloc in CHERI like so:
mmap something big, now I have a capability with a large lower/upper range
loop over the memory to create a linked free list, now I have a linked list where every next ptr has the same large lower/upper range as the original mmap
implement my_awesome_alloc() as popping off the list. now I’m returning übercapabilities
This isn’t about bypassing the capability model. Nothing is being bypassed here, the capability model is working as intended. But, I now have übercapabilities flowing through the system.
score 3 · canonical
Yes, that’s true, but it’s also unavoidable. The memory allocator is the only thing that knows what the bounds of the object are because it is the thing creating the object model. This is indistinguishable from the case where you create a large object via mmap, construct some internal structure, and hand out interior pointers from a C-language level. You need something (builtins, attributes, and so on) to indicate that the bounds should be reduced. I believe Alex Richardson’s PhD thesis has some discussion about this.
CHERI has no problems with further reducing bounds. A lot of our code is now compiled with sub-object bounds, where taking the address of a struct field gives you a pointer bounded to that specific field. For things using the container-of idiom (most invasive data structures in C), you can explicitly annotate structures or specific fields. This prevents overflow between struct fields (e.g. the Firefox vulnerability from a couple of years ago that used an intra-object overflow to bypass TLS certificate checks).
score 4 · canonical
Fil-C avoids it by forcing folks to use the stdfil allocators.
I get that Fil-C gets there by making trade-offs that you consider unacceptable. But the allocator problem is not unavoidable.
I’m curious, how much code can you compile that way?
An earlier project I did that made C safe in a similar way to Fil-C took that approach by default, but in practice, that thing required even more code changes than Fil-C.
In Fil-C, you can
zrestrict(ptr, type, count), which gives you a ptr restricted to that type and count, with a runtime check to ensure that this is safe. Interestingly, this operation is like gold for an abstract interpreter, so there will be an incentive to use it aggressively if you want to optimize your code for Fil-C. That’s definitely not as good as a compiler mode that gives it to you by default, though!Yes, and that’s why it’s useful!
Fil-C partly addresses this. If the overflow between struct fields trips the wrong type (like you overflow an int array field and the next thing after it is a ptr, then you’ll trap).
score 3 · canonical
You can only force people to use specific allocators if you can detect things that act like allocators. I’ve seen a lot of code that uses malloc to back another allocator, so you need to spot things that take a void* and then return a void* derived from it. A lot of things that are not allocators also do this.
It’s very variable. Almost no systems code unmodified, but quite a bit if you add a few annotations in headers. Most kernels (including Linux and NT) have a set of data structures derived from the 4BSD set, which use the container-of idiom, but you can annotate them elsewhere.
For a lot of userspace applications, it requires no code change, but they typically depend on libraries that do need some annotations. It also depends on the age of the code.
score 4 · canonical