And, of course, if you control allocation yourself somehow, nothing stops you from storing pointers as 32-bit (or maybe even 16-bit!) indexes into a buffer. Translate it into a pointer for memory operations by adding it to the base of your buffer (maybe left-shifting first if you only need aligned offsets). It requires up-front thought about how you design your application, but if you’ve identified it as desirable, it’s not difficult.
Generally malloc:ed memory is grown linearly from the end of the process data segment to the program break, so if you sbrk(0) before allocating any data I think you could do this with regular mallocs in any C-like language.
“Arena indices” in the post you’re replying to was referring to exactly that; I assumed people would know what that was shorthand for and didn’t elaborate. I was just pointing out that you can also easily do x32-style 32-bit absolute pointers within the standard 64-bit host ABI.
In Ninja, we used Node* as the unique identifier for files, so any given path was canonicalized into one of those and then we could use pointer comparisons instead of string comparisons for equailty. In my followup n2 experiment, I switched to using integer indexes within an array as ids for Rust ownership reasons, but also found that made it easy to then switch them to 32-bit integers.
This matters because these systems are basically large graph where each build step depends on a list of files, and with 32-bit file ids those lists can be arrays of 32-bit integers. At the time I found it wrote it made a few ms of difference on my builds and I didn’t measure memory, but due to how the program was structured it was a trivial change so why not!
And, of course, if you control allocation yourself somehow, nothing stops you from storing pointers as 32-bit (or maybe even 16-bit!) indexes into a buffer. Translate it into a pointer for memory operations by adding it to the base of your buffer (maybe left-shifting first if you only need aligned offsets). It requires up-front thought about how you design your application, but if you’ve identified it as desirable, it’s not difficult.
score 1 · canonical
Generally malloc:ed memory is grown linearly from the end of the process data segment to the program break, so if you
sbrk(0)before allocating any data I think you could do this with regular mallocs in any C-like language.score 1 · canonical
“Arena indices” in the post you’re replying to was referring to exactly that; I assumed people would know what that was shorthand for and didn’t elaborate. I was just pointing out that you can also easily do x32-style 32-bit absolute pointers within the standard 64-bit host ABI.
score 2 · canonical
Ah, no that’s on me for not reading thoroughly.
score 1 · canonical
In Ninja, we used
Node*as the unique identifier for files, so any given path was canonicalized into one of those and then we could use pointer comparisons instead of string comparisons for equailty. In my followupn2experiment, I switched to using integer indexes within an array as ids for Rust ownership reasons, but also found that made it easy to then switch them to 32-bit integers.This matters because these systems are basically large graph where each build step depends on a list of files, and with 32-bit file ids those lists can be arrays of 32-bit integers. At the time I found it wrote it made a few ms of difference on my builds and I didn’t measure memory, but due to how the program was structured it was a trivial change so why not!
score 1 · canonical