What does Flatpak get me, though? Do I have graphics? Sound? Vulkan/OpenGl? DBus? An accessibility API maybe? I don’t think there’s a way to do statically link most (any?) of those in my executable, so if they’re not as stable as the kernel we’re kinda back to square one.

Edit: okay, the concept of runtime there looks like what I want. I shall look into this.

score 2 · canonical

“I know”, I say, “I’ll use C - it’s been around for fifty years, it’s small and simple and got to be polished to a sheen by now.” Then I read an article like this one.

Which, you will note, is not about C. It’s about its standard library.

I know, I know. In practice a programming language isn’t just syntax and semantics, it’s also the standard library, the tools, the third party libraries, the package managers, the community, the entire freaking ecosystem. With C however, you can save yourself quite a bit of trouble if you avoid the standard library as much as you can, be extremely picky about your dependencies, then vendor and statically link pretty much everything.

Many depressing articles would suddenly be irrelevant to you if you did that. In this case, all you have to do is parse those numbers yourself. It’s work, but at least you can escape most of the madness.

score 1 · canonical

if you reeally need to check for underflow, and you need to use the libc function for some reason

Indeed: in many cases the standard lib is best avoided altogether (this was about errors, but there are locales too. And if you want your locale to influence how you interpret the comma and the point, you probably want to be explicit about it).

If proper floating point parsing isn’t too important, perhaps just do the most expedient thing that’s barely good enough. If it does matter though, you should probably roll your own. With any luck, there are good implementations out there you can steal from.

score 1 · canonical

How do you do the sync point in C99?

That’s the thing: I don’t.

The constraints for Monocypher are more or less, dependable C with zero dependencies. For me that’s C89 with volatile, // style comments, and declaring local variables anywhere. My only dependencies are stddef.h and stdint.h, from which I take size_t and the fixed size integers. I have no explicit runtime dependency, not even LibC — though compilers do sometime turn a loop of mine into a memcpy(3) call.

There’s not much I can do under such drastic constraints:

• No networking.
• No I/O.
• No explicit program aborts.
• No heap allocation.
• No multi-threading.

Monocypher does not, and will not, do multithreading. Only support it.

The code you see above is what I plan to ship:

• A toolkit for users to write their own implementation of Argon2.
• An single-thread implementation that uses that toolkit. The comments there aren’t pseudocode, they’re just comments, meant to document how one may parallelise Argon2.

Basically, Monocypher will “support” multi-threading by leaving it as an exercise to the user. That’s cheating, I know, but as far as I can tell multi-threading is the easy part. Here’s my pthreads code, minus error handling to reduce clutter:

#include <monocypher.h>
#include <pthread.h>
#include <stdlib.h>

typedef struct {
	pthread_t          thread;
	pthread_barrier_t *barrier;
	crypto_argon2_ctx *ctx;
	uint32_t           lane;
} thread_data;

static void *argon2_lane(void *ptr)
{
	thread_data *data      = ptr;
	uint32_t     lane      = data->lane;
	uint32_t     nb_passes = data->ctx->config.nb_passes;
	for (uint32_t pass = 0; pass < nb_passes; pass++) {
		for (uint32_t slice = 0; slice < 4; slice++) {
			crypto_argon2_segment(data->ctx, pass, slice, lane);
			pthread_barrier_wait(data->barrier);
		}
	}
	return NULL;
}

static int argon2_phtreads(uint8_t *hash, uint32_t hash_size,
                           crypto_argon2_config config,
                           crypto_argon2_inputs inputs,
                           crypto_argon2_extras extras)
{
	size_t       work_size = config.nb_blocks * 1024;
	size_t       data_size = config.nb_lanes  * sizeof(thread_data);
	void        *work_area = malloc(work_size);
	thread_data *lane_data = malloc(data_size);

	crypto_argon2_ctx ctx;
	crypto_argon2_init(&ctx, hash_size, work_area,
	                   config, inputs, extras);

	pthread_barrier_t barrier;
	pthread_barrier_init(&barrier, NULL, config.nb_lanes);

	for (size_t lane = 0; lane < config.nb_lanes; lane++) {
		lane_data[lane].barrier = &barrier;
		lane_data[lane].ctx     = &ctx;
		lane_data[lane].lane    = lane;
		pthread_create(&lane_data[lane].thread, NULL,
		               argon2_lane, lane_data + lane);
	}
	for (size_t lane = 0; lane < config.nb_lanes; lane++) {
		pthread_join(lane_data[lane].thread, NULL);
	}
	pthread_barrier_destroy(&barrier);

	crypto_argon2_final(&ctx, hash);
	free(work_area);
	free(lane_data);
	return 0;
}

I’ll need to add proper error handling, account for EAGAIN where appropriate, and perhaps add a couple asserts to guard the functions that are supposed to never fail. The result will go to the user manual, for users to copy & paste, or adapt to their own multi-threading API.

I use a similar approach with EdDSA, by the way. In Monocypher3 I had direct support for an incremental signature API, but the only way I could make it work with both BLAKE2b EdDSA and Ed25519 was a full blows v-table. It did the job, but it ultimately proved very inflexible.

Then one day a company approached me to ad Ed25519ph to Monocypher. Pretty quickly it was clear to me that my incremental interface with swappable hash couldn’t do it. I had to take my source code apart to assemble what they needed. It took me but a day, which they graciously paid full rate for.

That work gave me an idea: what if instead of using a v-table to swap in the parts I want into the full algorithm, I instead cut the algorithm into coarse constituent parts, and then use those parts to assemble whatever variation of the algorithm I, or my users, may need? Sure it falls short of direct support, but who cares, as long as it gives us the capability?

I think I managed to make it work. Heres’ Ed25519 and Ed25519ph, from the toolkit and SHA-512:

void crypto_ed25519_key_pair(u8 secret_key[64], u8 public_key[32], u8 seed[32])
{
	u8 a[64];
	COPY(a, seed, 32);                      // a[ 0..31]  = seed
	crypto_wipe(seed, 32);
	COPY(secret_key, a, 32);                // secret key = seed
	crypto_sha512(a, a, 32);                // a[ 0..31]  = scalar
	crypto_eddsa_trim_scalar(a, a);         // a[ 0..31]  = trimmed scalar
	crypto_eddsa_scalarbase(public_key, a); // public key = [trimmed scalar]B
	COPY(secret_key + 32, public_key, 32);  // secret key includes public half
	WIPE_BUFFER(a);
}

static void hash_reduce(u8 h[32],
                        const u8 *a, size_t a_size,
                        const u8 *b, size_t b_size,
                        const u8 *c, size_t c_size,
                        const u8 *d, size_t d_size)
{
	u8 hash[64];
	crypto_sha512_ctx ctx;
	crypto_sha512_init  (&ctx);
	crypto_sha512_update(&ctx, a, a_size);
	crypto_sha512_update(&ctx, b, b_size);
	crypto_sha512_update(&ctx, c, c_size);
	crypto_sha512_update(&ctx, d, d_size);
	crypto_sha512_final (&ctx, hash);
	crypto_eddsa_reduce(h, hash);
}

static void ed25519_dom_sign(u8 signature[64], const u8 secret_key[64],
                             const u8 *dom,     size_t dom_size,
                             const u8 *message, size_t message_size)
{
	u8 a[64];  // secret scalar and prefix
	u8 r[32];  // secret deterministic "random" nonce
	u8 h[32];  // publicly verifiable hash of the message (not wiped)
	u8 R[32];  // first half of the signature (allows overlapping inputs)
	const u8 *pk = secret_key + 32;

	crypto_sha512(a, secret_key, 32);
	crypto_eddsa_trim_scalar(a, a);
	hash_reduce(r, dom, dom_size, a + 32, 32, message, message_size, 0, 0);
	crypto_eddsa_scalarbase(R, r);
	hash_reduce(h, dom, dom_size, R, 32, pk, 32, message, message_size);
	COPY(signature, R, 32);
	crypto_eddsa_mul_add(signature + 32, h, a, r);

	WIPE_BUFFER(a);
	WIPE_BUFFER(r);
}

void crypto_ed25519_sign(u8 signature [64], const u8 secret_key[64],
                         const u8 *message, size_t message_size)
{
	ed25519_dom_sign(signature, secret_key, 0, 0, message, message_size);
}

int crypto_ed25519_check(const u8 signature[64], const u8 public_key[32],
                         const u8 *msg, size_t msg_size)
{
	u8 h_ram[32];
	hash_reduce(h_ram, signature, 32, public_key, 32, msg, msg_size, 0, 0);
	return crypto_eddsa_check_equation(signature, public_key, h_ram);
}

static const u8 domain[34] = "SigEd25519 no Ed25519 collisions\1";

void crypto_ed25519_ph_sign(uint8_t signature[64], const uint8_t secret_key[64],
                            const uint8_t message_hash[64])
{
	ed25519_dom_sign(signature, secret_key, domain, sizeof(domain),
	                 message_hash, 64);
}

int crypto_ed25519_ph_check(const uint8_t sig[64], const uint8_t pk[32],
                            const uint8_t msg_hash[64])
{
	u8 h_ram[32];
	hash_reduce(h_ram, domain, sizeof(domain), sig, 32, pk, 32, msg_hash, 64);
	return crypto_eddsa_check_equation(sig, pk, h_ram);
}

It’s not trivial. It’s not even easy, which is why it (optionally) ships with Monocypher. But it’s possible: one can look at the Ed25519 and Ed25519ph specs, look at Monocypher’s API (which you will note hides away the elliptic curve stuff), and come up with pretty much this code.

Argon2 by contrast is genuinely easy. I’m very happy about that, because it removes a major weakness from Monocypher. With that, it will get the compatibility and security that comes with processing parallel lanes in parallel. (One thing that makes password hashing & key derivation so tricky, is how its security is directly proportional to the speed of its implementation.)

Now I need to do some vectorisation… In pure C99. Can’t have it all, but I did manage to get significant gains from auto-vectorisation alone. With any luck, I may even saturate the memory bus.

score 1 · canonical

I’m currently adding multi-thread support for Argon2 in Monocypher. Without depending on any multi-thread API at all. I’m sticking to the “pure portable C99 with zero dependency” that makes this little library so easy to integrate pretty much everywhere.

Instead I chopped up the API into an init() update() final() structure that let users implement parallelism however they want:

void crypto_argon2_init(crypto_argon2_ctx *ctx, uint32_t hash_size,
                        void *work_area,
                        crypto_argon2_config config,
                        crypto_argon2_inputs inputs,
                        crypto_argon2_extras extras);
void crypto_argon2_segment(const crypto_argon2_ctx *ctx,
                           uint32_t pass, uint32_t slice, uint32_t lane);
void crypto_argon2_final(const crypto_argon2_ctx *ctx, uint8_t *hash);

The single thread implementation goes like this:

void crypto_argon2(u8 *hash, u32 hash_size, void *work_area,
                   crypto_argon2_config config,
                   crypto_argon2_inputs inputs,
                   crypto_argon2_extras extras)
{
	crypto_argon2_ctx ctx;
	crypto_argon2_init(&ctx, hash_size, work_area, config, inputs, extras);

	for (u32 pass = 0; pass < config.nb_passes; pass++) {
		for (u32, slice, 0, 4) {
			// May be done in parallel, one thread per lane.
			for (u32 lane = 0; lane < config.nb_lanes; lane++) {
				crypto_argon2_segment(&ctx, pass, slice, lane);
			}
			// Sync point: wait for all lanes to complete.
		}
	}

	crypto_argon2_final(&ctx, hash);
}

Early experiments with pthreads suggest this should easily accommodate any multi-thread API.

Next step is documentation, and provide a production worthy code example.

score 1 · canonical

So, website let user run arbitrary Python code on the server, right? Yeah, well, depending on the approach this could quickly devolve into a game of whack-a-mole. Can’t really be done indeed. For this I would probably try to write a Python interpreter that does not perform any I/O (yes, that means duplicating decades of work, or at least give up on all libraries that touch the FFI).

I mean, sandboxing a program, that not only isn’t made for this, but also runs arbitrary code, is asking for a heap of trouble. Proper sandboxing likely requires either rewriting anything that’s not meant for it, or put the whole thing in a (possibly formally verified) emulator, and probably not let the emulated program access any type of real time clock.

Thankfully now we can ship the Python interpreter in Web Assembly to run in the browser, so the only thing for users to hack is their own machine. It’s cheating of course, but for all my talk I have to admit I’d rather cheat than bootstrapping the entire computing world.

A company dealing in $billions however has no excuse in my opinion. If they can’t find a way to cheat, I do expect them to bootstrap the world. The regulator could start by holding the brass up top personally, civilly, and penally accountable for any hacking that results from the carelessness of their company. If Mr Altman faced prison time, you can bet there would be much fewer rogue AIs. Or at least they wouldn’t boast about them.

score 1 · canonical

So, the problem we’re trying to solve here is that JSON parsing is a bottleneck. Their solution is to do some cheaper pre-processing so they have less parsing to do. But then I have to ask:

Why are we using a slow text format to begin with?

I understand that not all past decisions can be reversed, and sometimes there’s no choice because of reasons¹. Still, before we even contemplate solving a problem, I believe we should think whether we could avoid it entirely.

In this case: how about using a custom binary format instead of JSON? You may not even have to roll your own if something like MessagePack is fast enough despite being pretty generic. And if you do roll your own, chances your format could be simple enough that it wouldn’t take more effort than implementing raw filtering.

[1]: Like, text is more readable and easier to debug when all your tools are text based. We started with JSON in our prototype and now our “production grade prototype” that calls itself an application is built around JSON and we’re stuck.

score 4 · canonical

What I’m wondering here, is how many rules would go away if instead of giving up on unification altogether, we stuck to local type inference? That would eliminate fewer rules for sure, but how few exactly?

There’s also the question of where the bloat comes from exactly: is it from unification alone, or is it from how unification interacts with everything else? If it’s the latter, then keeping “everything else” small could very well help us keeping a small type system. Personally I wouldn’t ask for much: basic built-in types, sum & product types, a built in array if singly linked lists aren’t enough… and not much else to be honest. Type classes maybe, existential quantification perhaps, and I’d skip inheritance for sure.

score 4 · canonical

While sandboxes are excellent, no sandbox will prevent a sufficiently-intelligent agent from finding ways to exceed its authorization.

Like, no sandbox is perfect? Like a vulnerability-free sandbox is not even possible?

Gimme a break.

While perfect certainty is impossible (probability theory 101), perfect software is merely hard. Sure there may be a whole freaking lot to check depending on the attack surface, but come on, current models don’t eat Linux privilege escalation bugs for breakfast.

And if it can’t be done with current software stacks, how about writing one from the ground up? It’s a big project for sure, but when you know the hardware, and the goals of your system are narrow like “make a sandbox for the LLM so it doesn’t turn clippy”, we’re talking like 20K lines of code, compiler included. We have examples.

The only truly impossible part here is convincing Nvidia to give you the datasheets.

score 1 · canonical

Interesting. I would have thought this commitment bias to be limited to humans. I would have thought ChatGPT rather immune to this, for I have seen it change its mind¹ rather easily when corrected.

[1]: please forgive the anthropomorphism

score 2 · canonical

You don’t have to use HTTP

I don’t know, I feel like firewalls increasingly make sure I do.

score 7 · canonical

Now that questionable quality is everywhere, package management and curation of software becomes more important, IMO.

Curation, unquestionably yes. Package management though? Take for instance the issue of static vs dynamic linking, with respect to security:

• Dynamic linking is better, because package maintainer can quickly apply a patch to a vulnerable library and instantly protect all programs that depends on it. Can’t trust developers with this one, some of them are bound to lag on updates, and we want to protect our users.
• Static linking is better, because developers can quickly apply a patch to a vulnerable library and instantly update their program without breaking anything. Can’t trust distros with this one, some of them are bound to lag on updates, and we want to protect our users.

What I’d really like in the Linux space, is a stable foundation. Something sufficient to build programs on top, including graphical & interaction intensive ones, that does not break users. Similar to the Linux kernel, or the old Windows APIs (before they started to break stuff). Something minimal, I’m not asking for a GUI toolkit, or even a font renderer. Just enough of an API so I can build my own, and have it work forever (that may involve protocols for screen readers & the like).

Now I wouldn’t mind a distro that encourages developers to use a specific set of dynamically linked dependencies. As long as they’re crystal clear which have a stable ABI, and the non-stable ones aren’t strictly necessary to build & deploy my program.

As a user though I love the official store front. Do keep pkg-mgr install awesome_software, automatic updates… I can’t just trust some random dev on the internet after all.

score 1 · canonical