So, the app spends a lot of time waiting for that server.
This means the client is server-bound, not that the client (or server) is I/O bound. The server could be spending an unnecessary amount of time serializing things or processing the request slowly. It rarely reaches an I/O hardware bottleneck.
I think this distinction is worth making, especially in the context of the post, which is arguing faster languages don’t make most software faster.
Certainly, if your client/server operation is bounded by the network connection, and the response time is comparable to ping time, no language or framework is going to make this faster. You’d need to either physically move the machines closer, or change the design.
But for client/server applications that do a lot of server-side computing, response time can dramatically exceed ping time, and rewriting the server in a faster language can absolutely get you a lot closer to the ping limit.
The first piece of software I ever put into production was an interface for querying SLURM clusters. The original Python script for doing the same thing took around 3s, and the Rust rewrite got this down to ~120ms. When I looked at the profiles, it turned out around ~70ms was spent waiting for the daemon to respond with the data, and no optimization, redesign, or language change would get me below that limit. But the rewrite moved us from being functionally process-bound to functionally I/O-bound.
So I’m very willing to believe that there’s a lot of software out there that is I/O-bound (in the sense of waiting for a response from a server) where a faster implementation of the server (or perhaps even the client) could dramatically speed up response time, even though ping hasn’t gotten faster, to the point of being I/O-bound (in the sense that you’re stuck physically waiting for packets to arrive).
That’s a serialized communication dependency. The same category of “wait” occurs when a thread is blocked waiting on another, something I wouldn’t call “I/O”.
To me, “I/O” bound means the request perf/latency limited by the communication medium which you can’t control. Here, speeding up the server’s processing also lowers client latency, so the request isn’t bound by the client->server comms channel itself.
A network request has always been considered IO. Of course there is computation going on on the other end of the request that you equally have to optimize. But a client making frequent network requests and waiting on the result is still IO-bound.
I don’t see how it’s IO bound if the performance is not bound / bottlenecked by the IO itself. It’s like saying the client is CPU bound because it has to wait on ALU instructions to execute the code that triggers the network request… The act of waiting on something doesn’t make the operation bound / bottlenecked on it
This means the client is server-bound, not that the client (or server) is I/O bound. The server could be spending an unnecessary amount of time serializing things or processing the request slowly. It rarely reaches an I/O hardware bottleneck.
score 1 · canonical
Come on. When one process waits on another process, that’s called IO bound. This shouldn’t have to be said but I guess we’re arguing in bad faith now.
score 1 · canonical
I think this distinction is worth making, especially in the context of the post, which is arguing faster languages don’t make most software faster.
Certainly, if your client/server operation is bounded by the network connection, and the response time is comparable to ping time, no language or framework is going to make this faster. You’d need to either physically move the machines closer, or change the design.
But for client/server applications that do a lot of server-side computing, response time can dramatically exceed ping time, and rewriting the server in a faster language can absolutely get you a lot closer to the ping limit.
The first piece of software I ever put into production was an interface for querying SLURM clusters. The original Python script for doing the same thing took around 3s, and the Rust rewrite got this down to ~120ms. When I looked at the profiles, it turned out around ~70ms was spent waiting for the daemon to respond with the data, and no optimization, redesign, or language change would get me below that limit. But the rewrite moved us from being functionally process-bound to functionally I/O-bound.
So I’m very willing to believe that there’s a lot of software out there that is I/O-bound (in the sense of waiting for a response from a server) where a faster implementation of the server (or perhaps even the client) could dramatically speed up response time, even though ping hasn’t gotten faster, to the point of being I/O-bound (in the sense that you’re stuck physically waiting for packets to arrive).
score 1 · canonical
That’s a serialized communication dependency. The same category of “wait” occurs when a thread is blocked waiting on another, something I wouldn’t call “I/O”.
To me, “I/O” bound means the request perf/latency limited by the communication medium which you can’t control. Here, speeding up the server’s processing also lowers client latency, so the request isn’t bound by the client->server comms channel itself.
I don’t see how this is bad faith
score 1 · canonical
That’s a very uncommon definition of IO.
A network request has always been considered IO. Of course there is computation going on on the other end of the request that you equally have to optimize. But a client making frequent network requests and waiting on the result is still IO-bound.
score 1 · canonical
The client is doing a network request, but it’s not bound by the speed of the network in this example. Thats what i mean by communication medium.
score 1 · canonical
I understand what you mean. Everything affects performance, you have to figure out where in the full chain is responsible for latency.
Do you understand that no matter where the latency is, a client waiting on a network request is still considered IO bound?
score 1 · canonical
I don’t see how it’s IO bound if the performance is not bound / bottlenecked by the IO itself. It’s like saying the client is CPU bound because it has to wait on ALU instructions to execute the code that triggers the network request… The act of waiting on something doesn’t make the operation bound / bottlenecked on it
score 1 · canonical
Ok then. We’ve both expressed our sides clearly. Best of luck to you.
score 1 · canonical