how does a client handle windowing for concurrent requests? that has often been a serious performance problem for similar protocols such as sftp, especially for high latency links
the random access read model requires the server to buffer the whole of a dynamically generated file, it can’t stream to the client like HTTP
how does a client handle windowing for concurrent requests? that has often been a serious performance problem for similar protocols such as sftp, especially for high latency links
can you elaborate on this? i was under the impression that allowing QUIC as a transfer protocol would solve this problem.
Your intro says it’s supposed to work over TCP or TLS, in which case concurrent transfers will probably need something like http/2 flow control. To make the most out of a link with a large bandwidth-delay product you need to keep enough requests in flight to fill the link. You might decide that you only support firehose mode, since your protocol isn’t interactive, in which case you can probably ditch flow control. IIRC the reason for the hpn-ssh patches was that ssh is fairly chatty so its throughput could be accidentally limited by its buffer size.
In general, multiplexing over TCP is a big danger area in protocol design, leading to lots of performance and complexity problems if you aren’t careful. When I see a protocol doing it I hope to see some discussion in the spec about how the dangers are avoided or otherwise addressed.
this is false. due to using client-assigned identifiers, it is entirely possible (and indeed, recommended) to send an OPEN and READ request in a single TCP packet.
the random access read model requires the server to buffer the whole of a dynamically generated file, it can’t stream to the client like HTTP
the protocol is not designed for transfer of dynamically generated files.
Ah I totally missed which end assigns the file handle. I thought that was part of the usual concurrent protocol pattern in which there’s a request ID which is echoed in the corresponding response.
On re-reading it now it looks like it only allows one open per connection since I can’t see what links READs to OPENs? What happens if there’s more than one OPEN?
If it doesn’t support uploads or dynamic queries then the title is overselling it just a wee bit.
On re-reading it now it looks like it only allows one open per connection since I can’t see what links READs to OPENs? What happens if there’s more than one OPEN?
ah, yeah, i guess i was thinking the token field does that, but that’s also supposed to be unique-per request… oops.
i guess READ needs an open_token field in addition to it’s own token field.
If it doesn’t support uploads or dynamic queries then the title is overselling it just a wee bit.
yeah, i definitely didn’t do a good job of communicating the protocol’s limited scope.
Some things that immediately occur to me:
no support for MIME types
no support for QUERY or POST
no redirects
the open/read pattern is poor for startup latency
how does a client handle windowing for concurrent requests? that has often been a serious performance problem for similar protocols such as sftp, especially for high latency links
the random access read model requires the server to buffer the whole of a dynamically generated file, it can’t stream to the client like HTTP
score 7 · canonical
can you elaborate on this? i was under the impression that allowing QUIC as a transfer protocol would solve this problem.
score 2 · canonical
Your intro says it’s supposed to work over TCP or TLS, in which case concurrent transfers will probably need something like http/2 flow control. To make the most out of a link with a large bandwidth-delay product you need to keep enough requests in flight to fill the link. You might decide that you only support firehose mode, since your protocol isn’t interactive, in which case you can probably ditch flow control. IIRC the reason for the hpn-ssh patches was that ssh is fairly chatty so its throughput could be accidentally limited by its buffer size.
In general, multiplexing over TCP is a big danger area in protocol design, leading to lots of performance and complexity problems if you aren’t careful. When I see a protocol doing it I hope to see some discussion in the spec about how the dangers are avoided or otherwise addressed.
score 9 · canonical
Oh, another thing: If you have a custom URL format, you need to specify how that relates to TLS server name authentication, as described in RFC 9525
score 2 · canonical
this is false. due to using client-assigned identifiers, it is entirely possible (and indeed, recommended) to send an
OPENandREADrequest in a single TCP packet.the protocol is not designed for transfer of dynamically generated files.
score 1 · canonical
Ah I totally missed which end assigns the file handle. I thought that was part of the usual concurrent protocol pattern in which there’s a request ID which is echoed in the corresponding response.
On re-reading it now it looks like it only allows one open per connection since I can’t see what links READs to OPENs? What happens if there’s more than one OPEN?
If it doesn’t support uploads or dynamic queries then the title is overselling it just a wee bit.
score 6 · canonical
ah, yeah, i guess i was thinking the token field does that, but that’s also supposed to be unique-per request… oops.
i guess READ needs an
open_tokenfield in addition to it’s owntokenfield.yeah, i definitely didn’t do a good job of communicating the protocol’s limited scope.
score 1 · canonical