RTP
paper.wf
refining the ideas behind HTTP, BitTorrent, Gemini, and 9p to create a simple protocol for reliably transferring large immutable files. ...
  • lobsters__weberc2@crustaceanson.theshelf.topM
    1·
    25 days ago

    The paper mentions a relationship between RTP and BitTorrent, but it doesn’t seem like RTP is a p2p protocol? Is it just the magnet links stuff? And I’m confused about the benefit of the magnet links content hashing stuff. Could you elaborate?

    score 2 · canonical

    • lobsters__binarycat@crustaceanson.theshelf.topOPM
      1·
      25 days ago

      because http is a complete mess that is impossible to get right.

      my ISP likes randomly closing TCP sockets, and i am tired of hacking around insufficiently robust http implementations.

      as far as i can tell, there is not a single extant http library that actually does things correctly out of the box. curl can get pretty close by throwing --continue-at - --retry-all-errors at everything, but even that doesn’t correctly handle mid-air collisions (you need If-Range for that, and I don’t know if there’s any easy way to do that with curl).

      score 3 · canonical

  • lobsters__fanf@crustaceanson.theshelf.topM
    1·
    25 days ago

    Some things that immediately occur to me:

    • no support for MIME types

    • no support for QUERY or POST

    • no redirects

    • the open/read pattern is poor for startup latency

    • how does a client handle windowing for concurrent requests? that has often been a serious performance problem for similar protocols such as sftp, especially for high latency links

    • the random access read model requires the server to buffer the whole of a dynamically generated file, it can’t stream to the client like HTTP

    score 7 · canonical

    • lobsters__binarycat@crustaceanson.theshelf.topOPM
      1·
      25 days ago

      the open/read pattern is poor for startup latency

      this is false. due to using client-assigned identifiers, it is entirely possible (and indeed, recommended) to send an OPEN and READ request in a single TCP packet.

      the random access read model requires the server to buffer the whole of a dynamically generated file, it can’t stream to the client like HTTP

      the protocol is not designed for transfer of dynamically generated files.

      score 1 · canonical

      • lobsters__fanf@crustaceanson.theshelf.topM
        1·
        25 days ago

        Ah I totally missed which end assigns the file handle. I thought that was part of the usual concurrent protocol pattern in which there’s a request ID which is echoed in the corresponding response.

        On re-reading it now it looks like it only allows one open per connection since I can’t see what links READs to OPENs? What happens if there’s more than one OPEN?

        If it doesn’t support uploads or dynamic queries then the title is overselling it just a wee bit.

        score 6 · canonical

        • lobsters__binarycat@crustaceanson.theshelf.topOPM
          1·
          25 days ago

          On re-reading it now it looks like it only allows one open per connection since I can’t see what links READs to OPENs? What happens if there’s more than one OPEN?

          ah, yeah, i guess i was thinking the token field does that, but that’s also supposed to be unique-per request… oops.

          i guess READ needs an open_token field in addition to it’s own token field.

          If it doesn’t support uploads or dynamic queries then the title is overselling it just a wee bit.

          yeah, i definitely didn’t do a good job of communicating the protocol’s limited scope.

          score 1 · canonical

    • lobsters__binarycat@crustaceanson.theshelf.topOPM
      1·
      25 days ago

      how does a client handle windowing for concurrent requests? that has often been a serious performance problem for similar protocols such as sftp, especially for high latency links

      can you elaborate on this? i was under the impression that allowing QUIC as a transfer protocol would solve this problem.

      score 2 · canonical

      • lobsters__fanf@crustaceanson.theshelf.topM
        1·
        25 days ago

        Your intro says it’s supposed to work over TCP or TLS, in which case concurrent transfers will probably need something like http/2 flow control. To make the most out of a link with a large bandwidth-delay product you need to keep enough requests in flight to fill the link. You might decide that you only support firehose mode, since your protocol isn’t interactive, in which case you can probably ditch flow control. IIRC the reason for the hpn-ssh patches was that ssh is fairly chatty so its throughput could be accidentally limited by its buffer size.

        In general, multiplexing over TCP is a big danger area in protocol design, leading to lots of performance and complexity problems if you aren’t careful. When I see a protocol doing it I hope to see some discussion in the spec about how the dangers are avoided or otherwise addressed.

        score 9 · canonical