Oddbean new post about login | logout Okay so I got two computers connected in a P2P fashion but I can only send binary messages Is there a good binary communications format that lets me send and receive simple text messages along with files (binary blobs) that's not HTTP? Sorry I'm a frontend dev I only know about HTTP and It feels wrong to re-implement HTTP #asknostr
Websockets
protocol buffers?
Or messagepack Or what is that Bencode
TCP Sockets
So, the http is just a protocol right? which means that ws:// is another. If I'm not mistaken, webrtc is also a different protocol. Nostr itself is a protocol over ws (web sockets). To send and receive binary over the wire all you need is to define the layer (for example TCP or UDP) and then select a protocol or make your own.
Just send over type + lenght + data for each fields over a TCP connection. It's easy to implement
it's also very easy to get it wrong by not at minimum using constants and it's far easier to use a TLV reader/writer library to do this, or at least a reader/writer library like the buffer type in golang the ease with which one grows the types of messages and the ease with which they can have one glitch appear due to a typo or moment of memory failure (human) makes it an extremely tiresome experience debugging APIs built from such primitives
The kernel does the buffering for you, you don't need a library for that. This is my long time favourite because of it's simpleness and complete control over what's going on in lower layers. Simple to debug. HZRD can probably make a decision for himself if he wants to dive in, or go in a different direction. He is not a newbie in programming, and after talking to him in person, I'm sure as hell he can do it.
but why waste your life rebuilding tools that already exist that let you skip right past hours and days of mysterious bugs due to your home-made codec? not to mention that they have to be reimplemented in another language as well that's why i'm saying grpc/protobuf it is the equal of the textual jsonrpc2, and look at the state of nostr's API codec! just look at it! home made, even if it uses an encoder library it's a hand made RPC API and it's poorly implemented as a result
just speaking from experience here, i have done a lot of custom codecs and custom RPC APIs and i don't do them anymore... i used my experience to improve the way that fiatjaf's golang library for nostr is written and architected, but i would not write my own codecs now so readily or at least use tools that make it a lower debug time and easier maintenance thing, and actually, if i could avoid it, i'd not rewrite the RPC API framework and codec, and just use grpc/protobuf i have a very nice guide on how to do grpc/protobuf apps in golang https://github.com/quanterall/kitchensink
the only exception to my advice about custom codecs is when you are writing a low level, high performance network transport i was working on such a thing last year and it is the single case where extreme performance and optimisation is warranted if the data volume is less than 1Mbit typical average throughput it's not worth optimising it
I think I could build a simple one. but in that case what are some good common practices? how do you commonly start a message and end a message?
Byte count followed by bytes. That is the most basic.
Byte count followed by bytes. That is the most basic.