Oddbean

without an adequate reader/writer type framework that is fraught with bugs, don't set a newbie up with such bad advice he could figure that out himself and make that mistake himself but you helping him is not acceptable mentoring

it's also very easy to get it wrong by not at minimum using constants and it's far easier to use a TLV reader/writer library to do this, or at least a reader/writer library like the buffer type in golang the ease with which one grows the types of messages and the ease with which they can have one glitch appear due to a typo or moment of memory failure (human) makes it an extremely tiresome experience debugging APIs built from such primitives

The kernel does the buffering for you, you don't need a library for that. This is my long time favourite because of it's simpleness and complete control over what's going on in lower layers. Simple to debug. HZRD can probably make a decision for himself if he wants to dive in, or go in a different direction. He is not a newbie in programming, and after talking to him in person, I'm sure as hell he can do it.

but why waste your life rebuilding tools that already exist that let you skip right past hours and days of mysterious bugs due to your home-made codec? not to mention that they have to be reimplemented in another language as well that's why i'm saying grpc/protobuf it is the equal of the textual jsonrpc2, and look at the state of nostr's API codec! just look at it! home made, even if it uses an encoder library it's a hand made RPC API and it's poorly implemented as a result

just speaking from experience here, i have done a lot of custom codecs and custom RPC APIs and i don't do them anymore... i used my experience to improve the way that fiatjaf's golang library for nostr is written and architected, but i would not write my own codecs now so readily or at least use tools that make it a lower debug time and easier maintenance thing, and actually, if i could avoid it, i'd not rewrite the RPC API framework and codec, and just use grpc/protobuf i have a very nice guide on how to do grpc/protobuf apps in golang https://github.com/quanterall/kitchensink

the only exception to my advice about custom codecs is when you are writing a low level, high performance network transport i was working on such a thing last year and it is the single case where extreme performance and optimisation is warranted if the data volume is less than 1Mbit typical average throughput it's not worth optimising it