interesting, and you've tried strfry and failed? There was a rust relay that I saw a while back that seemed customizable, can't find it though.
Not really sure what's entailed in building your own relay from scratch, but that might be a good place to start for custom use case? If the requirements are the same across relay functionality, then maybe you can build the foundation and then your "nice to haves" on to of that? Or maybe you've tried this already?
strfry was easy to use and install, but I never really tinkered with it more than was required. That rust one also looked good, but i don't know rust so I left it out. The other relays were like personal relays and stuff. Someone who would know would be nostr:npub1x0r5gflnk2mn6h3c70nvnywpy2j46gzqwg6k7uw6fxswyz0md9qqnhshtn but I think he's moved away from nostr to something else.