Oddbean

a Relay Test Suite is in the works... I've already got ideas for 94 tests and as I write these tests they expand out into even more tests. I hoping that exposing relay behavior will drive relay software towards consistently supporting the same way of doing certain things, and that this will help nostr compatibility. Tests include things like: * Can the public write events? * Can an authenticated person write somebody else's events? * Does the relay respond to minimally constrained filters? * Does limit 0 give an EOSE? * When you write an event and get OK-true, can you read it directly back? * Does limit behave properly when a filter has multiple pubkeys and multiple kinds? * Does it require AUTH to read DMs and giftwraps? * When submitting a replaceable event, are the prior events replaced? * When submitting a replaceable event with newer ones already present, is it rejected or ignored? * Does deletion of a replaceable event preserve the events that are newer than the deletion event? * Does it prompt for auth initially? * How large of an event can it handle? * Does it verify signatures? * How does it react to invalid UTF8? * Does it handle all JSON escape codes? What about surrogate pairs? * Does it preserve field order in events? * Does it preserve fields in events not defined by the nostr spec? * How many subscriptions does it allow simultaneously? * Do events with negative created_at values work? What about exponential notation? What about numbers that won't fit in a u64? ... that kind of thing. If you want to help contribute, even if you just have an idea of what to test, come on over to https://github.com/mikedilger/relay-tester and give feedback, open an issue, or better yet submit a PR. What about the other things I'm working on? * Gossip is on pause while we wait for the UI developer to handle some personal affairs, then we will have annotations and NIP-17 DMs. * Chorus is stable and upgraded to use Pocket * Pocket has been completed and is stable too, but I haven't tried to make it useful outside of the scope of a relay yet.

Very cool. I might use you then. I haven't tried setting up the other relays besides nostr-rs-relay and chorus. FYI The tester is designed to test a relay software implementation, not a live running relay, because: 1. It might crash a live relay, 2. It will put a lot of junk events into the relay, and 3. it might get false results if it ends up finding events it didn't inject But I'm sure someone will ignore that recommendation.

Building image is done ✅ I'm now able to rebuild the binary each time you'll update the cod and make new tests. Each relay will have SSL cert and tested with wss//. Current tests on a strfry default config relay: RESULTS: nip11_provided: PASS claimed_support_for_nip4: PASS claimed_support_for_nip9: PASS claimed_support_for_nip11: PASS claimed_support_for_nip26: INFO (not supported) claimed_support_for_nip29: INFO (not supported) claimed_support_for_nip40: PASS claimed_support_for_nip42: INFO (not supported) claimed_support_for_nip45: INFO (not supported) claimed_support_for_nip50: INFO (not supported) claimed_support_for_nip59: INFO (not supported) claimed_support_for_nip65: INFO (not supported) claimed_support_for_nip70: UNTESTED claimed_support_for_nip94: INFO (not supported) claimed_support_for_nip96: INFO (not supported) public_can_write: PASS public_can_read_back: PASS supports_eose: PASS find_by_id: UNTESTED find_by_pubkey_and_kind: UNTESTED find_by_pubkey_and_tags: UNTESTED find_by_kind_and_tags: UNTESTED find_by_tags: UNTESTED find_by_pubkey: UNTESTED find_by_scrape: UNTESTED find_replaceable_event: UNTESTED find_parameterized_replaceable_event: UNTESTED replaceable_event_removes_previous: UNTESTED replaceable_event_doesnt_remove_future: UNTESTED parameterized_replaceable_event_removes_previous: UNTESTED parameterized_replaceable_event_doesnt_remove_future: UNTESTED since_until_include_equals: UNTESTED limit_zero: UNTESTED event_always_gets_ok_reply: UNTESTED auth_always_gets_ok_reply: UNTESTED limit_works_across_multiple_filter_groups: UNTESTED serves_post_eose_events: UNTESTED no_timeout_while_subscribed: UNTESTED nip4_dms_require_auth: UNTESTED delete_by_id: UNTESTED delete_by_id_of_others: UNTESTED resubmission_of_deleted_by id: UNTESTED delete_by_npnaddr: UNTESTED delete_by_npnaddr_of_others: UNTESTED delete_by_npnaddr_preserves_newer: UNTESTED resubmission_of_deleted_by_npnaddr: UNTESTED resubmission_of_olderthan_deleted_by_npnaddr: UNTESTED resubmission_of_newerthan_deleted_by_npnaddr: UNTESTED delete_by_pnaddr: UNTESTED delete_by_pnaddr_of_others: UNTESTED delete_by_pnaddr_preserves_newer: UNTESTED delete_by_pnaddr_bound_by_dtag: UNTESTED resubmission_of_deleted_by_pnaddr: UNTESTED resubmission_of_olderthan_deleted_by_pnaddr: UNTESTED resubmission_of_newerthan_deleted_by_pnaddr: UNTESTED deleted_returns_ok_false: UNTESTED prompts_for_auth_initially: FAIL can_auth_as_unknown: UNTESTED unknown_can_write_own: UNTESTED unknown_can_readback_own: UNTESTED unknown_can_write_other: UNTESTED unknown_can_readback_other: UNTESTED can_auth_as_known: UNTESTED known_can_write_own: UNTESTED known_can_readback_own: UNTESTED known_can_write_other: UNTESTED known_can_readback_other: UNTESTED giftwraps_require_auth: UNTESTED large_contact_lists: UNTESTED preserves_json_field_order: UNTESTED preserves_nonstandard_json_fields: UNTESTED handles_event_kind_larger_than_16bit: UNTESTED handles_filter_kind_larger_than_16bit: UNTESTED handles_event_created_at_larger_than_64bit: UNTESTED handles_filter_created_at_larger_than_64bit: UNTESTED accepts_events_before_nostr_was_invented: UNTESTED accepts_negative_event_created_at: UNTESTED accepts_events_in_the_near_future: UNTESTED accepts_events_in_the_distant_future: UNTESTED accepts_negative_filter_created_at: UNTESTED accepts_filter_created_at_in_near_future: UNTESTED accepts_filter_created_at_in_distant_future: UNTESTED handles_all_json_escape_codes: UNTESTED handles_surrogate_pairs: UNTESTED verifies_signatures: UNTESTED accepts_invalid_utf8: UNTESTED accepts_null_characters: UNTESTED handles_filter_prefixes: UNTESTED keeps_ephemeral_events: UNTESTED max_subscriptions: UNTESTED allows_immediate_reconnect: UNTESTED idle_timeout_if_unsubscribed: UNTESTED handles_empty_tags: UNTESTED

I waited about a year for somebody else to do it and eventually got tired of waiting. Then when I started thinking about tests I couldn't stop. I couldn't sleep I kept getting up and writing down more test ideas.