Hey frens, someone familiar with Redis here? #asknostr
I am looking to use Redis to store millions of random walks, which are nothing more complex that an ordered array of numbers e.g. [0,1,2,3,4,5].
I've read the Docs about data structures, but I am still unsure whether I should store them as Redis strings e.g. "0,1,2,3,4,5", or as Redis Lists.
Lists seem more appropriate, however I know they are implemented using skip lists, which I feel would be a waste of memory since the operations I will need to do are simply GETALL and SET. I've read somewhere that, if the size of the list is lower than a specified parameter, they aren't implemented using skip lists but normal arrays, which are more memory efficient. To give some context, the size of the walk on average will be 7, and with overwhelming probability not longer than 100.
What do u think is best in my situation? Any advice or feedback is appreciated
If it were me, since the arrays will be short, I'd just store as string and convert to array when retrieved.
It doesn’t seem like you would be using push or pop functionality so I’d keep it simple with a string value. You mention list seems more appropriate but I reckon you can view your walk as a constant instead of an order list and then string does become more appropriate.
Since you need basic read and write functionality and you want to minimize storage of data, then store the array as binary data with the “set” command. You’ll need to serialize and deserialize the list and data in your app, which adds complexity but offers much higher performance.
Next optimization would be to pack the list into as little bits as possible, so if values of each element is small then you can even represent them with a bit set, single byte, short, etc. depending on the range of the number.
Thanks for the answer. Yeah the more I think about it, the more this seems the easiest solution. Currently I am storing it as "0,1,2,3,4,5", which makes debugging easier. Then as I said, I can encode it more efficiently in binary.
Oddbean new post about login | logout