#gm

realy managed not to die after 55gb of data at an eventual average speed of 1.2mb/s throughput

today i will make the import function batch like 10k events at a time, so it doesn't get bogged down with so many little writes

after stopping it, it eventually gets to compacting the tables and cpu goes to 100%

i'm sure the tables compact better when they are written in a batch

it's also making me pause to think about how maybe it would be better to keep the files on a filesystem, and then you don't even need this export step, it's just a matter of copying those files... ah but too complexity... 

well, here goes, batching it is...

outside it's rainy and windy like crazy, i woke up and it looks still very dark but i see all these cars driving around the streets, i'm like, oh, it must be past 7am already, yup

well, here goes 

 Gm