Oddbean new post about | logout

Notes by youtube video dumps | export

 youtube videos.xz v2 with columns: videoid|unixtime|username|title

search videos by keyword in title:
curl -s https://files.catbox.moe/rhus4h.xz | xz -dc | grep -Pi ".{44}\Kdance"

search videos by username:
curl -s https://files.catbox.moe/rhus4h.xz | xz -dc | grep -i "linustech"

total videos: 36349

1,4M	videos.xz
3,7M	videos-v2.txt

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

search videos by keyword in title:
curl -s https://files.catbox.moe/3ccjsx.xz | xz -dc | grep -Pi ".{44}\Kdance"

search videos by username:
curl -s https://files.catbox.moe/3ccjsx.xz | xz -dc | grep -i "linustech"

total videos: 89074
crawling speed: 219 tps
1,3M	videos.xz
3,5M	videos-v2.txt

#youtube #videos #search #local #dbdump #siterip 
 yt.sh tool

https://files.catbox.moe/fyx18n.sh

this generates grid of random videos to show in web browser
./yt.sh show_videos

making such grid is simple thumbnails are available with videoid @ https://i.ytimg.com/vi/videoid/hqdefault.jpg

#youtube #videos #search #local #dbdump #siterip 
 yt.sh tool

https://files.catbox.moe/fyx18n.sh

this generates grid of random videos to show in web browser
./yt.sh show_ui

making such grid is simple thumbnails are available with videoid @ https://i.ytimg.com/vi/videoid/hqdefault.jpg

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

https://files.catbox.moe/fyx18n.sh

this generates grid of random videos to show in web browser
./yt.sh show_ui

making such grid is simple thumbnails are available with videoid @ https://i.ytimg.com/vi/videoid/hqdefault.jpg

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

search videos by keyword in title:
curl -s https://files.catbox.moe/i07ohj.xz | xz -dc | grep -Pi ".{44}\Kdance"

search videos by username:
curl -s https://files.catbox.moe/i07ohj.xz | xz -dc | grep -i "linustech"

total videos: 33127
crawling speed: 237 tps
564K	videos.xz
1,5M	videos-v2.txt

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

curl -s https://files.catbox.moe/jdt9va.xz | xz -dc | grep -Pi ".{44}\Kdance"

total videos: 29212
crawling speed: 235 tps
336K	videos.xz
1,8M	videos-v2.txt

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

curl -s https://files.catbox.moe/jdt9va.xz | xz -dc | grep -Pi ".{44}\Kdance"

total videos: 29212
crawling speed: 235 tps
336K	videos.xz
1,8M	videos-v2.txt

#youtube #videos #search #local #dbdump #siterip 
 youtube videos.xz v2 with columns: videoid|unixtime|username|title

curl -s https://files.catbox.moe/jdt9va.xz | xz -dc | grep -Pi ".{44}\Kdance"

total videos: 29212
crawling speed: 235 tps
336K	videos.xz
55M	videos.txt

#youtube #videos #search #local #dbdump #siterip 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/es1y7s.xz | xz -dc | grep -Pi ".{12}dance"

total videos: 604390
crawling speed: 68 tps
21M	videos.xz
50M	videos.txt 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/7b5k1n.xz | xz -dc | grep -Pi ".{12}dance"

total videos: 500021
crawling speed: 68 tps
17M	videos.xz
42M	videos.txt 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/mi1kxf.xz | xz -dc | grep -Pi ".{12}dance"

total videos: 500021
crawling speed: 68 tps
9,3M	videos.xz
42M	videos.txt 
 #youtube #videos #search #local #dbdump #siterip

for comparison, english wikipedia db dump is ~21 GiB in size
so only ids + titles of youtube videos contain more text than whole english wikipedia 
 #youtube #videos #search #local #dbdump #siterip

rough estimate, search database of 800 million videos would equal to 24 GiB .xz file,
65 GiB decompressed 
 #youtube #videos #search #local #dbdump #siterip

"YouTube has at least 800 million videos on the platform.
However, some sources claim more than 1 billion videos. Ultimately, the exact number of videos on YouTube is unknown."
- Steve Allen, November 23, 2023

so far we have crawled 308 142 videos
lots more to find 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/mi1kxf.xz | xz -dc | grep -Pi ".{12}dance"

total videos: 308142
crawling speed: 118 tps
9,3M	videos.xz
25M	videos.txt 
 "#youtube #videos #search #local #dbdump #siterip curl -s https://files.catbox.moe/mi1kxf.xz | xz -dc | grep -Pi ".{12}dance" total videos: 304467 crawling speed: 114 tps 9,3M videos.xz 25M videos.txt" 
 #youtube #videos #search #local #dbdump #siterip curl -s https://files.catbox.moe/mi1kxf.xz | xz -dc | grep -Pi ".{12}dance" total videos: 300097 crawling speed: 111 tps 9,3M videos.xz 25M videos.txt 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/jopmko.gz | gzip -dc | grep -i ".{12}dance"

total videos: 200186 crawling speed: 132 tps
8,9M videos.txt.gz
17M videos.txt 
 #youtube #videos #search #local #dbdump #siterip

curl -s https://files.catbox.moe/de0umy.gz | gzip -dc | grep "dance"

total videos: 150405 crawling speed: 77 tps
6,9M videos.txt.gz
13M videos.txt 
 #youtube #videos #search #local #dbdump #siterip
torrent download:
https://x0.at/pMAg.torrent

download:
https://files.catbox.moe/ukeq7n.gz

or use directly:
curl -s https://files.catbox.moe/ukeq7n.gz | gzip -dc | grep "dance"
total videos: 67624 crawling speed: 10 tps

2,7M videos.txt.gz
5,5M videos.txt 
 #youtube #videos #search #local #dbdump #siterip
torrent download:
magnet:?xt=urn:btih:2a905722c410b465ce826d48891a267d4467698b&dn=videos.txt.gz&xl=2727491&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=wss%3A%2F%2Ftracker.webtorrent.dev&ws=https%3A%2F%2Ffiles.catbox.moe%2Fukeq7n.gz

download:
https://files.catbox.moe/ukeq7n.gz

or use directly:
curl -s https://files.catbox.moe/ukeq7n.gz | gzip -dc | grep "dance"
total videos: 67624 crawling speed: 10 tps

2,7M videos.txt.gz
5,5M videos.txt 
 #youtube #videos #search #local #dbdump #siterip
download: https://files.catbox.moe/hexsg2.gz

or use directly:
```bash
curl -s https://files.catbox.moe/hexsg2.gz | gzip -dc | grep "dance"
```
total videos: 52355 crawling speed: 4 tps

2,0M videos.txt.gz
4,1M videos.txt 
 #youtube #videos #search #local #dbdump #siterip
download: https://files.catbox.moe/hexsg2.gz

or use directly:
curl -s https://files.catbox.moe/hexsg2.gz | gzip -dc | grep "dance"

total videos: 52355 crawling speed: 4 tps

2,0M videos.txt.gz
4,1M videos.txt 
 #youtube #videos #search #local #dbdump #siterip
https://files.catbox.moe/hexsg2.gz

usage:
curl -s https://files.catbox.moe/hexsg2.gz | gzip -dc | grep "dance"

total videos: 52355 crawling speed: 4 tps

2,0M videos.txt.gz
4,1M videos.txt 
 #youtube #videos
https://files.catbox.moe/m3eooa.gz

total videos: 50342 | crawling speed: 6 tps

1,9M videos.txt.gz
3,9M videos.txt 
 #youtube #videos
https://files.catbox.moe/m3eooa.gz

1

1,9M videos.txt.gz
3,9M videos.txt 
 #youtube #videos
https://files.catbox.moe/m3eooa.gz

1

1,9M videos.txt.gz
3,9M videos.txt 
 47311 #youtube #videos
https://files.catbox.moe/rq5uio.gz

1,8M videos.txt.gz
3,7M videos.txt 
 1192 #youtube #videos
https://files.catbox.moe/u6021q.txt

generated with yt.sh
fileloader: a5d115ec9ab03d3550011f24b74e176c3eab0fd8ca00d7a34761c791cdfcd3fe