Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Tube Archivist is quite heavyweight as it's meant to do heavy full archiving of YouTube channels and search through positively huge libraries. I'm getting the sense that it's a data hoarding tool, not a casual web video watching tool. I found that I just want to add a few channels to my media library, for which I use Jellyfin already.

For people looking for a more lightweight option of that kind, I run the following script hourly [1]. This script uses yt-dlp to go through a text file full of YouTube RSS urls (either a channel RSS or a playlist RSS works for channels where you're only interested in a subset of videos) [2] and downloads the latest 5 videos organized in folders based on channel name. I watch these files by adding the output folder in a Jellyfin "Movies" type library sorted by most recent. The script contains a bunch of flags to make sure Jellyfin can display video metadata and thumbnails without any further plugins, and repackages videos in a format that is 1080p yet plays efficiently even in web browsers on devices released in at least the last 10 years.

It uses yt-dlp's "archive" functionality to keep track of videos it's already downloaded such that it only downloads a video once, and I use a separate script to clean out files older than two weeks once in a while. Running the script depends on ffmpeg (just used for repackaging videos, not transcoding!), xq (usually comes packaged with jq or yq) and yt-dlp being installed. You sometimes will need to update yt-dlp if a YouTube side change breaks it.

For my personal usage it's been honed for a little while and now runs reliably for my purposes at least. Hope it's useful to more people.

[1]: https://pastebin.com/s6kSzXrL

[2]: E.g. https://danielmiessler.com/p/rss-feed-youtube-channel/



Hope you don't mind that I adapted this into a quick container image[1]. (Feel free to scoff at the idea of taking a simple bash script and making it into a massive Docker image; you're right, I just wanted a convenient way to run it in Linux environments that I don't have great control over.) I know it's not a huge script, but nonetheless if you want I can add a LICENSE/copyright notice in my fork/adaptation, if you want to pick a license for this script.

[1]: https://github.com/jchv/ytdl-pvr


Oh great! Yeah as you can probably tell from the script I'm using it as a (locally built) container in my own setup. Feel free to pretend it's BSD licensed if that helps :)


Downloading whole channels and searching them shouldn't be heavy weight. I do that with yt-dlp and an index stored in a SQLite db.

inb4 dropbox/rsync reference. yeah yeah, I'm not saying everybody should do it like this, I'm just saying that archiving and indexing/searching needn't be heavyweight. I'm sure there's plenty of utility in a nice GUI for it, but it could easily be a light weight GUI.


Looks great, but I couldn't get it working, what version of xq are you using this against? 1.2.1 does not seem to have a -r argument.


You need to use yq, which has an xq binary.

(edit: removed my "plug" since I mentioned it elsewhere.)


Ah sorry, missed that, thanks!


No problem! It is a bit obscure, to be honest, I was also a bit confused at first until it hit me.


yt-dlp also has flags to write subtitle files and .info.json files, which at least Emby can automatically pick up and use, if not Jellyfin.

I haven’t yet wired up the bits to use whispercpp to automatically generate subtitles for downloads, but I have done so on an ad-hoc basis in the past and gotten (much) better results than the YouTube auto-generated subtitles.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: