Welcome to another Overly Complicate Project! This time, it started with some advice from our friends at r/DataHoarder and a fun tool called “youtube-dl”. This has taken a bit of tinkering and some custom code, but I now have an all-in-one solution that downloads Youtube videos from a playlist/channel, confirms progress to save bandwidth on future downloads, and stores them into a Plex library for local viewing. Let’s begin.
I first started with a VM running Windows Server 2019. Following the steps below, you can install the WSL (Windows Subsystem for Linux) and have a full Ubuntu/Linux shell to use. I chose Ubuntu 16.04LTS as this is my favorite version of the server software.
https://docs.microsoft.com/en-us/windows/wsl/install-on-server
After installing WSL, I ran the below in the BASH/Ubuntu shell to install Pip, ffmpeg (used for video conversion by youtube-dl) and youtube-dl:
sudo apt update
sudo apt install python-pip ffmpeg
sudo pip install youtube-dlThis will install all necessary packages needed and get you into a running state for youtube-dl. For my server, I have a 200GB boot disk and a 10TB secondary disk. So, opening the bash shell and changing to that disk, I made a folder called “youtube” to store all my videos into. As a test, you can run youtube-dl against a video of your choosing to confirm everything works. This is a basic command I use for everything which sets the filename, retries, progress file, etc:
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HERETo break this down:
- -o: Output of Playlist (channel as well)/Title.Extension of file
- –format: Best video and audio on the requested video
- –continue: continue if this was interrupted at last download progress
- –sleep-interval: sleep for 2 seconds between steps
- –verbose: verbose output
- –download-archive: track progress of downloaded videos to save time and bandwidth in PROGRESS.txt file
- –ignore-error: ignores errors and keeps processing
- –retries: retries when error 404 or similar found
- –write-info-json: output JSON string with information about video
- –embed-subs/–all-subs: use ffmpeg to burn in subs into video
The above is what I use for everything except 4K channels which are just too much space to hit in bulk. Using this, I tossed several of those commands into a .sh file such as this:
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HERE
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HERE
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HERE
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HERE
youtube-dl -o '%(playlist)s/%(title)s.%(ext)s' --format bestvideo+bestaudio/best --continue --sleep-interval 2 --verbose --download-archive PROGRESS.txt --ignore-errors --retries 10 --write-info-json --embed-subs --all-subs YOUTUBE_URL_HEREThen, simply run in your BASH application: sh YOUR_SCRIPT_NAME.sh
Cool, right? Now you have a script with all your commands to download your favorite channels or unlisted/public playlists. One of the cool things with the integration of BASH/Windows is now you can also make a Windows BATCH (.bat) file to launch this. Making a .bat file called whatever you want (runme.bat is my favorite), put in to call the script you built, first changing to your directory of YOUR script to properly run:
@ECHO off
echo Launching youtube download helper script in BASH...
timeout /t 5
bash -c "cd /mnt/e/youtube/;sh YOUR_SCRIPT_NAME.sh"
echo Completed!
timeout /t 60Neat, now you can single click the .bat file to launch your downloads! BUT, there’s something else you can do now: Windows Task Scheduler. Go into “Task Scheduler” in Windows, and create a simple task. In this, set it to whatever time you want and have it run daily/weekly/however you’d like. I have an example here of the one I use:
Have this simply run your BATCH (.bat) file and it will now fire off automatically as you requested. This completes your automated downloads portion. I took this a step further, however, because a lot of my favorite music and music videos get taken down frequently and I wanted a nice, simple way to search and watch/listen to them. Enter Plex.
Grab a copy of Plex server from online and install on your Windows system. Having some horsepower here is definitely recommended: minimum quad core and 6GB+ of RAM (I’m running 8 cores and 12GB due to extra processing of 40K+ videos).
Under Plex, configure a Library of “Other Videos” and search for your top directory where your Youtube videos will be downloading from. It will then scan and add any videos it found by name to make searching easier for the future. I also went into my Plex server options and configured it to check the library for changes every 12 hours or so to catch anything downloaded overnight. I wake up in the morning and my newly downloaded videos are processed and ready for viewing!
I hope you find this interesting and informative. This has been a long project and has gotten very complicated as I built a Perl based wrapper to automate more of this. I encourage you to tinker and make this more effective and easier for your specific situations. Maybe a wrapper script to pull URLs from a file? Good luck and happy tinkering!