Porteus switched from youtube-dl to yt-dlp - and how to get the most recent version

[Rava]

[posts split from FFMPEG ]

i happen to have a script already written for that.. i guess i could easily swap out the live update script on the server (the updater app will still say youtube-dl, but would supply yt-dlp instead..)

Is it already live?
Or must the Porteus 5.0 user do some manual trick to have yt-dlp instead of youtube-dl?

Added in 4 minutes 58 seconds:
https://slakfinder.org/index.php?act=se ... e=#results has 2 results:
https://slakfinder.org/sbomt64-15.0/net ... 4-1smt.txz (repository SBOmt)
and
https://packages.slackonly.com/pub/pack ... slonly.txz (repository slackonly)
Which one to choose?

[beny]

hi rava, the yl-dlp from git is better, you can put the directory in every place you like and work standalone,but you can try the sbomt one too

[Rava]

^
Thanks beny… while you replied I made the module out of SBOmt, because… for me slackonly was not reacting.

Is there a way to automatically download the git version?

Created yt-dlp-2022.01.21-x86_64-1smt.xzm

Here a comparison. Cave! A download started via youtube-dl will not be resumed by yt-dlp
I cancelled the youtube-dl download after it finished the video part and during the audio part.

Here is the abysmal slow speed that is youtube-dl

Code: Select all

guest@porteus:/EINSORTIEREN$ youtube-dl -f 243+250 https://www.youtube.com/watch?v=g8kcyRv8cV4
[youtube] g8kcyRv8cV4: Downloading webpage
[download] Destination: Drauf sein auf TikTok - Likes für den Rausch _ STRG_F-g8kcyRv8cV4.f243.webm
[download] 100% of 42.01MiB in 12:27
[download] Destination: Drauf sein auf TikTok - Likes für den Rausch _ STRG_F-g8kcyRv8cV4.f250.webm
[download]  65.5% of 15.50MiB at 55.65KiB/s ETA 01:38^C
ERROR: Interrupted by user

And here is yt-dlp as comparison:

Code: Select all

guest@porteus:/EINSORTIEREN$ yt-dlp -f 243+250 https://www.youtube.com/watch?v=g8kcyRv8cV4
[youtube] g8kcyRv8cV4: Downloading webpage
[youtube] g8kcyRvyt-dlp -8cV4: Downloading android player API JSON
[info] g8kcyRv8cV4: Downloading 1 format(s): 243+250
[download] Destination: Drauf sein auf TikTok - Likes für den Rausch _ STRG_F [g8kcyRv8cV4].f243.webm
[download] 100% of 42.01MiB in 00:43
[download] Destination: Drauf sein auf TikTok - Likes für den Rausch _ STRG_F [g8kcyRv8cV4].f250.webm
[download] 100% of 15.50MiB in 00:19
[Merger] Merging formats into "Drauf sein auf TikTok - Likes für den Rausch _ STRG_F [g8kcyRv8cV4].webm"
Deleting original file Drauf sein auf TikTok - Likes für den Rausch _ STRG_F [g8kcyRv8cV4].f243.webm (pass -k to keep)
Deleting original file Drauf sein auf TikTok - Likes für den Rausch _ STRG_F [g8kcyRv8cV4].f250.webm (pass -k to keep)

and since both fully downloaded the video part let's compare just these:
youtube-dl - f243.webm - 100% of 42.01MiB in 12:27
yt-dlp - f243.webm - 100% of 42.01MiB in 00:43

I think 12:27 vs 00:43 speaks for itself. Also I like the coloured output of yt-dlp (only visible during the download, the colours vanish after the download is finished)

Added in 34 minutes 6 seconds:
Overall, I suggest Porteus should drop youtube-dl and switch to yt-dlp.

The most recent module sizes look better for youtube-dl:
1.66 MB 031-youtube-dl-2021.12.17-noarch-rava.xzm
3.51 MB 031-yt-dlp-2022.01.21-x86_64-1smt.xzm

Maybe the yt-dlp gets smaller when using the git sources, or maybe when the download via update-browser works we get a smaller module.

For now, upgrading to yt-dlp via update-browser does not work: by the version number it is obviously that was is listed as youtube-dl is really youtube-dl

Code: Select all

root@porteus:/mybin/linux/yt-dlp# update-browser -y
 Starting checks ... 
[OK] User is root.
[OK] Distro is Porteus
Downloading live script ...
Downloading: update-youtube-dl-live  DONE
Work will be done in: /tmp 
youtube-dl is not installed
Checking http://dl.porteus.org/x86_64/current/modules
The Porteus SERVER version is :  2021.12.17 

The latest youtube-dl version is    :  2021.12.17 

 You can choose to download the premade module from our server, 
 or download the version from github. 

 Would you like to download the porteus server version? [y/n]
 Would you like to download  youtube-dl-2021.12.17  ? [y/n]
root@porteus:/mybin/linux/yt-dlp# 

I pressed "Ctrl+C" at the last line but that is not echoed in the output, still the script exited immediately.

[beny]

hi rava, just for the record the SBOmt packages are build from the sbo-script from the mantainer of slackpkg+ and slackfinder,so i think we can trust the source of this packages.

[Rava]

^
I am also sure we can trust the packages.

My want of having a smaller module is not about trust.

[ncmprhnsbl]

Is it already live?

lol
it is now (give some time for mirror propagation, if you've set your mirror via fastest-mirror, which you should)

[Rava]

it is now (give some time for mirror propagation, if you've set your mirror via fastest-mirror, which you should)

I presume first running (as root)

Code: Select all

 fastest-mirror

and then (as root)

Code: Select all

update-browser

Added in 8 minutes 34 seconds:
^ that was on my i586-Port4.0 - now back on x86-64-Port5.0:

Code: Select all

root@porteus:/Lsfind#  fastest-mirror
Checking fastest mirror ...
Testing http://ftp.vim.org/ftp/os/Linux/distr/porteus/ 0.215483
Testing http://ftp.nluug.nl/os/Linux/distr/porteus/ 0.084592
Testing http://mirrors.dotsrc.org/porteus/ 0.147822
Testing https://mirrors.dotsrc.org/porteus/ 0.258695
Testing http://ftp.cc.uoc.gr/pub/linux/porteus/ 0.211628
Testing http://www6.frugalware.org/mirrors/linux/porteus/ 0.280224
Testing http://ftp.riken.jp/Linux/porteus/ 0.790946
Testing http://www.mirrorservice.org/sites/dl.porteus.org/ 0.196288
Testing http://linux.rz.rub.de/porteus/ 0.158406
Testing http://ftpmirror1.infania.net/mirror/porteus/ 0.662943
Testing https://mirror.yandex.ru/mirrors/porteus/ 0.323233

Results:

0.084592	http://ftp.nluug.nl/os/Linux/distr/porteus/
0.147822	http://mirrors.dotsrc.org/porteus/
0.158406	http://linux.rz.rub.de/porteus/
0.196288	http://www.mirrorservice.org/sites/dl.porteus.org/
0.211628	http://ftp.cc.uoc.gr/pub/linux/porteus/
0.215483	http://ftp.vim.org/ftp/os/Linux/distr/porteus/
0.258695	https://mirrors.dotsrc.org/porteus/
0.280224	http://www6.frugalware.org/mirrors/linux/porteus/
0.323233	https://mirror.yandex.ru/mirrors/porteus/
0.662943	http://ftpmirror1.infania.net/mirror/porteus/
0.790946	http://ftp.riken.jp/Linux/porteus/

 Would you like to add http://ftp.nluug.nl/os/Linux/distr/porteus/ as your main Porteus server? [y/n]
 Porteus server updated in /etc/porteus.conf

So far, so good.

Added in 2 minutes 20 seconds:

Code: Select all

root@porteus:/Lsfind# update-browser -y
 Starting checks ... 
[OK] User is root.
[OK] Distro is Porteus
Downloading live script ...
Downloading: update-youtube-dl-live  DONE
Work will be done in: /tmp 

 Note: now using yt-dlp instead of youtube-dl.
update-youtube-dl-live
you have version 2022.01.21 installed
Checking http://ftp.nluug.nl/os/Linux/distr/porteus//x86_64/current/modules
The Porteus SERVER version is :   

The latest yt-dlp version is    :  2022.08.19 

 You can choose to download the premade module from our server, or download the version from github. 

 Would you like to download the porteus server version? [y/n]

Downloading:   DONE

[ERROR] Download of the yt-dlp module failed.

mirror propagation complete, but update-youtube-dl-live still fails. Now trying the github version.

Added in 6 minutes 16 seconds:

Code: Select all

root@porteus:/Lsfind# update-browser -y
 Starting checks ... 
[OK] User is root.
[OK] Distro is Porteus
Downloading live script ...
Downloading: update-youtube-dl-live  DONE
Work will be done in: /tmp 

 Note: now using yt-dlp instead of youtube-dl.

you have version 2022.01.21 installed
Checking http://ftp.nluug.nl/os/Linux/distr/porteus//x86_64/current/modules
The Porteus SERVER version is :   

The latest yt-dlp version is    :  2022.08.19 

 You can choose to download the premade module from our server, 
 or download the version from github. 

 Would you like to download the porteus server version? [y/n]
 Would you like to download  yt-dlp-2022.08.19  ? [y/n]
Downloading: yt-dlp https://objectsgithubusercontentcom/github-production-release-asset-2e65be/307260205/98dc3b19-1288-44ba-a6b6-0d8f117bb700?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220830%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220830T045013Z&X-Amz-Expires=300&X-Amz-Signature=64a48bbeaa56ca4212cffcebbaf93d0ae7dac2fb76dba39d1cac5f98003db88a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=307260205&response-content-disposition=attachment%3B%20filename%3Dyt-dlp&response-content-type=application%2Foctet-st06:50:1 DONE
Downloading: SHA2-256SUMS https://objectsgithubusercontentcom/github-production-release-asset-2e65be/307260205/c8d33a62-f409-4fba-99ff-fb80c5554c65?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220830%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220830T044944Z&X-Amz-Expires=300&X-Amz-Signature=6c07d67770e33aff4df2d6eb06ab23834d5b0e9e4dad2398edad8bb44ef0ff85&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=307260205&response-content-disposition=attachment%3B%20filename%3DSHA2-256SUMS&response-content-type=application%2Foctet-st06:50: DONE
Verifying download integrity..
Server  : a062861444f63fce2e70f6a62fa0cc7b19e9066d379033bf9b71ac2fc3969ab6
Download: a062861444f63fce2e70f6a62fa0cc7b19e9066d379033bf9b71ac2fc3969ab6
Download verified.
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on yt-dlp-2022.08.19-noarch-1ncm.xzm, block size 262144.
[=================================================================-] 11/11 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
	compressed data, compressed metadata, compressed fragments,
	compressed xattrs, compressed ids
	duplicates are removed
Filesystem size 2373.78 Kbytes (2.32 Mbytes)
	95.37% of uncompressed filesystem size (2489.09 Kbytes)
Inode table size 210 bytes (0.21 Kbytes)
	63.64% of uncompressed inode table size (330 bytes)
Directory table size 182 bytes (0.18 Kbytes)
	85.45% of uncompressed directory table size (213 bytes)
Number of duplicate files found 0
Number of inodes 9
Number of files 2
Number of fragments 1
Number of symbolic links 0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 7
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Your module  yt-dlp-2022.08.19-noarch-1ncm.xzm  is ready in /tmp 
 Please copy it to your modules folder or somewhere safe. 

root@porteus:/Lsfind# l /tmp/yt-dlp-2022.08.19-noarch-1ncm.xzm 
-rw-r--r-- 1 root 2433024 2022-08-30 06:50 /tmp/yt-dlp-2022.08.19-noarch-1ncm.xzm

As I surmised, the resulting module would be smaller than my own.

We remember, mine is 3.51 MB 031-yt-dlp-2022.01.21-x86_64-1smt.xzm or all in bytes:

Code: Select all

root@porteus:/Porteus_modules# l *yt-dlp*
-rw-r--r-- 1 guest 3678208 2022-08-29 17:34 031-yt-dlp-2022.01.21-x86_64-1smt.xzm
lrwxrwxrwx 1 root       37 2022-08-30 00:45 031-yt-dlp-RECENT.xzm -> 031-yt-dlp-2022.01.21-x86_64-1smt.xzm
-rw-r--r-- 1 root  2433024 2022-08-30 06:50 yt-dlp-2022.08.19-noarch-1ncm.xzm
root@porteus:/Porteus_modules# mv yt-dlp-2022.08.19-noarch-1ncm.xzm 031-yt-dlp-2022.08.19-noarch-1ncm.xzm
root@porteus:/Porteus_modules# ln -sf 031-yt-dlp-2022.08.19-noarch-1ncm.xzm 031-yt-dlp-RECENT.xzm
root@porteus:/Porteus_modules# l *yt-dlp*
-rw-r--r-- 1 guest 3678208 2022-08-29 17:34 031-yt-dlp-2022.01.21-x86_64-1smt.xzm
-rw-r--r-- 1 root  2433024 2022-08-30 06:50 031-yt-dlp-2022.08.19-noarch-1ncm.xzm
lrwxrwxrwx 1 root       37 2022-08-30 06:53 031-yt-dlp-RECENT.xzm -> 031-yt-dlp-2022.08.19-noarch-1ncm.xzm

As you might surmise, in porteus/base/ I have a symlink "031-yt-dlp-RECENT.xzm" that links to /Porteus_modules/031-yt-dlp-RECENT.xzm - that is a symlink that links to the most recent module, in our case to 031-yt-dlp-2022.08.19-noarch-1ncm.xzm

Added in 5 minutes 50 seconds:
Why do I do so?
Because that way I can several different porteus/base/* setup (XFCE, KDE, Openbox, whatnot), and all link to the most recent of my modules without the need of updating anything in base/ - I only have to update the nnn-whatever-RECENT.xzm to what the most RECENT actual module is.

Code: Select all

root@porteus:/Porteus_modules# l *RECENT.xzm
lrwxrwxrwx 1 root 37 2022-08-30 06:53 031-yt-dlp-RECENT.xzm -> 031-yt-dlp-2022.08.19-noarch-1ncm.xzm
lrwxrwxrwx 1 root 72 2022-08-19 11:42 985-palemoon-settings--RECENT.xzm -> 985-palemoon-settings--RECENT--2022-08-19-Addons-Updates-Deactivated.xzm
lrwxrwxrwx 1 root 40 2021-11-23 09:32 991-usr_local_bin_Nemesis_RECENT.xzm -> 991-usr_local_bin_Nemesis_2021-11-23.xzm
lrwxrwxrwx 1 root 32 2022-08-29 00:57 991-usr_local_bin_RECENT.xzm -> 991-usr_local_bin_2022-08-29.xzm
lrwxrwxrwx 1 root 35 2021-09-23 00:22 992-rootcopy_5.0rc3a-RECENT.xzm -> 992-rootcopy_5.0rc3a_2021-09-23.xzm
lrwxrwxrwx 1 root 35 2022-08-29 00:57 992-rootcopy_5.0rc3c-RECENT.xzm -> 992-rootcopy_5.0rc3c_2022-08-29.xzm
lrwxrwxrwx 1 root 35 2021-11-23 09:31 992-rootcopy_Nemesis_RECENT.xzm -> 992-rootcopy_Nemesis_2021-11-23.xzm

Unlike what some names might suggest, 992-rootcopy_5.0rc3c-RECENT.xzm and 992-rootcopy_5.0rc3c_2022-08-29.xzm - as you can guess by its name part of "2022-08-29" is no 5.0rc3* module, but a 5.0 finale version one… I am just too lazy to update my make-991-usr_local_bin.sh and make-992-rootcopy.sh scripts [*] that create my most recent settings module on command and also do the symlink creation to that most recent module.
__________________
[*] as you can see here

Code: Select all

root@porteus:/usr/local/bin# l make-99*
-rwxr--r-- 1 root 2822 2021-11-16 03:39 make-991-usr_local_bin.sh
lrwxrwxrwx 1 root   25 2021-02-27 23:06 make-992-rootcopy.sh -> make-991-usr_local_bin.sh

make-991-usr_local_bin.sh is a script and make-992-rootcopy.sh is a symlink, and make-991-usr_local_bin.sh determines what to do according to the name it is executed as.

[Rava]

Now the only thing to do is deactivate my own made module and activate my RECENT module like so

Code: Select all

root@porteus:/mnt/live/memory/images# deactivate yt-dlp-2022.01.21-x86_64-1smt.xzm
Updating shared library links:  /sbin/ldconfig
root@porteus:/mnt/live/memory/images# activate /Porteus_modules/031-yt-dlp-RECENT.xzm 
Updating shared library links:  /sbin/ldconfig

and test yt-dlp

Code: Select all

guest@porteus:~$ yt-dlp 

Usage: yt-dlp [OPTIONS] URL [URL...]

yt-dlp: error: You must provide at least one URL.
Type yt-dlp --help to see a list of all options.
guest@porteus:~$ yt-dlp --version
2022.08.19

As always, you, Sir ncmprhnsbl are the best. <B

[Rava]

I forgot to add:

If you added the most recent youtube-dl module to your porteus/base/ remove that module and put the created like described above yt-dlp module in there instead.

[JTD]

Is there a reason for why the module is called "yt-dlp-2022.01.21-x86_64-1smt.xzm"? The one on GitHub is 2022.08.19, unless that is not related to the version naming scheme at all.

[Rava]

^
JTD, I do not follow, mine got called

The latest yt-dlp version is : 2022.08.19
[…]
Creating 4.0 filesystem on yt-dlp-2022.08.19-noarch-1ncm.xzm, block size 262144.
[…]
Your module yt-dlp-2022.08.19-noarch-1ncm.xzm is ready in /tmp

when creating the module using the latest GitHub code

The one on GitHub is 2022.08.19, unless that is not related to the version naming scheme at all.

Yes, it is, and indeed that is just what happens. As quoted above, the module that got created is named yt-dlp-2022.08.19-noarch-1ncm.xzm

And finally, the module name yt-dlp-2022.08.19-noarch-1ncm.xzm and yt-dlp --version match:

Code: Select all

guest@porteus:/$ yt-dlp --version
2022.08.19

update-youtube-dl-live works perfectly for me.

[JTD]

Ohhh I see! I only checked the commands on this thread and not the file on the server so I was wondering why some of these were named with that January date, for example:

Code: Select all

root@porteus:/Porteus_modules# l *yt-dlp*
-rw-r--r-- 1 guest 3678208 2022-08-29 17:34 031-yt-dlp-2022.01.21-x86_64-1smt.xzm
lrwxrwxrwx 1 root       37 2022-08-30 00:45 031-yt-dlp-RECENT.xzm -> 031-yt-dlp-2022.01.21-x86_64-1smt.xzm

I imagine you were still testing it out before pushing the real version (2022.08.19) to the server?

[Rava]

I imagine you were still testing it out before pushing the real version (2022.08.19) to the server?

Our amazing ncmprhnsbl codes update-browser, not me.
And while I tested it (could be I was the first one to test the uploaded-to-server version, though, since ncmprhnsbl put it live after my request, see posts above) there was no Porteus server version, so I went with the created from github version - again, see posts above.

With many of my modules in base/ that are getting updated regularly - palemoon, interlink, palemoon-settings, yt-dlp, and my 991-usr_local_bin_RECENT.xzm and 992-rootcopy_RECENT.xzm are all symlinks.

Since I now have the most current version and put it into my porteus/base/ as 031-yt-dlp-RECENT.xzm - which is a symlink to where most of my modules are (on a different drive, cause sda1 is only 1500 MB of size, not all my modules fit on there) the most recent version that sits on a different partition of the internal hard drive - so that I can have dozens of Porteus5.0-whatever-flavour folders and most files in porteus/base/ are symlinks that not need to be changed - since they link to another symlink on the target drive that is also called in this case 031-yt-dlp-RECENT.xzm - and that finally links to the most recent version.

So, even when I have a script that creates the most recent version of whatever module for me, I just have to update the symlink on the drive that holds the module and all symlinks in the dozens of sda1/Porteus5.0-whatever-flavour folders will always load the most recent version by themselves.

To illustrate it how its done maybe a [ code ] snippet works better than my non-native tongue attempt in explaining it verbally.

Code: Select all

root@porteus:/mnt/sda1/Porteus_5.0/porteus/base# l 031-yt-dlp-RECENT.xzm 
lrwxrwxrwx 1 root 51 2022-08-30 00:43 031-yt-dlp-RECENT.xzm -> /mnt/sda7/Porteus_modules/5.0/031-yt-dlp-RECENT.xzm
root@porteus:/mnt/sda1/Porteus_5.0rc3b/porteus/base# cd /mnt/sda7/Porteus_modules/5.0/
root@porteus:/mnt/sda7/Porteus_modules/5.0# l 031-yt-dlp-RECENT.xzm
lrwxrwxrwx 1 root 37 2022-08-30 06:53 031-yt-dlp-RECENT.xzm -> 031-yt-dlp-2022.08.19-noarch-1ncm.xzm
root@porteus:/mnt/sda7/Porteus_modules/5.0# l 031-yt-dlp-2022.08.19-noarch-1ncm.xzm
-rw-r--r-- 1 root 2433024 2022-08-30 06:50 031-yt-dlp-2022.08.19-noarch-1ncm.xzm

So, when updating to a new yt-dlp - say, theoretically there was a yt-dlp-2022.08.31-noarch-1ncm.xzm - all I have to do to make dozens of sda1/Porteus5.0-whatever-flavour folders use at next boot the most recent version (which then would be the theoretically 031-yt-dlp-2022.08.31-noarch-1ncm.xzm all there is to be done is doing this:

Code: Select all

root@porteus:/# cd /mnt/sda7/Porteus_modules/5.0/
root@porteus:/mnt/sda7/Porteus_modules/5.0# ln -sf 031-yt-dlp-2022.08.31-noarch-1ncm.xzm 031-yt-dlp-RECENT.xzm

The "f" parameter is important since the target symlink already exists and you want to forcefully change it to link to a newer version than the previously 031-yt-dlp-2022.08.19-noarch-1ncm.xzm

[Rava]

For the first time I got an warning by yt-dlp suggesting I install additional software.
I was downloading a review about "THE LORD OF THE RINGS： THE RINGS OF POWER (EPISODES 1 AND 2) REVIEW ｜ Double Toasted" - sorry the all CAPS shouting is the original title.

Code: Select all

WARNING: [youtube] yMXL4NiKJ0A: nsig extraction failed: You may experience throttling for some formats
         Install PhantomJS to workaPhantomJSround the issue
         n = 8TtXRVMfcSQxyW7AMA5yN ; player = https://www.youtube.com/s/player/f96f6702/player_ias.vflset/en_US/base.js

Anyone else ever encountered that warning?
Anyone installed PhantomJS and compared the non-PhantomJS video result with the PhantomJS result?

[beny]

hi Rava give a tube link to the movie