repository for xzm modules

Post#1 by **rych** » 12 Aug 2023, 09:51

@ncmprhnsbl, and others: could user supplied xzm modules be hosted in a directory under http://porteus.org/porteus-mirrors.html allowing module providers to upload? Then possibly indexing the modules at https://pkgs.org/ or another aggregator -- at the moment Porteus isn't even listed

Post#2 by **Ed_P** » 12 Aug 2023, 23:35

I'm not in favor of that idea but a webpage on the porteus-mirrors.html with links to the Porteus forum topics about user xzm modules would be useful imho.

Post#3 by **roro** » 13 Aug 2023, 07:49

Hi rych,
that's a good idea
Currently you can find xzm modules here and there and everywhere
Very tedious if you are looking for a very specific module

Post#4 by **Ed_P** » 13 Aug 2023, 14:09

When looking for modules here I find this works with Google:
site:forum.porteus.org modulename.xzm

Post#5 by **vaguiners** » 18 Aug 2023, 16:32

Is Porteus already illegal by not hosting the source code of all programs and libraries present, which is an obligation in 90% of open source licenses? A greater tragedy would be if users could upload anything, including proprietary software, it would surely become a den of piracy.

Post#6 by **Rava** » 27 Aug 2023, 07:28

rych wrote: ↑
12 Aug 2023, 09:51
@ncmprhnsbl, and others: could user supplied xzm modules be hosted in a directory under http://porteus.org/porteus-mirrors.html

Most people would not know, but porteus-mirrors.html is just created by a very simple bash script by yours truly that I gave to one admin years ago. Prior to that it was just a text file with no links (called porteus-mirrors.txt - that is also still online: http://porteus.org/porteus-mirrors.txt )
For now I do not find the script locally

but I recall parts of this code:

Code: Select all

<html><head><title>porteus-mirrors.txt - created on 2018-02-19</title>
<style type="text/css">body { color:#6f3; background-color:#000;} a:link { color:#66f; text-decoration:none; font-weight:bold; } a:visited { color:#a6d; text-decoration:none; font-weight:bold; } a:focus, a:hover { color:#f33; text-decoration:none; background-color:#fff; font-weight:bold; } a:active { color:#fff; background-color:#000; font-weight:bold; } </style>

When I recall correctly, it simply looks for lines beginning with

Code: Select all

http://
https://
ftp://

and if a line starts like this, it will convert that line into a link.
If the line starts any other way, the line stays as it is.

It is a very simple script and sure not very foolproof, but when you give it a sane input file it does its job well enough.

But that has nothing to do with hat OP is asking. Just some background info how porteus-mirrors.html gets created.

Post#7 by **rych** » 15 Oct 2023, 09:03

roro wrote: ↑
13 Aug 2023, 07:49
that's a good idea
Currently you can find xzm modules here and there and everywhere

There is now the Porteus eXchange Telegram group:
https://t.me/PorteusX

...where we can exchange our modules, in appropriate subforums which in Telegram are called "topics". Telegram will host our modules for free indefinitely with unspecified limits (limitless). Telegram has a nice Telegram client for Windows and Linux.

Post#8 by **Vic** » 15 Oct 2023, 17:33

Hi rych.

Great idea.

Telegram seem to only be for a phone unless I missed something, as usual.

I do not have a phone.

Vic

Post#9 by **Rava** » 15 Oct 2023, 18:25

Vic wrote: ↑
15 Oct 2023, 17:33
Telegram seem to only be for a phone unless I missed something, as usual.

You can run the desktop version, but that needs your phone Telegram app to sent you its confirmation so you can log into you account using the desktop version.

Sounds Kafkaesque ? Pretty much.

So yeah. I created an extra thread for that :
Telegram questions

Post#10 by **Vic** » 16 Oct 2023, 01:04

Thanks Rava. I saw your thread after I posted.

Also thanks because now I know who Franz Kafka is.

Vic

Post#11 by **rych** » 23 Oct 2024, 11:18

rych wrote: ↑
15 Oct 2023, 09:03
There is now the Porteus eXchange Telegram group:
https://t.me/PorteusX

Apparently some people cannot or do not wish to use the great Telegram platform, with its free storage, great Desktop client etc.

I've come up with a solution where I can upload all "my modules", i.e. the modules I use, to github using LFS for binary modules storage. It's available under my public repository https://github.com/rychphd/Porteus/tree/modules.

"modules" is the name of the branch I created on my repository:

Code: Select all

root@porteus:/mnt/sda5/porteus/modules# git switch modules
Already on 'modules'

I've already done all the setup, so I just begin adding/updating modules as follows:

Adding all modules, nearly 2GB, verbose:

Code: Select all

root@porteus:/mnt/sda5/porteus/modules# git add . --verbose
add '005-scim-anthy-22.08.10-x86_64.xzm'
add '011-slapt-get-0.11.6-x86_64-2gv-zstd.xzm'
add 'CursorThemes/Bibata-Original-Classic.xzm'
add 'DependencyCheckers/elfutils-0.186-x86_64-1.xzm'
add 'DependencyCheckers/pax-utils-1.3.3-1-x86_64.xzm'
add 'DependencyCheckers/python-pyelftools-0.27-3.xzm'
add 'Fonts/004-IPAfont-00303-noarch-1jay.xzm'
add 'Fonts/HanaMin.xzm'
add 'Fonts/HinaMincho.xzm'
add 'Fonts/HinaMinchoGoogle1.1.xzm'
add 'Fonts/HiraginoMinchoGothicProN.xzm'
add 'Fonts/IPAmjMincho.xzm'
add 'Fonts/Kozuka-Gothic-Pr6N.xzm'
add 'Fonts/Kozuka-Mincho-Pr6N.xzm'
add 'Fonts/MPLUS1.xzm'
add 'Fonts/MPLUS1Code.xzm'
add 'Fonts/MPLUS2.xzm'
add 'Fonts/MPLUSCodeLatin.xzm'
add 'Fonts/adobe-caslon-pro.xzm'
add 'Fonts/adobe-source-han-sans-jp-fonts-2.002-2.fc34.noarch.xzm'
add 'Fonts/noto-fonts.xzm'
add 'Fonts/odosung-font-1.4.2-noarch-1_Ahau.xzm'
add 'Fonts/sazanami-fonts-ttf-20040629-noarch-5.xzm'
add 'GPU/intel-gmmlib-21.3.1-x86_64-1salix15.0.xzm'
add 'GPU/intel-media-driver-21.3.5-x86_64-1salix15.0.xzm'
add 'GPU/libva-utils-2.13.0-x86_64-1.xzm'
add 'Git/git-2.46.2-lfs-3.5.1-x86_64-15.0.xzm'
add 'Inkscape1.2.xzm'
add 'LyX/aspell-0.60.8-x86_64-3.xzm'
add 'LyX/aspell-en-2019.10.06_0-x86_64-1.xzm'
add 'LyX/enchant-2.3.0-x86_64-1.xzm'
add 'LyX/lyx-2.3.7-x86_64-1cf.xzm'
add 'LyX/mythes-1.2.4-x86_64-2_slonly.xzm'
add 'LyX/mythes-dict-en.xzm'
add 'Musescore/jack2-1.9.19-x86_64-1alien.xzm'
add 'PDF/unpaper-7.0.github.xzm'
add 'SpeechDispatcher/dotconf-1.3-x86_64-3.xzm'
add 'TexLive/tk-8.6.12-x86_64-1.xzm'
add 'Texstudio/Convert latest ubuntu22 deb from here.html'
add 'Texstudio/hunspell-1.7.0-x86_64-1.xzm'
add 'Texstudio/libquazip5-1_0.9.1-x86_64.xzm'
add 'Texstudio/texstudio_4.8.5-0~202410101624~ubuntu22.04.1_amd64.xzm'
add 'Typora/pandoc-3.3-amd64.xzm'
add 'Typora/typora_1.9.3_amd64.xzm'
add 'UFSD-utils-from-Root-Mounter-for-Android-by-Paragon-2017-2024-04-15_tm.xzm'
add 'Wine/0050-multilib-lite-current-20231214.xzm'
add 'Wine/wine-9.0-x86_64-1sg.xzm'
add 'btop-static-1.2.0-x86_64.xzm'
add 'djview/djview-4.12-2-x86_64.xzm'
add 'djview/djvulibre-3.5.28-x86_64-3.xzm'
add 'drawing-1.0.2-no-help-no-locale-2023-05-04_tm.xzm'
add 'drawio-amd64-21.5.1.xzm'
add 'evince-3.36.5.xzm'
add 'fsearch-0.3.alpha.xzm'
add 'fzf-0.27.2-1-x86_64.xzm'
add 'gnome-disk-utility-42.0-x86_64-1_SBo.xzm'
add 'grsync-1.3.0.x86_64.xzm'
add 'gtksourceview4-4.8.2-x86_64-1gv.xzm'
add 'hardinfo-20230302-x86_64-1.xzm'
add 'i3slkiller/(2) i3slkiller's modules (Porteus 5.01) - Page 2 - Porteus.html'
add 'i3slkiller/gimp-2.10.36-porteus5.01-x86_64-3-i3slkiller.xzm'
add 'i3slkiller/gthumb-3.12.4-porteus5.01-x86_64-5-i3slkiller.xzm'
add 'i3slkiller/imageformats-12-porteus5.01-x86_64-1-i3slkiller.xzm'
add 'i3slkiller/kf5libs-5.90.0-porteus5.01-x86_64-1-i3slkiller.xzm'
add 'i3slkiller/krita-5.2.2-porteus5.01-x86_64-2-i3slkiller.xzm'
add 'i3slkiller/qownnotes-24.1.1-porteus5.01-x86_64-1-i3slkiller.xzm'
add 'i3slkiller/wireshark-4.2.2-porteus5.01-x86_64-1-i3slkiller.xzm'
add 'imagemagick-7.1.0_22-x86_64-1.xzm'
add 'jq-1.6-x86_64-2salix15.0.xzm'
add 'krop-0.6.1_e96d42b-if-kde-2024-01-04_tm.xzm'
add 'lshw-2.19.x86_64.xzm'
add 'marktext(markdown)-amd64.xzm'
add 'meld3-3.22.0-x86_64-1_SBo-2023-08-23.xzm'
add 'mypaint-2.0.1-x86_64-1salix15.0.xzm'
add 'pdfarranger-1.8.2-mcomix-2.1.0.x86_64.xzm'
add 'qpdfview-qt5-0.5.0-x86_64-openbox.xzm'
add 'qt5-base-5.15.3_20220318-x86_64-xfce-cinnamon-openbox-lxde-mate-gnome.xzm'
add 'qt5-extralib-5.15.3_20220318-x86_64.xzm'
add 'qtfm-6.2.0-x86_64-1_ncm.xzm'
add 'rclone-v1.62.2.x86_64.xzm'
add 'sane-1.0.32-x86_64-1.xzm'
add 'smartmontools-linux-x86_64-static-7.5-r5579.xzm'
add 'speech-dispatcher-0.10.2-x86_64-5.xzm'
add 'sublime-text-v4.4180-x86_64.xzm'
add 'syncthingtray-1.6.2_nowebengine-porteus5.01-x86_64-1-i3slkiller.xzm'
add 'vlc-3.0.20-x86_64-1-fixed.xzm'
add 'xdpyinfo-1.3.2-x86_64-4.xzm'
add 'xournalpp/portaudio-19.7.0-x86_64-2gv.xzm'
add 'xournalpp/xournalpp-1.1.3.xzm'
add 'xpdf-4.05-x86_64-1_slack15.0.xzm'
add 'zoom-5.14.10.3738.xzm'
add 'zz-changes-exit-test.xzm'

Commit as per git procedure:

Code: Select all

root@porteus:/mnt/sda5/porteus/modules# git commit --allow-empty --allow-empty-message -m ''
[modules 9b31eb0] 
 92 files changed, 272 insertions(+)
...

And finally push (upload/update):

Code: Select all

root@porteus:/mnt/sda5/porteus/modules# git push
Enumerating objects: 111, done./90), 1.9 GB | 11 MB/s                           
Counting objects: 100% (111/111), done.
Delta compression using up to 4 threads
Compressing objects: 100% (110/110), done.
Writing objects: 100% (110/110), 15.82 KiB | 2.64 MiB/s, done.
Total 110 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
To https://github.com/rychphd/Porteus.git
   309cdf0..9b31eb0  modules -> modules

The only problem is the huge (1.7GB) additional /mnt/sda5/porteus/modules/.git/lfs/objects folder. LFS developers insist for keeping a copy of the same xzm files inside that folder (unless you use an exotic copy-on-write FS like btfrs -- of course noone is going to switch from ext4 for that). So, I suggest that for our purposes one can simply delete that folder!?! Which I did without any apparent effect so far.

That's it for the upload/update workflow. Everyone can upload their entire module collection.

Post#12 by **rych** » 23 Oct 2024, 11:26

I'll describe how to set it up like that for sharing your own modules later, if there is interest. But for now let me teach you how you can download "my modules". Firstly, of course, by simply downloading individual modules from https://github.com/rychphd/Porteus/tree/modules

But also by using a git one-liner checkout, for example:

Code: Select all

git clone -b modules https://github.com/rychphd/Porteus.git --depth 1 modules

-- an entire modules collection will be downloaded into the new subfolder "modules"

Could someone please test and tell me if it works? In the end don't forget to delete the stupid huge duplicate modules/.git/lfs/objects folder.

Post#13 by **bytie** » 23 Oct 2024, 12:13

Code: Select all

guest@porteus /tmp/modules $ git lfs fetch --all
fetch: 90 objects found, done.                                                                                                                                                                     
fetch: Fetching all references...
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.                                                   
error: failed to fetch some objects from 'https://github.com/rychphd/Porteus.git/info/lfs'

Post#14 by **rych** » 24 Oct 2024, 04:26

bytie, thanks, indeed github limit turned out to be 1GB. Either I move it to GitLab which has 5GB limit, or to MEGA which has 50GB quota! Thanks for checking anyway, it was a good try, but maybe indeed git is a wrong infrastructure for the large xzm repository... I'll make a public MEGA share maybe.