glast revisited: gh-down

For discussions about programming and projects not necessarily associated with Porteus.
User avatar
M. Eerie
Moderator
Moderator
Posts: 711
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

glast revisited: gh-down

Post#1 by M. Eerie » 09 Feb 2025, 19:48

Based on this oneliner, I have come up with a more complete solution.
This script tries to be an interactive and user-friendly solution for managing and downloading GitHub repository (latest) assets.

Beware:
  • It is a work in progress although it works well most of the time.
  • If you abuse Github by sending requests to the server, the Github API will ban you for a while.
HOW IT WORKS:
  1. First of all, the script relies on having fzf, jq and curl installed on your system.
  2. It creates a dedicated download directory (GH-DOWN) within the user's default downloads folder. (Change to suit your needs).
  3. Manages two local lists: .GH-REPOS and .GH-TOPICS. The former stores your favorite repositories, while the latter contains your dictionary of topic searches.
  4. From fzf window, you can switch to .GH-REPOS with Ctrl-r, while Ctrl-t is used to switch to .GH-TOPICS.
  5. From Topics list, you can discover new repos by typing any topic and ADD it (Ctrl-a) to the .GH-REPOS list. Then you can search for it (Enter) or REMOVE it (Ctrl-Del)
  6. Similarly, once a search for repositories matching the searched term has been launched, you can add them to the repos list using the shortcut Ctrl-a.
  7. In the repository list, you can delete the highlighted repository (Ctrl-Del) or search for its latest release to download (Enter).
  8. Finally, once the latest release assets are presented (if they exist), you can multi-select them and download them all at once. *

*In case it's an AppImage, its extension is normalized and it is marked as executable.

^^Tip: If you have some repos added to your list, RATHER THAN BROWSING FROM ONE TO ANOTHER, USE YOUR MOUSE to jump to a particular repo, avoiding sending unnecesary request to Github

Script code follows:

Code: Select all

#!/bin/bash
#
# gh-down: My GitHub latest release downloader
# v2025-01 M. Eerie --> forum.porteus.org
# ------------------------------------------------------------------------------
#set -x
# Helper functions to show messages with colors
error()   { printf "\e[1;31m%s\e[0m\n" "$@"; }
success() { printf "\e[1;32m%s\e[0m\n" "$@"; }
warn()    { printf "\e[1;33m%s\e[0m\n" "$@"; }
notify()  { printf "\e[1;34m%s\e[0m\n" "$@"; }
confirm() { read -n 1 -rsp "$(printf '\e[1;36m%s\e[0m' "$1 [y/*]: ")" && echo && [[ "${REPLY,,}" == y ]]; }

# ------------------------------------------------------------------------------
# Function to show progress with a green fill box
show_progress() {
    local total=$1      # Total size
    local progress=$2   # Current progress
    local width=50      # Width of the bar
    local green="\033[32m"  # Green color
    local reset="\033[0m"   # Reset
    local percentage=$(( (progress * 100) / total ))
    [[ "$percentage" -gt 100 ]] && percentage=100 # Prevent division by 0 TODO
    local blocks=$(( (width * progress) / total ))
    local completed=$(printf "%${blocks}s" | sed 's/ /█/g')
    local remaining=$(printf "%$((width - blocks))s" | sed 's/ /░/g')
    printf "\r[${green}${completed}${reset}${remaining}] %3d%%" $percentage
}

# Download directory
DEF_DOWNLOAD_DIR="$(xdg-user-dir DOWNLOAD)"
DOWNLOAD_DIR="${DEF_DOWNLOAD_DIR:-$HOME}/GH-DOWN"  # Adjust to suit your needs
mkdir -p "$DOWNLOAD_DIR"

# Update local repository and topic lists
_REPOS="$HOME/.GH-REPOS"
_TOPICS="$HOME/.GH-TOPICS"

update_list() {
    local list="$1"
    [[ ! -f "$list" ]] && touch "$list"
    awk '!seen[$0]++' "$list" > "$list.tmp" && mv "$list.tmp" "$list"
}

# ------------------------------------------------------------------------------
# Function to toggle between repository and topic lists using fzf
list_mode() {
    local MODE="${1:-repos}" FILE PREVIEW PROMPT HEADER OUTPUT KEY SELECTED
    MODE="${MODE##*-}"

    while true; do
      if [[ "$MODE" == "repos" ]]; then
            FILE="$_REPOS"
            PROMPT="Select Repository: "
            HEADER="=== REPOS ==="
            PREVIEW="echo -e '[ENTER: select] [CTRL-A: Add] [CTRL-DELETE: Delete] [CTRL-T: Topics]\n'; "
            PREVIEW+="curl -s https://api.github.com/repos/{} | jq -r '.description + \"\n\nLatest release: \" + (.updated_at | fromdate | strftime(\"%Y-%m-%d\"))'"
      else
            FILE="$_TOPICS"
            PROMPT="Select/Search new terms: "
            HEADER="=== TOPICS ==="
            PREVIEW="echo -e '[ENTER: select] [CTRL-A: Add] [CTRL-DELETE: Delete] [CTRL-T: Repositories]\n'"
      fi

      OUTPUT=$(fzf \
        --bind "ctrl-a:execute-silent(echo {q} >> $FILE && sort -u $FILE -o $FILE)+reload-sync(cat $FILE)" \
        --bind "ctrl-delete:execute-silent(sed -i 's_'"{}"'__; /^$/d' $FILE)+reload-sync(cat $FILE)" \
        --bind home:top \
        --bind end:last \
        --bind "del:clear-query" \
        --bind "?:toggle-preview" \
        --expect "ctrl-c,ctrl-r,ctrl-t,esc" \
        --prompt "$PROMPT" \
        --preview-window "down,wrap,25%" \
        --preview "$PREVIEW" \
        --header "$HEADER" \
        --reverse \
        < "$FILE")

      KEY=$(head -n1 <<< "$OUTPUT")
      SELECTED=$(tail -n +2 <<< "$OUTPUT")

      # Reset $SELECTED when changing mode
      if [[ "$KEY" == "ctrl-t" || "$KEY" == "ctrl-r" ]]; then
          SELECTED=""
      fi

      case "$KEY" in
          "ctrl-t") MODE="topics" ;;
          "ctrl-r") MODE="repos" ;;
          "ctrl-c") exit 1 ;; #TODO
             "esc") break ;;
      esac

      [[ -n "$SELECTED" ]] && case "$MODE" in
           repos) download_assets "$SELECTED" ;;
          topics) discover_repos "$SELECTED" ;;
               *) exit ;;
      esac

    done
}

urlencode() {
  # OLD echo ${1// /"%20"}
  echo $(jq -rn --arg str "$1" '$str|@uri')
}

# ------------------------------------------------------------------------------
# Function discover_repos: 󰣐 󱐮
# Runs from the topic list (or from the prompt) and searches
# repositories on GitHub using the selected term. If any are found,
# allows the user to select a repository and download its assets.
discover_repos() {
    local repos query="$1"
    repos=$(curl -s "https://api.github.com/search/repositories?q=$(urlencode "$query")&sort=stars&order=desc" \
              | jq -r '.items[].full_name' \
              | fzf --reverse \
                    --header="Results for: $query" \
                    --multi \
                    --marker='🟊 ' \
                    --bind "del:clear-query" \
		    --bind home:top \
	            --bind end:last \
                    --bind "ctrl-a:execute-silent(echo {+} | tr ' ' '\n' >> $_REPOS && sort -u $_REPOS -o $_REPOS)" \
                    --preview-window="down,wrap,25%" \
                    --preview="echo {} && curl -s https://api.github.com/repos/{} | jq -r '.description' && \
                                if grep -q '^{}$' $_REPOS; then echo -e '\033[32m[+]Added\033[0m'; fi")
    if [[ -z "$repos" ]]; then
        error "No repository found for the term: $query"
        return 1
    fi
    #sleep 2 # Limit requests to GitHub API
    download_assets "$repos"
}

# -------------------------------------------------------------------
# Function download_assets:
# Called with the full name of a repository. Gets the latest release
# and allows the user to select (with support for multiple items) the assets to download.
download_assets() {
    local repo="$1" response tag_name selected_assets asset_name asset_info asset_url asset_size temp_file app_name

    response=$(curl -s "https://api.github.com/repos/$repo/releases/latest")
    if [[ -z "$response" || "$(echo "$response" | jq -r '.message')" == "Not Found" ]]; then
        error "No release found for the repository '$repo'."
        return 1
    fi

    tag_name=$(echo "$response" | jq -r '.tag_name')
    selected_assets=$(echo "$response" | jq -r '.assets[]?.name' \
                      | fzf --multi --reverse --marker="󱐮 " --bind "del:clear-query" --header="[TAB] Select items to download (Release: $tag_name)")
    [[ -z "$selected_assets" ]] && return 0

    while IFS= read -r asset_name; do
        asset_info=$(echo "$response" | jq -r ".assets[] | select(.name == \"$asset_name\")")
        asset_url=$(echo "$asset_info" | jq -r '.browser_download_url')
        asset_size=$(echo "$asset_info" | jq -r '.size')
        notify "Downloading: $asset_name ($tag_name, $asset_size bytes)"
        temp_file=$(mktemp)
        (
            curl -L "$asset_url" --output "$temp_file" 2>/dev/null &
            local curl_pid=$!
            while kill -0 $curl_pid 2>/dev/null; do
                local downloaded
                downloaded=$(stat -c%s "$temp_file" 2>/dev/null || echo 0)
                show_progress "$asset_size" "$downloaded"
                sleep 0.1
            done
            show_progress "$asset_size" "$asset_size"  # Ensure full bar
        )
        printf "\n"
        mv "$temp_file" "$DOWNLOAD_DIR/$asset_name"

        # AppImage: Normalize extension and mark as executable
        extension="${asset_name##*.}"
        if [[ "${extension,,}" == "appimage" ]]; then
            app_name="${asset_name%.*}.AppImage"
            mv -f "$DOWNLOAD_DIR/$asset_name" "$DOWNLOAD_DIR/$app_name" 2>/dev/null
            chmod +x "$DOWNLOAD_DIR/$app_name"
        fi
        sleep 1 # Limit requests to GitHub API 403 Forbidden
    done <<< "$selected_assets"
    success "Downloads completed."
}

# ------------------------------------------------------------------------------
# Main function
main() {
    # Check necessary dependencies
    for dep in fzf jq curl; do
        command -v "$dep" >/dev/null 2>&1 || { error "Dependency '$dep' is not installed."; exit 1; }
    done
[[ $# -gt 1 ]] && { error "Usage: $0 [-r|--repos] [-t|--topics]"; exit 1; }
update_list "$_REPOS"
update_list "$_TOPICS"

case "$1" in
    -r|--repos) list_mode "repos" ;;
    -t|--topics) list_mode "topics" ;;
    *) list_mode ;;
esac

[[ $? -ne 0 ]] && warn "Aborted!!..." || success "Exiting..."
}

# Execute the script
main "$@"
***UPDATE Feb. 10: Translated to english version
Last edited by M. Eerie on 02 Mar 2025, 08:44, edited 10 times in total.
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=102066#p102066
https://forum.porteus.org/viewtopic.php?p=102306#p102306
https://forum.porteus.org/viewtopic.php?p=72741#p72741

User avatar
M. Eerie
Moderator
Moderator
Posts: 711
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

glast revisited: gh-down

Post#2 by M. Eerie » 09 Feb 2025, 19:53

This is a working demo:


Image
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=102066#p102066
https://forum.porteus.org/viewtopic.php?p=102306#p102306
https://forum.porteus.org/viewtopic.php?p=72741#p72741

User avatar
M. Eerie
Moderator
Moderator
Posts: 711
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

glast revisited: gh-down

Post#3 by M. Eerie » 10 Feb 2025, 09:23

Translated script to english as last night (local time) I was in a hurry. :)
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=102066#p102066
https://forum.porteus.org/viewtopic.php?p=102306#p102306
https://forum.porteus.org/viewtopic.php?p=72741#p72741

User avatar
M. Eerie
Moderator
Moderator
Posts: 711
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

glast revisited: gh-down

Post#4 by M. Eerie » 10 Feb 2025, 09:34

For those interested, below I share my local lists:

.GH-REPOS
Note: Not all repos listed here contain downloadable items... I just add them for convenience (to keep track of them)

Code: Select all

0ad-matters/0ad-appimage
4gray/iptvnator
ajeetdsouza/zoxide
alyssaxuu/screenity
AppImage/AppImageKit
AppImage/pkg2appimage
arduino/Arduino
aristocratos/btop
arrowtype/recursive
bash-suite/loader
BootcampXperience/crea_audiolibro
BurntSushi/ripgrep
cormullion/juliamono
creio/dots
doublecmd/doublecmd
DrewThomasson/ebook2audiobook
dylanaraps/pure-bash-bible
endeavouros-team/repo
Ferran226/text-to-speech-python
filebrowser/filebrowser
flameshot-org/flameshot
fontforge/fontforge
FreeTubeApp/FreeTube
ful1e5/Bibata_Cursor
hanzala123/arch2appimage
imagemagick/imagemagick
iptv-org/awesome-iptv
ivan-hc/KDE-games-suite-appimage
ivan-hc/MPV-appimage
ivan-hc/Spotify-appimage
jarun/ddgr
jarun/nnn
jcarolinares/thingiverse_downloader
jlevy/the-art-of-command-line
joshuah345/linux-dotfiles
keepassxreboot/keepassxc
kovidgoyal/kitty
Kron4ek/Conty
ksnip/ksnip
LeCoupa/awesome-cheatsheets
lettier/gifcurry
lichess-org/fishnet
lichess-org/lila
LunarVim/LunarVim
mifi/lossless-cut
ml-research/liground
nelsonenzo/tmux-appimage
neovim/neovim
notepad-plus-plus/notepad-plus-plus
NvChad/NvChad
peazip/PeaZip
porteux/porteux
qarmin/czkawka
ryanoasis/nerd-fonts
sharkdp/bat
sharkdp/fd
SpaceVim/SpaceVim
srevinsaju/Brave-AppImage
srevinsaju/zap
starship/starship
sxyazi/yazi
Syllo/nvtop
telegramdesktop/tdesktop
tonsky/FiraCode
ToxicFrog/Ligaturizer
ungoogled-software/ungoogled-chromium-portablelinux
vapoursynth/vapoursynth
vim/vim-appimage
vinifmor/bauh
VSCodium/vscodium
webpro/awesome-dotfiles
Y2Z/monolith
zen-browser/desktop
.GH-TOPICS

Code: Select all

appimage
arduino
audiolibro
bash
blender
chess
dotfiles
dron
ffmpeg
geany
github
iptv
mpv
neovim
notepad-plus-plus
portable
radio
raspberry
spacevim
telegram
themes
thingiverse
tmux
tts
tts linux
vapoursynth
whatsapp
xfce4
xfce4 dotfiles
xfce4 themes
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=102066#p102066
https://forum.porteus.org/viewtopic.php?p=102306#p102306
https://forum.porteus.org/viewtopic.php?p=72741#p72741

vinnie
Shogun
Shogun
Posts: 210
Joined: 13 Jun 2024, 08:25
Distribution: alpine

glast revisited: gh-down

Post#5 by vinnie » 10 Feb 2025, 15:48

Ah this may be useful because I use a lot of appimages and even if I don't use them often it wouldn't hurt to keep an eye on them all.
Here is an incomplete list of appimages I have.

Code: Select all

find -iname "*.appimage"  -print0 | xargs -0P 0 -n1 -- basename | sort -f
20221214_96744b30_Linux_welle-io-x86_64.AppImage
AisleRiot-Solitaire_3.22.33-1-archimage3.4.4-2-x86_64.AppImage
Altus-5.4.0.AppImage
balenaEtcher-1.19.21-x64.AppImage
benben-0.6.2-x86_64.AppImage
Chawan-x86_64-6dfcadb.AppImage
Chromium-stable-131.0.6778.85-x86_64.AppImage
DeerPortal-0.8.1-x86_64.AppImage
DeltaChat-1.46.5.AppImage
DeltaChat-1.46.8.AppImage
digiKam-8.5.0-Qt6-x86-64.appimage
dmidiplayer-1.7.5-x86_64.AppImage
Document-Viewer_46.3.1-2-archimage3.4.4-2-x86_64.AppImage
Drawpile-2.2.2-beta.4-x86_64.AppImage
firefox-devedition-129.0.r20240722091823-x86_64.AppImage
freetube_0.22.1_amd64.AppImage
HandBrake_1.9.0-1-archimage3.4.4-2.1-x86_64.AppImage
HexWalk_1.8.0.AppImage
Hypnotix_4.9-1-archimage4.3-x86_64.AppImage
i2p.AppImage
Inkscape-091e20e-x86_64.AppImage
KDiskMark-3.1.4-fio-3.35-x86_64.AppImage
Koodo-Reader-1.7.4-x86_64.AppImage
LibreOffice-fresh.basic-x86_64.AppImage
linux-Miru-5.2.3.AppImage
mediainfo-20.09.glibc2.3-x86_64.AppImage
midi123-2.2.0-x86_64.AppImage
minimetro-gamma26-X86_64.appimage
MKVToolNix_GUI-89.0-x86_64.AppImage
mldonkey-next-v0.9.0.AppImage
MuseScore-Studio-4.4.4.243461245-x86_64.AppImage
novelWriter-2.5.3.AppImage
openxcom_git_2024_06_25_0105-x86_64.AppImage
pavucontrol-qt-2.1.0-1-anylinux-x86_64.AppImage
Pinta-x86-64.AppImage
PulseAudio_Volume_Control-23-x86_64.AppImage
qbittorrent-4.6.7_x86_64.AppImage
qpdf-11.9.1-x86_64.AppImage
Qt_DAB-6.X-x86_64.AppImage
Qt_DAB-6.X-x86_64.AppImage
RetroShare-0.6.7-2.glibc2.29-x86_64.AppImage
rustdesk-1.2.7-x86_64.AppImage
sayonara-1.10.0-stable1-g57fab52c.AppImage
session-desktop-linux-x86_64-1.13.2.AppImage
shotcut-linux-x86_64-250125.AppImage
simplex-desktop-x86_64.AppImage
SpeedCrunch-0.12-x86_64.AppImage
Spivak-Host-GStreamer-x86_64.AppImage
subtitlecomposer-git-latest-x86_64.AppImage
Synfig_Studio-x86_64.AppImage
Telegram-A-x86_64_10.9.7.AppImage
Telegram_Desktop-x86_64.AppImage
ungoogled-chromium_126.0.6478.126-1.AppImage
ventoy-1.0.99-x86_64.AppImage
Veracrypt-1.26.18-gui-x86_64.AppImage
VirtualBox-KVM-7.0.20-x86_64.AppImage
VLC-media-player_3.0.21-12-archimage4.3-x86_64.AppImage
vmm.AppImage
YTDownloader_Linux.AppImage
Zoom_Workplace-6.1.11.1545.glibc2.27-x86_64.AppImage

By the way, I actually didn't understand the script too much because it is complex for me and not so much for Spanish porque lo entiendo un poquito, todavia I understand a little bit more english :)
Nemesis + xfce rulez :punk:

Post Reply