USM bug reports

[brokenman]

Bogomips can you point me to the latest version of your file?

[Ed_P]

^ I believe the only version is posted at the bottom of this posting: viewtopic.php?f=140&t=6966&p=55568#p55562

[port]

I agree bash is really fast (I could even say superfast) but my idea of changing to another "real language" is not looking for increasing speed but increasing robustness, I'm thinking about improving quality rather than improving performance. Now you can say, ok and what's quality for you? well, my thinking is a "real language" will provide a higher abstraction to deal with real issues (objects, strategies and so on) and a cleaner and fail-safe programming style.

Quality for me is a robust, well thought out design.And I have every faith in brokenman having done a robust design. With a robust design it's easy to deal with exceptions, especially unforeseen exception conditions.

just a fast reply to say I have never criticized brokenman's job or capability by no means, I think he's a very good programmer... just talking about bash programming in general and weak points in bash programming. I think usm could be a little more robust in the way it is your own code or in the assumptions it make or trust on.

[brokenman]

Thanks Ed_P. It just that when I do a diff against this file there are hundreds of changes.

@port I agree that USM could be more robust. It could certainly handle errors better. I don't think that the choice of language makes it any less robust though. This is probably more down to my poor coding and lack of planning. I would love to see it done in python or C if anyone has the time.

[Ed_P]

Thanks Ed_P. It just that when I do a diff against this file there are hundreds of changes.

Hopefully this helps.

Code: Select all

funcpackageGet changes

AFTER LINE 31: }; export -f silent_check_update
ADD

title()         # 2015-04-23 21:09:32
{
  local t; local l; local u;
  [[ ! $eqs ]] &&
  eqs="============================================================"
  l=${#1}; u=${eqs:0:l}; t="$'\n'$u$'\n'${1:0:l}$'\n'$u$'\n'"
  eval echo $t
}

MANY ## LINES ARE DELETED. LINES 74, 106, 159, 173, 200, 213, 216, 226, 229, 231, 235, 242, 255, 258, 284, 304, 
321, 323, 340, 382, 392, 485, 492, 493, 501, 547, 548

AFTER LINE 86:  [ "$ARCH" = "x86_64" ] && ARCHI="x86_64" || ARCHI="x86" # Make an exception for alien repo
REPLACE LINES 87-89 WITH:
# Bogomips 30-5-17
title "HEURISTIC USM"; dbg=0;
for ((i=0; i<${#PKGLIST[*]}; i++)); do  pkg=${PKGLIST[i]};
#for pkg in ${PKGLIST[@]}; do
   local LONG=`find -L $DBDIR -type f -name "OPTIFEST"|xargs grep -i "Package: $pkg"`
((dbg)) && echo "pkg='$pkg' LONG='$LONG'"
   [[ $LONG ]] || {
   # Heuristic fix Dependent upon Tag Id or Lack of, for picking Distro
    #   REGEX DISTINGUISHING SUFFIX
    s=${pkg%-*-*-*}; [[ ${pkg#$s} =~ ^-.*-.*-[0-9]+(.+).t[xg]z$ ]] && c=${BASH_REMATCH[1]} || c=[0-9];
   # GREP
    local LONG=$(find -L $DBDIR -type f -name "OPTIFEST" | xargs grep -i  "Package: ${pkg%-*-*-*}-[^-]*-[^-]*-[0-9]*$c.t[xg]z$")
((dbg)) && echo "LONG='$LONG' pkg='${LONG##* }'  DBDIR='$DBDIR' c='$c'"
    pkg=${LONG##* }; echo -e ${PKGLIST[i]} "\033[1msuperseded\033[0m" by $pkg; PKGLIST[i]=$pkg;
    }
   local DIST=`cut -d':' -f1 <<<$LONG|awk -F/ '{print $(NF-1)}'`
((dbg)) && echo "PKGLIST[$i]='${PKGLIST[i]}'";

AFTER LINE 364: resolve_deps(){
ADD:
# Debugging Switch      Bogomips
dbg=0;

REPLACE LINES 388-389 WITH:
   # Bogonips 31-5-17
   # Making two assumptions for Newer Version
   # 1. Tag Id stays the same     2. Dependencies remain Unchanged
   e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1);
((dbg)) && echo "PKG='$PKG' e='$e' DBDIR='$DBDIR'"
    [[ $e ]] || {
    #   REGEX DISTINGUISHING SUFFIX
    s=${PKG%-*-*-*}; [[ ${PKG#$s} =~ ^-.*-.*-[0-9]+(.+).t[xg]z$ ]] && c=${BASH_REMATCH[1]} || c=[0-9];
   # GREP
    e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^${PKG%-*-*-*}-[^-]*-[^-]*-[0-9]*$c.t[xg]z$" | head -n1);
((dbg)) && echo "s='$s' c='$c' e='$e'"
    }
    TARGLIB=${e%%:*};
#   TARGLIB=`find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1 | awk -F: '{print$1}'`

[Bogomips]

^ Thanks Ed.

@ brokenman

Edit of funcpackageGet

1. Replace 31 with ed0
2. Replace 86-88 with ed1
3. After 345 insert ed2
4. Replace 368 with ed3

Confirmation

Did not use SELECT ALL to copy from post, but instead highlighted with mouse in order to verify the copy. Had trouble doing it with sed for the diff. so did it with arithmetic instead.

Code: Select all

root@porteus:/home/guest# f=funcpackageGet    
root@porteus:/home/guest# diff  funcpackageGet.hic   <(cat <(head -n30 $f) ed0 <(tail -n+32 $f | head -n54) ed1 <(tail -n+89 $f | head -n257) ed2  <(tail -n+346 $f | head -n22) ed3 <(tail -n+369 $f))
31,32c31
< 
< title()                       # 2015-04-23 21:09:32
---
> title()         # 2015-04-23 21:09:32
40d38
< 
99c97
<       local LONG=`find -L $DBDIR -type f -name "OPTIFEST"|xargs grep -i "Package: $pkg"`
---
>    local LONG=`find -L $DBDIR -type f -name "OPTIFEST"|xargs grep -i "Package: $pkg"`
101,102c99,100
<       [[ $LONG ]] || {
<       # Heuristic fix Dependent upon Tag Id or Lack of, for picking Distro
---
>    [[ $LONG ]] || {
>    # Heuristic fix Dependent upon Tag Id or Lack of, for picking Distro
105c103
<       # GREP
---
>    # GREP
110c108
<       local DIST=`cut -d':' -f1 <<<$LONG|awk -F/ '{print $(NF-1)}'`
---
>    local DIST=`cut -d':' -f1 <<<$LONG|awk -F/ '{print $(NF-1)}'`
205c203
<               if [[ $REPLY =~ ^[Rr]$ ]]; then
---
>               if [[ $REPLY =~ ^[Rr]$ ]]; then 
341c339
< 
---
> 
389c387
<               [ $dloop -eq $dlimit ] && { echo "Depth $dlimit of resolution reached. Exiting."; cleanup; }
---
>               [ $dloop -eq $dlimit ] && { echo "Depth $dlimit of resolution reached. Exiting."; cleanup; } 
393,396c391,394
<       # Bogonips 31-5-17
<       # Making two assumptions for Newer Version
<       # 1. Tag Id stays the same     2. Dependencies remain Unchanged
<       e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1);
---
>    # Bogonips 31-5-17
>    # Making two assumptions for Newer Version
>    # 1. Tag Id stays the same     2. Dependencies remain Unchanged
>    e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1);
401c399
<       # GREP
---
>    # GREP
406c404
< #     TARGLIB=`find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1 | awk -F: '{print$1}'`
---
> #   TARGLIB=`find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1 | awk -F: '{print$1}'`
425c423
<
---
>
432c430
<
---
>
492c490
<
---
>
538c536
<
---
>
     

Edits

• ed0

  Code: Select all

  title()			# 2015-04-23 21:09:32
  {
    local t; local l; local u;
    [[ ! $eqs ]] &&
    eqs="============================================================"
    l=${#1}; u=${eqs:0:l}; t="$'\n'$u$'\n'${1:0:l}$'\n'$u$'\n'"
    eval echo $t
  }

• ed1

  Code: Select all

  # Bogomips 30-5-17
  title "HEURISTIC USM"; dbg=0;
  for ((i=0; i<${#PKGLIST[*]}; i++)); do  pkg=${PKGLIST[i]};
  #for pkg in ${PKGLIST[@]}; do
  	local LONG=`find -L $DBDIR -type f -name "OPTIFEST"|xargs grep -i "Package: $pkg"`
  ((dbg)) && echo "pkg='$pkg' LONG='$LONG'"
  	[[ $LONG ]] || {
  	# Heuristic fix Dependent upon Tag Id or Lack of, for picking Distro
      #   REGEX DISTINGUISHING SUFFIX
      s=${pkg%-*-*-*}; [[ ${pkg#$s} =~ ^-.*-.*-[0-9]+(.+).t[xg]z$ ]] && c=${BASH_REMATCH[1]} || c=[0-9];
  	# GREP
      local LONG=$(find -L $DBDIR -type f -name "OPTIFEST" | xargs grep -i  "Package: ${pkg%-*-*-*}-[^-]*-[^-]*-[0-9]*$c.t[xg]z$")
  ((dbg)) && echo "LONG='$LONG' pkg='${LONG##* }'  DBDIR='$DBDIR' c='$c'"
      pkg=${LONG##* }; echo -e ${PKGLIST[i]} "\033[1msuperseded\033[0m" by $pkg; PKGLIST[i]=$pkg;
      }
  	local DIST=`cut -d':' -f1 <<<$LONG|awk -F/ '{print $(NF-1)}'`
  ((dbg)) && echo "PKGLIST[$i]='${PKGLIST[i]}'";

• ed2

  Code: Select all

  # Debugging Switch      Bogomips
  dbg=0;

• ed3

  Code: Select all

  	# Bogonips 31-5-17
  	# Making two assumptions for Newer Version
  	# 1. Tag Id stays the same     2. Dependencies remain Unchanged
  	e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1);
  ((dbg)) && echo "PKG='$PKG' e='$e' DBDIR='$DBDIR'"
      [[ $e ]] || {
      #   REGEX DISTINGUISHING SUFFIX
      s=${PKG%-*-*-*}; [[ ${PKG#$s} =~ ^-.*-.*-[0-9]+(.+).t[xg]z$ ]] && c=${BASH_REMATCH[1]} || c=[0-9];
  	# GREP
      e=$(find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^${PKG%-*-*-*}-[^-]*-[^-]*-[0-9]*$c.t[xg]z$" | head -n1);
  ((dbg)) && echo "s='$s' c='$c' e='$e'"
      }
      TARGLIB=${e%%:*};
  #	TARGLIB=`find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1 | awk -F: '{print$1}'`

Edit Update Tue Jul 18 20:19:01 UTC 2017

Edit of funcpackageGet

1. Replace 31 with ed0

   Code: Select all

   guest@porteus:~$ sed -n "30,32p" /usr/share/usm/funcpackageGet 
   }; export -f silent_check_update
   
   package_get(){

2. Replace 86-88 with ed1

   Code: Select all

   guest@porteus:~$ sed -n "85,89p" /usr/share/usm/funcpackageGet
   [ "$ARCH" = "x86_64" ] && ARCHI="x86_64" || ARCHI="x86" # Make an exception for alien repo
   for pkg in ${PKGLIST[@]}; do
           local LONG=`find -L $DBDIR -type f -name "OPTIFEST"|xargs grep -i "Package: $pkg"`
           local DIST=`cut -d':' -f1 <<<$LONG|awk -F/ '{print $(NF-1)}'`
           local PKGSHORT=${pkg%-*-*-*}

3. After 345 insert ed2

   Code: Select all

   guest@porteus:~$ sed -n "345,346p" /usr/share/usm/funcpackageGet
   resolve_deps(){
   PKGQ+=( "${1##*/}" )

4. Replace 368 with ed3

   Code: Select all

   guest@porteus:~$ sed -n "367,369p" /usr/share/usm/funcpackageGet
           sleep 2
           TARGLIB=`find -L $DBDIR -type f -name "LIBS.TXT" | xargs grep -i "^$PKG" | head -n1 | awk -F: '{print$1}'`
           [ $CLI ] && [ -z $TARGLIB ] && fatality "$SCRIPT $FUNCNAME $LINENO" "`gettext "Please wait 6 hours for the next database update. Could not find:"`" "LIBS.TXT"

[Ed_P]

I think one of the problems using forum code posts is the forum's software tends to shrink leading blanks in lines. If you look at some of the diff's output the difference between the lines is one is indented more than the other. Code wise they are identical.

I don't understand the difference with your line numbers and mine though. I used the file from 3.2.2 what did you use?

[Bogomips]

^ Thanks for pointing this out, Ed.

Code: Select all

root@porteus:/home/guest# usm -v
 You are using USM version:  3.2.0

Even though using usm 3.2.0, the number of comment lines seem to vary depending on where from it is got. Edited post to be unambigious.

[Ed_P]

Me too.

Code: Select all

guest@porteus:~$ su
Password: 
root@porteus:/home/guest# usm -v
 You are using USM version:  3.2.0

root@porteus:/home/guest# 

[Bogomips]

Am pleased to be able to share a regular procedure I have established to update the usm mirrors using a couple of scripts. The one will set the fastest mirror available for each of the repositories, while the other can be used when the usm is snagged on a mirror, to push through the update, using the output of the first, come what may, within reason of course: [INFO & SCRIPT] Slackware Mirror Settings for USM

[Ed_P]

^ "one" "other"? From copying your scripts I have: badm.sh, fasm.sh, masm.sh, safm.sh, sam.sh and sesm.sh in addition to usmfix.sh. Which ones are you referring to as "one" and "other"??

[Bogomips]

^ Ed, this is aimed at allcomers who are not point and click, someone like LordVortex maybe, who was having trouble getting the salix update through. They would follow this link [INFO & SCRIPT] Slackware Mirror Settings for USM and find just two scripts, the one being sesm.sh and the other safm.sh. if anything is not clear, please reply in the topic so that I can improve on the procedure description.

I have: badm.sh, fasm.sh, masm.sh, safm.sh, sam.sh and sesm.sh in addition to usmfix.sh

As with all things it's only in practice that one finds out which are useful, and which less useful. As the above two already incorporate fasm.sh and masm.sh respectively, that only leaves script to check a repository's mirror (aka crm.sh) which I sometimes use. And the script to show all mirrors being used (aka sam.sh), could also be useful. As you will see if you follow the link, badm.sh is a one off needed to tune the mirror testing, which one does not need to use if one does not need to tune the testing.

[Blaze]

With this repo http://mirrors.xmission.com/salix/x86_6 ... ware-14.2/ in usm - Salix database get updates without any problems.

[port]

^ Thanks Ed.

@ brokenman

Edit of funcpackageGet

1. Replace 31 with ed0
2. Replace 86-88 with ed1
3. After 345 insert ed2
4. Replace 368 with ed3

Confirmation

I have problems with funcpackageGet Heuristic

Code: Select all

# usm -g text2pdf                                                                                                                                                                                                     
                                                                                                                                                                                                                                               
 The following items were found.                                                                                                                                                                                                               
 Choose an number to confirm.                                                                                                                                                                                                                  
 ctrl+c to quit                                                                                                                                                                                                                                
                                                                                                                                                                                                                                               
1) text2pdf-1.1-x86_64-1_slonly.txz                                                                                                                                                                                                            
#? 1                                                                                                                                                                                                                                           
                                                                                                                                                                                                                                               
Processing:   text2pdf-1.1-x86_64-1_slonly.txz                                                                                                                                                                                                 
                                                                                                                                                                                                                                               
=============                                                                                                                                                                                                                                  
HEURISTIC USM
=============

/usr/share/usm/funcpackageGet: line 126: declare: 12K: value too great for base (error token is "12K")


# grep 'declare -i CSIZE' /usr/share/usm/funcpackageGet
       declare -i CSIZE=`tr -d [:alpha:] <<<$HSIZE`
        declare -i CSIZE=`tr -d [:alpha:] <<<$HSIZE`

[Bogomips]

@ port Thanks for feedback.

• Pristine usm

  Code: Select all

  root@porteus:/home/guest# usm -g text2pdf
  
   The following items were found.
   Choose an number to confirm. 
   ctrl+c to quit
  
  1) text2pdf-1.1-x86_64-1_slonly.txz
  #? 1
  
  Processing:   text2pdf-1.1-x86_64-1_slonly.txz 
  
   The following packages are required. 
  text2pdf-1.1-x86_64-1_slonly.txz [12K] [not installed]
  
  Total size: 12 KB
  
   Press [r] to remove packages, [q] to quit, or enter to start downloading.
  
   Updates are available. 
   Program update:   not required 
   Database update:  available 
   Please run:  usm -u all  and/or  usm -u usm 
   This message can be disabled in /etc/usm/usm.conf
   ############################### 
  text2pdf-1.1-x86_64-1_slonly.txz already exists

• Heuristic usm

  Code: Select all

  root@porteus:/home/guest# cp /usr/share/usm/funcpackageGet  /usr/share/usm/funcpackageGet.bak
  root@porteus:/home/guest# cp $g/usm/funcpackageGet.hic /usr/share/usm/funcpackageGet 
  
  root@porteus:/home/guest# usm -g text2pdf
  
   The following items were found.
   Choose an number to confirm. 
   ctrl+c to quit
  
  1) text2pdf-1.1-x86_64-1_slonly.txz
  #? 1
  
  Processing:   text2pdf-1.1-x86_64-1_slonly.txz
  
  =============
  HEURISTIC USM
  =============
  
  
   The following packages are required.
  text2pdf-1.1-x86_64-1_slonly.txz [12K] [not installed]
  
  Total size: 12 KB
  
   Press [r] to remove packages, [q] to quit, or enter to start downloading.
  
   Updates are available.
   Program update:   not required
   Database update:  available
   Please run:  usm -u all  and/or  usm -u usm
   This message can be disabled in /etc/usm/usm.conf
   ###############################
  text2pdf-1.1-x86_64-1_slonly.txz already exists

Edit of funcpackageGet

1. Replace 31 with ed0
2. Replace 86-88 with ed1
3. After 345 insert ed2
4. Replace 368 with ed3

Confirmation

Please do not do the edit. Only use the complete file posted here: USM Skirting Sync Issue
The problem with using line numbers to edit is that the number of comment lines in usm 3.2 vary between releases.