HOWTO search multiple modules
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
HOWTO search multiple modules
Has anyone ever attempted to create a search script that is able to temporary "mount" several Port modules and search for a file pattern in these modules? (Like a local search could be done e.g. via "find [path] | grep [pattern]"
The script should be able to do so without having to use lots of RAM.
Currently I cannot think of an elegant way to code it by myself.
Update
I also asked on https://stackoverflow.com and posted a working result in #9: search multiple modules
The script should be able to do so without having to use lots of RAM.
Currently I cannot think of an elegant way to code it by myself.
Update
I also asked on https://stackoverflow.com and posted a working result in #9: search multiple modules
Cheers!
Yours Rava
Yours Rava
- brokenman
- Site Admin
- Posts: 6105
- Joined: 27 Dec 2010, 03:50
- Distribution: Porteus v4 all desktops
- Location: Brazil
Re: search multiple modules
I'm not on linux right now but a simple loop should be able to list the content modules and then grep for what you need.
You would call this script with 2 arguments.
1. Path to a bunch of modules
2. Argument to grep for.
./path/to/script.sh /path/to/mymodules docs
This would search for modules at /path/to/mymodules and show results containing the word docs.
Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do
lsxzm $module | grep "$2"
done
1. Path to a bunch of modules
2. Argument to grep for.
./path/to/script.sh /path/to/mymodules docs
This would search for modules at /path/to/mymodules and show results containing the word docs.
How do i become super user?
Wear your underpants on the outside and put on a cape.
Wear your underpants on the outside and put on a cape.
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: search multiple modules
Currently I simply try to find where the 2 missing dependencies for mtPaint are for the most recent 5.0 testversion with Openbox DE.
The missing libs are
Put your code into a text executable named /usr/local/bin/lsxzmgrep . The result looks like so:
So, it lists the files, but one crucial info is missing: the name of the module(s) that contain the found files. All we know that they are in one of the modules in /mnt/sda1/Porteus_4.0/porteus/base
The for do loop could list all modules it is searching, but best would be to only list the ones that get results with the scripts grep pattern "$2"...
I only could think one one way achieving it: buffering the grep results instead of printing it right away, and if the grep found something, then first print the module name, and then the results, if grep finds nothing continue with the next module just like the loop would have done anyway.
Or even better, do it like grep would do it when searching multiple files, print the module name with added ": " in front of the "grep result" list:
Maybe there is even a trick I am not aware of, so that we can archive the above without the need of buffering the grep results and put together the lines like
echo ${module}:{grep_result(n)}
echo ${module}:{grep_result(n+1)}
...
The missing libs are
Code: Select all
ldd /usr/bin/mtpaint|grep not
libtiff.so.3 => not found
libjpeg.so.8 => not found
Code: Select all
root@porteus:/mnt/sda1/Porteus_4.0/porteus/base# lsxzmgrep . libtiff.so.3
/usr/lib64/libtiff.so.3
/usr/lib64/libtiff.so.3.9.7
The for do loop could list all modules it is searching, but best would be to only list the ones that get results with the scripts grep pattern "$2"...
I only could think one one way achieving it: buffering the grep results instead of printing it right away, and if the grep found something, then first print the module name, and then the results, if grep finds nothing continue with the next module just like the loop would have done anyway.
Or even better, do it like grep would do it when searching multiple files, print the module name with added ": " in front of the "grep result" list:
Code: Select all
root@porteus:/mnt/sda1/Porteus_4.0/porteus/base# lsxzmgrep . libtiff.so.3
./example1.xzm: /usr/lib64/libtiff.so.3
./example1.xzm: /usr/lib64/libtiff.so.3.9.7
echo ${module}:{grep_result(n)}
echo ${module}:{grep_result(n+1)}
...
Cheers!
Yours Rava
Yours Rava
- Ed_P
- Contributor
- Posts: 8360
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
Re: search multiple modules
If the "missing" libs are in the Porteus base then the app, mtpaint, will run. They don't need to be in the module also.
Ed
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: search multiple modules
No, it won't.
I currently run 5.0 from a different base folder. And quite obviously, 5.0 lacks the 2 libs as listed above.
This is the reason why I got the mtPaint error /when running 5.0 and trying to start mtpaint) with the listed missing libs as quoted above. See here for the quote search multiple modules - post #3 by Rava
When booting 4.0 it sure works okay. But this is not the question of the thread anyway. *points to thread title*
Cheers!
Yours Rava
Yours Rava
- Ed_P
- Contributor
- Posts: 8360
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
Re: search multiple modules
Ok, I missed the part that this was done with Porteus 4.0's base where mtpaint works.Rava wrote: ↑21 Dec 2018, 20:47Put your code into a text executable named /usr/local/bin/lsxzmgrep . The result looks like so:Code: Select all
root@porteus:/mnt/sda1/Porteus_4.0/porteus/base# lsxzmgrep . libtiff.so.3 /usr/lib64/libtiff.so.3 /usr/lib64/libtiff.so.3.9.7
As for brokenman's code this might yield something close to what you're looking for.
Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do
echo " * " $module
lsxzm $module | grep "$2"
done
Last edited by Ed_P on 22 Dec 2018, 06:28, edited 1 time in total.
Reason: Tweaked the echo.
Reason: Tweaked the echo.
Ed
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: search multiple modules
Indeed it is not ideal, but for now it works okay, since the module name is important, or else the result is more or less meaningless. "Yes, the files are in one of the 4 or 8 modules... but dunno in which ones" is sure not a result anyone wants.Ed_P wrote: ↑22 Dec 2018, 00:24As for brokenman's code this might yield something close to what you're looking for.Yes, it will display all module names and the name proceeding the grep hits is the module name you want.Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do echo $module lsxzm $module | grep "$2" done
Code: Select all
root@porteus:/mnt/sda1/Porteus_4.0/porteus/base# lsxzmgrep . libtiff.so.3
./000-kernel.xzm
./001-core.xzm
/usr/lib64/libtiff.so.3
/usr/lib64/libtiff.so.3.9.7
./001-de-core_locales.xzm
./002-xorg.xzm
./003-xfce.xzm
./010-nvidia-304.137-k.4.16.3-x86_64-don.xzm
./991-usr_local_binV2018-06-25.xzm
Update
And after adding a small module, mtpaint works like a charm.
Code: Select all
root 454656 2018-12-22 03:03 023-mtpaint-libs.xzm
Code: Select all
root@porteus:/# 1024calc 454656
444 KB
444.00 KB
root@porteus:/# cat /usr/local/bin/1024calc
#!/usr/bin/awk -f
BEGIN{ x = ARGV[1]
split("B KB MB GB TB PB",type)
for(i=5;y < 1;i--)
y = x / (2**(10*i))
print y " " type[i+2]
printf("%.2f %s\n",y,type[i+2])
}
And I even inserted more than the libs mtpaint originally complained as quoted above, I also added higher numbered / never versions of the libs, just to make sure it won't fail.
Cheers!
Yours Rava
Yours Rava
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: search multiple modules
I asked about it on https://stackoverflow.com/questions/539 ... ve-results and got this code snipped as solution:
But sadly that won`t work:
Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do
lsxzm ${module} | grep "$2" | sed -r "s/.+/${module}\n&/"
done
Code: Select all
/porteus/base$ lsxzmgrep . libtiff.so.3
sed: -e expression #1, char 10: number option to `s' command may not be zero
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
sed: -e expression #1, char 11: unknown option to `s'
Cheers!
Yours Rava
Yours Rava
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: search multiple modules
And a bit later someone posted a working code:I was hoping there being some neat bash script coding-fu that gets the trick done without the need of storing grep's result, but seems there is no such trick. Or the ones who know of such trick not reply.
Code: Select all
#!/bin/bash
for module in $(find $1 -type f -name "*.xzm"); do
result=$(lsxzm $module | grep "$2")
if [ $? -eq 0 ]; then
# grep returned OK, write out results
echo $module
echo "$result"
fi
done
Cheers!
Yours Rava
Yours Rava
- Ed_P
- Contributor
- Posts: 8360
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
Re: HOWTO search multiple modules
I was thinking about playing with an "if [ $? -eq 0 ]; then" option but after you showed the simple script showed only 2 libs I figured why bother.
"I"https://stackoverflow.com/questions/53918069/live-linux-grep-script-only-print-positive-results wrote:I try to code a bash 4 (or sh, if that's possible) script that does the following: 1st parameter: the path to the compressed modules to search, could be "." 2nd the search pattern to look for
Ed
- brokenman
- Site Admin
- Posts: 6105
- Joined: 27 Dec 2010, 03:50
- Distribution: Porteus v4 all desktops
- Location: Brazil
Re: HOWTO search multiple modules
Or even better, do it like grep would do it when searching multiple files, print the module name with added ": " in front of the "grep result" list:
Code: Select all
grep -Hn
How do i become super user?
Wear your underpants on the outside and put on a cape.
Wear your underpants on the outside and put on a cape.
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: HOWTO search multiple modules
Hmm I got an update on stackoverflow:brokenman wrote: ↑26 Dec 2018, 13:47This will show you the filename and the line number.Code: Select all
grep -Hn
Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do
lsxzm ${module} | grep "$2" | sed -r "s#.+#${module}\n&#"
done
Code: Select all
/porteus/base# /tmp/lsxzmgrep2 . output_dummy
./002-xorg.xzm
/usr/lib64/mpg123/output_dummy.so
When I combine that with your approach
Code: Select all
for module in $(find $1 -type f -name "*.xzm"); do
lsxzm ${module} | grep -Hn "$2" | sed -r "s#.+#${module}\n&#"
done
Code: Select all
/porteus/base# /tmp/lsxzmgrep2 . output_dummy
./002-xorg.xzm
(standard input):2562:/usr/lib64/mpg123/output_dummy.so
Cheers!
Yours Rava
Yours Rava
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: HOWTO search multiple modules
So, to conclude the HOWTO, for now it seems this script codedoes create the best results.
Thanks to all that helped in creating the code!
____________________________________________
(I should add a bit of debugging, e.g. when lsxzmgrep is just started like so its output is less helpful to the user, to say the least:
How should the user know that the correct syntax for lsxzmgrep is "lsxzmgrep [path] [grep-pattern]")
Code: Select all
#!/bin/bash
for module in $(find $1 -type f -name "*.xzm"); do
lsxzm ${module} | grep "$2" | sed -r "s#.+#${module}\n&#"
done
Thanks to all that helped in creating the code!
____________________________________________
(I should add a bit of debugging, e.g. when lsxzmgrep is just started like so its output is less helpful to the user, to say the least:
Code: Select all
root@porteus:/tmp/64bit-kernel4.20.2# lsxzmgrep initrd
find: ‘initrd’: No such file or directory
root@porteus:/tmp/64bit-kernel4.20.2# lsxzmgrep . initrd
./06-crippled_sources-4.20.2-64bit.xzm
/usr/src/linux-4.20.2/include/config/blk/dev/initrd.h
./06-crippled_sources-4.20.2-64bit.xzm
/usr/src/linux-4.20.2/include/linux/initrd.h
./06-crippled_sources-4.20.2-64bit.xzm
/usr/src/linux-4.20.2/init/.do_mounts_initrd.o.cmd
./06-crippled_sources-4.20.2-64bit.xzm
/usr/src/linux-4.20.2/init/do_mounts_initrd.c
./06-crippled_sources-4.20.2-64bit.xzm
/usr/src/linux-4.20.2/tools/testing/selftests/rcutorture/doc/initrd.txt
Cheers!
Yours Rava
Yours Rava
- Rava
- Contributor
- Posts: 5401
- Joined: 11 Jan 2011, 02:46
- Distribution: XFCE 5.01 x86_64 + 4.0 i586
- Location: Forests of Germany
Re: HOWTO search multiple modules
How can a user give a command the wrong syntax without calling the command? Please do explain how this is possible.
Your reply makes no sense, like, at all.
Cheers!
Yours Rava
Yours Rava