shlibs https://shlibs.org shlibs, the CLI Assistant and Script Manager Wed, 17 Jun 2020 07:14:50 +0000 en-US hourly 1 https://i1.wp.com/shlibs.org/wp-content/uploads/2020/02/cropped-window-1.png?fit=32%2C32&ssl=1 shlibs https://shlibs.org 32 32 171626642 shlibs Settings https://shlibs.org/2020/03/26/shlibs-settings/ https://shlibs.org/2020/03/26/shlibs-settings/#respond Thu, 26 Mar 2020 08:15:26 +0000 https://shlibs.org/?p=672 Read More »shlibs Settings]]> All the options meant to be customizable by the end user are contained by a single file located at var/shlibs_settings.sh. In this way shlibs settings section complies with one of the main goals of the application: simplicity.

Do not hesitate to tweak shlibs settings. If running into any problems while doing that, use shlibs -reset to restore shlibs_settings.sh to default content.

The settings are fully described below. When altering their values, in order to maintain maximum POSIX compatibility, please keep original format:

variable=value
export variable

Full details of shlibs settings:

  • SHLIBS_MATCH_PAGE_SIZE
    • meaning: the number of results per page in query and setup script modes
    • default value: 10
    • constraints: try to fit the number of results in available height, afterwards adjust values to fit in interval 3-50
  • SHLIBS_MATCH_MAX
    • meaning: the maximum number of results shown per query
    • default value: 400
    • constraints: minimum 5
    • other: use 0 (zero) to display all results
  • SHLIBS_MAX_RESULTS_MEM
    • meaning: the maximum number of results kept in memory before switching to files storage
    • default value: 10000
    • constraints: none
    • other: for most purposes this setting should remain default or high in value unless the system has a seriously low amount of memory available
  • SHLIBS_SHOW_OPTIONS
    • meaning: show/hide brief details about the options available
    • default value: 0 (true, show)
    • constraints: values 0 or 1
  • SHLIBS_MATCH_HIGHLIGHT
    • meaning: the character used to highlight instances of shlibs found while in setup script mode
    • default value: * , which triggers highlights of instances in format ***shlibs***
    • other:
      1. specific for queries occurring in setup script mode ( shlibs -s /path/to/script )
      2. additional highlighting of shlibs instances and the replacements is done using bold red characters (starting with version 1.1.16)
  • SHLIBS_CLEANUP_DISPLAY
    • meaning: keep the terminal clean
    • default value: 0 (true)
    • constraints: values 0 or 1
    • other: setting this to 1 can be used to keep track of operations
  • SHLIBS_TERMINAL_CHAR_WIDTH
    • meaning: increases readability by folding shlibs output to the number of characters indicated by this variable
    • default value: 80
    • constraints: minimum 36
    • other: extremely useful for small size screens/terminals
  • SHLIBS_FORCE_SHELL
    • meaning: forces shlibs to run using specified shell(s)
    • default value: ” (empty string – meaning auto selection which provides maximum compatibility)
    • other:
      1. use comma separated, no whitespace list of shells
      2. shlibs checks the shells in the order given; if using bash as the first and only shell in the list you are severely restricting the compatibility of your applications/scripts, but you can benefit from bash specific advantages such as arrays, extended parameter expansion a.o.
  • SHLIBS_SHOW_VERSION_IN_QUERY
    • meaning: advanced options useful when working with multiple versions of libraries
    • default value: 1 (false)
    • constraints: values 0 or 1
    • other: includes the version of libraries in the results list of query / setup script modes
]]>
https://shlibs.org/2020/03/26/shlibs-settings/feed/ 0 672
shlibs Queries https://shlibs.org/2020/03/16/shlibs-queries/ https://shlibs.org/2020/03/16/shlibs-queries/#respond Mon, 16 Mar 2020 11:42:37 +0000 https://shlibs.org/?p=501 Read More »shlibs Queries]]> shlibs at its simplest is a script manager. shlibs runs interpretable scripts based on codes (for libs using official format) or paths.
When launched with no options, shlibs goes into query mode. In this mode a user can search utility (library) code or path using keywords. Let’s see how to perform effective shlibs queries and what options are available and can help during search.

shlibs query screenshot

Depending on user’s needs, shlibs can perform simple lexical queries, or complex queries using focused search, library versions and path lookups.
The results of search operations are displayed using a paged list which can be navigated back and forth. A slightly modified form of query takes place when setting up shell scripts ( -s option ), but the main principles remain the same and a couple of options required by scripts are added.
Let’s take a look at each one of them in detail.

Simple Lexical Queries with shlibs

When not involved in the development of shlibs core and shlibs libraries, or not using custom libraries, it is recommended to use simple lexical queries.
Simple lexical queries and other advanced queries are all using keywords, spaces ( ) and commas (,).

Space character acts as logical AND operator between keywords. This means that shlibs will look for libraries containing all of the keywords which are separated by space characters, without looking at the order.

Comma character acts as logical OR operator between keywords. This means that shlibs will look for libraries containing some of the keywords which are separated by comma characters.

In order to start a search type the keyword(s) and press Enter (Return) key. Any keywords added after a query is complete will be automatically separated with a comma from the previous existing keywords or sequences of keywords. In any circumstance the user can declare his own spaces and commas and these will be treated as specified above.

Let’s take a look at a few examples using default libraries packed with shlibs:

  • the following query will succeed because there are libraries containing keyword math, or keyword trim
shlibs successful query of comma separated keywords
  • the following query will fail because there are no libraries containing keyword math and keyword trim
shlibs failed query of keywords separated by space character
  • the following query will succeed because there are libraries containing sequence ‘keyword math and keyword trim‘, or sequence ‘keyword string and keyword shuffle

Basic Query Options

When in query mode a couple of options are available and can help while searching. By default these options are briefly described when usable. In order to hide these instructions and make room for query results, you can go to var/shlibs_settings.sh file and change the value of variable SHLIBS_SHOW_OPTIONS to 1 (important! see Settings affecting shlibs queries).
Basic query options are single letter options which can be used at any time in a query. By default, when unassigned single characters are used for a query, shlibs will show the examples ( corresponding to option [ x ] ).

  • query mode help [ h ]
  • query mode examples [ x ]
  • navigate back [ b ] and forth [ f ]
Navigate Forward to the next page by entering f
Navigate Forward to the next page by entering f
Navigate Back to the previous page by entering b
Navigate Back to the previous page by entering b
  • clear keywords and results list [ c ]
  • quit shlibs query [ q ]

Library Specific Options

  • showing library help [ h+# ]
    This is done by entering h+#, where # is the number in square brackets at the beginning of a row in the results list.
Showing library help in query mode
Showing library help in query mode
  • showing library examples [ x+# ]
    This is done by entering x+#, where # is the number in square brackets at the beginning of a row in the results list.
Showing library examples in query mode
Showing library examples in query mode
  • selecting library [ # ]
    This is done by entering the number in square brackets at the beginning of a row in the results list. Once this number is entered shlibs displays relevant info on how to use selected library and exits.
Selecting a library in shlibs query mode
Selecting a library in shlibs query mode

Focused Search and Path Lookup in shlibs and shlibs Queries

Official libraries have unique code names. For this reason it is sufficient to use library code to call an official utility i.e. shlibs str005. Behind the scenes shlibs actually looks for any library files having the code provided as name while maintaining priorities as follows:

  • official libraries highest prority; if a library found then run it, if multiple libs found then output error and exit, if no libs found here go to next priority.
  • dev libraries secondary priority; if a library found then run it, if multiple libs found then output error and exit, if no libs found here go to next priority.
  • community libraries no priority; if a library found then run it, if none or multiple libs found then output error and exit.

It is quite possible a situation where you call your own libraries in dev folder and instead get an official library with the same name just because of the default priority lookup described above.
To avoid such situations paths must be specified.
shlibs uses UNIX style paths which start with the main type of library as follows:

  • /official/ , or official/
  • /dev/ , or dev/
  • /community/ , or community/

After the main type of library, a relative path to the desired library must be specified.
In the example below dev section contains two libraries having the same name dupli001, but different locations.

Duplicate library names in the same section
Duplicate library names in the same section

In order to call each one of these libraries we can use:

  • shlibs dev/dupli001
  • shlibs dev/foo/dupli001

It is intended to keep the same codes for the official libraries during all releases of shlibs in such a way that your scripts which include calls to shlibs official libraries will always work even when upgraded to use latest versions of shlibs and shlibs official libraries.
However given the flexibility allowed for dev and community sections, it is highly recommended to use path lookups in your scripts.


Following the specifications above, shlibs queries implement focused search and unfocused search techniques.

Focused search is the default type of search implemented by shlibs. This style of search does not allow the usage of Basic Regular Expressions. Focused search allows the user to narrow down the search to a specific section by using the format section/keywords i.e. community/keyword_in_both_dupli001_files.
If we consider the example above where the file name dupli001 can only be found in dev section, and keyword ‘search_term’ it is only contained by dupli001 files, a shlibs query using format community/search_term will fail (please note the fact that community is stripped from the keywords line).

Focused search 1
Focused search 1
Focused search 2
Focused search 2

If we are not narrowing down the search we should be able to see the libraries in dev section as usual:

Focused search 3
Focused search 3

Unfocused search enables the use of Basic Regular Expressions, but does not allow narrowing down search to specific section i.e. community/search_term cannot be used to narrow down the search. Use sequence ‘k+‘ in front of the keywords in order to enable unfocused search i.e. k+str*.

Unfocused search 1
Unfocused search 1
Unfocused search k+str*Unfocused search k+str*
Unfocused search k+str*

If we are not using k+ in front of str* then shlibs performs focused search which disables BRE and the result will be different:

Focused search str*
Focused search str*

Using Library Versions with shlibs

shlibs library versions folders structure
Folders Structure required by shlibs Libraries Versions

When the official format for libraries is implemented, shlibs can recognize library versions if used.
The following example shows two versions of official libraries in use: 1.0.0 and 1.0.7. Please note the folders’ structure used for versions.

Users developing shlibs libraries for official or private use can enable the display of library versions in query / setup script mode by changing the value of variable SHLIBS_SHOW_VERSION_IN_QUERY to 0. In order to enable this option:

  • open file var/shlibs_settings.sh
  • change variable value to 0
    SHLIBS_SHOW_VERSION_IN_QUERY=0

The following query using keyword ‘random’ outputs 4 libraries: 2 for each of versions 1.0.0 and 1.0.7. In this example the versions of the libraries are different.

shlibs Versions and Libraries with the Same Code and Different Version

It is quite possible that some libraries in different versions have the same code and same version. In this case, if no other instructions given, shlibs will use the library in the latest available version of the libraries.

shlibs Versions and Libraries with the Same Code and the Same Version

If versions found, irrespective of the value of SHLIBS_SHOW_VERSION_IN_QUERY, when a library is selected in query mode, additional info is displayed as shown in the example below where the user selected library no 2 in the results list (str022).

In order to select a certain version of a library, the option -v is used. This option must be used immediately after shlibs i.e. shlibs -v 2 str022.
Option -v must be followed by the release number of the version as indicated in the query results list. If the release number of a library does not exist, shlibs returns the latest available version.
A couple of other options can be used with -v :

  • shlibs -v 1 str005
    • runs version one of library str005
  • shlibs -v 1 -h str005
    • shows the help section of version one of library str005
  • shlibs -v 1 -x str005
    • shows examples section of version one of library str005
  • shlibs -v 1 -p str005
    • returns the path to version 1 of library str005
  • shlibs tst001 -v 2 str005
    • runs the tests of version two of str005
  • shlibs -v 2 tst001 -v 1 str005
    • runs the tests of version one of str005, using the version two of tst001 (shlibs main testing utility)

Settings Affecting shlibs Queries

As described in this post, all shlibs settings can be changed from the same file located at var/shlibs_settings.sh.
The following variables declared in var/shlibs_settings.sh file can affect the queries:

  • SHLIBS_MATCH_PAGE_SIZE
    • meaning: the number of results per page in query and setup script modes
    • default value: 10
    • constraints: try to fit the number of results in available height, afterwards adjust values to fit in interval 3-50
  • SHLIBS_MATCH_MAX
    • meaning: the maximum number of results shown per query
    • default value: 400
    • constraints: minimum 5
    • other: use 0 (zero) to display all results
  • SHLIBS_MAX_RESULTS_MEM
    • meaning: the maximum number of results kept in memory before switching to files storage
    • default value: 10000
    • constraints: none
    • other: for most purposes this setting should remain default or high in value unless the system has a seriously low amount of memory available
  • SHLIBS_SHOW_OPTIONS
    • meaning: show/hide brief details about the options available
    • default value: 0 (true, show)
    • constraints: values 0 or 1
  • SHLIBS_MATCH_HIGHLIGHT
    • meaning: the character used to highlight instances of shlibs found while in setup script mode
    • default value: * , which triggers highlights of instances in format ***shlibs***
    • other:
      1. specific for queries occurring in setup script mode ( shlibs -s /path/to/script )
      2. additional highlighting of shlibs instances and the replacements is done using bold red characters (starting with version 1.1.16)
  • SHLIBS_CLEANUP_DISPLAY
    • meaning: keep the terminal clean
    • default value: 0 (true)
    • constraints: values 0 or 1
    • other: setting this to 1 can be used to keep track of operations
  • SHLIBS_TERMINAL_CHAR_WIDTH
    • meaning: increases readability by folding shlibs output to the number of characters indicated by this variable
    • default value: 80
    • constraints: minimum 36
    • other: extremely useful for small size screens/terminals
  • SHLIBS_SHOW_VERSION_IN_QUERY
    • meaning: advanced options useful when working with multiple versions of libraries
    • default value: 1 (false)
    • constraints: values 0 or 1
    • other: includes the version of libraries in the results list of query / setup script modes
]]>
https://shlibs.org/2020/03/16/shlibs-queries/feed/ 0 501
shlibs Audits and Security https://shlibs.org/2020/02/29/shlibs-audits-and-security/ https://shlibs.org/2020/02/29/shlibs-audits-and-security/#respond Sat, 29 Feb 2020 06:32:06 +0000 https://shlibs.org/?p=229 Read More »shlibs Audits and Security]]> shlibs does not include any compiled parts in it’s structure. The application contains only folders and text files (most of them shell scripts) which are easily editable at any level.
Given the fact that shlibs parts are easily editable, malicious actions could be added in shlibs packages hosted at other locations. For this reason it is critical to verify downloaded packages against the checksums displayed at shlibs.org /shlibs.org especially if shlibs was downloaded from non-official locations.

shlibs (core and the official libraries) does not bring in anything compiled to your system and instead makes use of and assumes POSIX tools are available and are acting as required by the standard POSIX® IEEE® Std 1003.1-2017. Internal tests are performed and errors are handled when operating with these tools. This means that on a fresh and secure installation of an OS, you should be able to use a verified release of shlibs without any concerns regarding security.
On systems already in use it is sometimes possible that the POSIX tools expected by shlibs might be tempered with or replaced by other tools using the same names at the same standard locations. If you know or suspect your system’s POSIX tools are not acting as required by standard POSIX® IEEE® Std 1003.1-2017 it is better to try and run the application on a different system because shlibs makes extensive use of POSIX standard tools.

Security audits can be easily performed on release versions of shlibs. Once such audit passes, a developer/company can decide to add own libraries and distribute for internal use the updated package and updated checksums.

While it is still better to use shlibs from internal storage devices, it is also a great idea to keep shlibs on a portable storage device and use it on the go.
shlibs it’s designed to run properly from a portable storage device when using default settings (keep SHLIBS_MAX_RESULTS_MEM default or high in value unless the system has a seriously low amount of memory available).

]]>
https://shlibs.org/2020/02/29/shlibs-audits-and-security/feed/ 0 229