From: "André Erdmann" <dywi@mailerd.de> To: gentoo-commits@lists.gentoo.org Subject: [gentoo-commits] proj/R_overlay:gsoc13/next commit in: doc/rst/ Date: Wed, 26 Jun 2013 17:29:24 +0000 (UTC) [thread overview] Message-ID: <1372267719.0678580564dcf6aa8e2587d3b04d62544265ca74.dywi@gentoo> (raw) commit: 0678580564dcf6aa8e2587d3b04d62544265ca74 Author: André Erdmann <dywi <AT> mailerd <DOT> de> AuthorDate: Wed Jun 26 17:28:39 2013 +0000 Commit: André Erdmann <dywi <AT> mailerd <DOT> de> CommitDate: Wed Jun 26 17:28:39 2013 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=06785805 doc/rst: "run hooks", "distmap" --- doc/rst/usage.rst | 643 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 383 insertions(+), 260 deletions(-) diff --git a/doc/rst/usage.rst b/doc/rst/usage.rst index 078a7ac..2daedce 100644 --- a/doc/rst/usage.rst +++ b/doc/rst/usage.rst @@ -1010,6 +1010,22 @@ A local directory will never be modified. consider using one of the **websync** repo types, websync_repo_ and websync_pkglist_. +--------- + distmap +--------- + +*roverlay* uses a text file to store information about files in the package +mirror directory (OVERLAY_DISTDIR_ROOT_). This is necessary for comparing +package files from repos with files for which an ebuild has already been +created (in previous runs). + +With the help of the *distmap file*, *roverlay* is able to determine whether +upstream has changed a package file silently and creates a revision-bumped +ebuild for the *new* package file. + +The *distmap file* can optionally be compressed (bzip2 or gzip), which +reduces its size considerably. + ===================== Additions Directory @@ -1790,6 +1806,354 @@ Moving such packages to a "R-package" sub directory would be possible, too: END; +============= + Event Hooks +============= + +*roverlay* is able to call a script when certain events occur, e.g. after +successful overlay creation, which can be used to perform additional actions +without touching *roverlay's* source code. + +To realize this, *roverlay* determines whether a given event is permitted +(`event policy`_) and, if so, creates a `hook environment`_ and runs the +script. Additionally, shell scripts can load *roverlay's* *$FUNCTIONS* file, +which provides extra functionality. + +.. Note:: + + *roverlay* waits until the script terminates and thus possibly waits + forever. + + +---------------------- + Default event script +---------------------- + +The default event script (``mux.sh``) loads *$FUNCTIONS* and then runs the +following script files (by sourcing them), in order: + +#. all files from ADDITIONS_DIR_/hooks/<event> that end with *.sh* + (``<ADDITIONS_DIR>/hooks/<event>/*.sh``) +#. all files ADDITIONS_DIR_/hooks that end with *.<event>* + (``<ADDITIONS_DIR>/hooks/*.<event>``) + +So, there are two naming schemes for *hook scripts*. +Either one is acceptable, but it is advised to stay consistent. +Having the same script at both locations results in executing it twice. + +.. Note:: + + The default event script enables *nounset* behavior, which causes the + shell command interpreter to exit abnormally if an unset variable is + accessed. + + +++++++++++++++++++++++++++ + Activating a hook script +++++++++++++++++++++++++++ + +Activating a hook script can be done by symlinking it: + +.. code-block:: text + + ln -s <real script> ${ADDITIONS_DIR}/hooks/<event>/<name>.sh + # or + ln -s <real script> ${ADDITIONS_DIR}/hooks/<name>.<event> + + +++++++++++++++++++++++++++ + Adding a new hook script +++++++++++++++++++++++++++ + +As hinted before, *hook scripts* are simple shell scripts. The following +template gives an idea of how to write them: + +.. code-block:: sh + + #!/bin/sh + #set -u + + # load essential functions + # (not necessary when using the default event script) + . "${FUNCTIONS?}" || exit + + ## load additional function files, if any + #$lf <name(s)> + + # script body + # + # when redirecting output to $DEVNULL, use ">>" instead of ">" as + # $DEVNULL could be a file + #ls >>${DEVNULL} + # + # ... + + + # the script must not exit if everything went well (return is ok) + return 0 + + +-------------- + Event Policy +-------------- + +The *event policy* controls whether a certain event actually triggers a script +call or not. +It is constructed by parsing the EVENT_HOOK_RESTRICT_ config option: + +* a word prefixed by ``-`` means *deny specific event* (-> blacklist) +* the asterisk char ``*`` (or ``+*``) sets the policy to + *allow unless denied* (blacklist) or *allow all* +* a word prefixed by ``+`` or without a prefix char means + *allow specific event* (-> whitelist) +* the asterisk char with ``-`` as prefix (``-*``) sets the policy to + *deny unless allowed* (whitelist) or *deny all* + + +The policy defaults to *allow all* if ``EVENT_HOOK_RESTRICT`` is not set in +the config file. An empty string sets the policy to *deny all*. + + +------------------ + Hook Environment +------------------ + +.. table:: environment variables provided by *roverlay* + + +----------------+-------------------+-----------------------------------------+ + | variable | source | notes / description | + +================+===================+=========================================+ + | PATH | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | LOGNAME | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | SHLVL | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | TERM | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | HOME | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | ROVERLAY_PHASE | event | event that caused the script to run | + +----------------+-------------------+-----------------------------------------+ + | OVERLAY | config | overlay directory (`OVERLAY_DIR`_), | + +----------------+-------------------+ initial working directory | + | S | *$OVERLAY* | | + +----------------+-------------------+ | + | PWD | *$OVERLAY* | | + +----------------+-------------------+-----------------------------------------+ + | DISTROOT | config | package mirror directory | + | | | (`OVERLAY_DISTDIR_ROOT`_) | + +----------------+-------------------+-----------------------------------------+ + | TMPDIR | os.environ, | directory for temporary files | + | | *fallback* | | + +----------------+-------------------+ | + | T | *$TMPDIR* | | + +----------------+-------------------+-----------------------------------------+ + | ADDITIONS_DIR | config | directory with supplementary files | + | | | (`OVERLAY_ADDITIONS_DIR`_) | + +----------------+-------------------+ | + | FILESDIR | *$ADDITIONS_DIR* | | + +----------------+-------------------+-----------------------------------------+ + | SHLIB | *$ADDITIONS_DIR*, | A list of directories with shell | + | | *installed?* | function files | + | | | (optional, only set if any dir exists) | + +----------------+-------------------+-----------------------------------------+ + | FUNCTIONS | *$ADDITIONS_DIR*, | file with essential shell functions | + | | *installed?* | (optional, only set if it exists) | + +----------------+-------------------+-----------------------------------------+ + | DEBUG | log level | *shbool* (``y`` or ``n``) that | + | | | indicates whether debug messages should | + | | | be printed | + +----------------+-------------------+-----------------------------------------+ + | VERBOSE | log level | *shbool* | + +----------------+-------------------+-----------------------------------------+ + | QUIET | log level | *shbool* that indicates whether scripts | + | | | should be quiet | + +----------------+-------------------+-----------------------------------------+ + | NO_COLOR | *n/a* | *shbool*. Always set to *y* since | + | | | colored output should not be produced | + +----------------+-------------------+-----------------------------------------+ + | NOSYNC | config | *shbool* that indicates whether data | + | | | transfers from/to remote machines is | + | | | allowed (NOSYNC_) | + +----------------+-------------------+-----------------------------------------+ + | EBUILD | config | the *ebuild* executable | + +----------------+-------------------+-----------------------------------------+ + | GIT_EDITOR | *n/a* | set to */bin/false* | + +----------------+-------------------+ | + | GIT_ASKPASS | *n/a* | | + +----------------+-------------------+-----------------------------------------+ + + +The default *essential shell functions* file (*$FUNCTIONS*) makes, +when included in the hook script, most of the enviroment variables readonly. + + +.. table:: variables provided by *$FUNCTIONS* + + +-----------------+-------------------------------------------------------+ + | variable | description | + +=================+=======================================================+ + | IFS_DEFAULT | default *internal field separator* | + +-----------------+-------------------------------------------------------+ + | IFS_NEWLINE | *IFS* for iterating over text lines | + +-----------------+-------------------------------------------------------+ + | DEVNULL | */dev/null* target (could also be a file) | + +-----------------+-------------------------------------------------------+ + | EX_ERR | default error exit code | + +-----------------+-------------------------------------------------------+ + | EX_ARG_ERR | default exit code for arg errors | + +-----------------+-------------------------------------------------------+ + | SCRIPT_FILENAME | file name of the hook script | + +-----------------+-------------------------------------------------------+ + | SCRIPT_NAME | name of the hook script (without file extension) | + +-----------------+-------------------------------------------------------+ + | lf | reference to a function that loads additional shell | + | | function files | + +-----------------+-------------------------------------------------------+ + + +*$FUNCTIONS* also provides a number of shell functions: + +.. code-block:: sh + + # --- message --- + # + # void veinfo ( message ) + # Prints a message to stdout if $DEBUG is set to 'y'. + # + # void einfo ( message ) + # Prints a message to stdout if $VERBOSE is set to 'y'. + # + # void ewarn ( message ) + # Prints a message to stderr unless $QUIET is set to 'y'. + # + # void eerror ( message ) + # Prints a message to stderr. + # + # + # --- core --- + # + # @noreturn die ( [message], [exit_code] ), raises exit() + # Lets the script die with the given message/exit code. + # + # @noreturn OUT_OF_BOUNDS(), raises die() + # Lets the script die due to insufficient arg count. + # + # int run_command ( *cmdv ) + # Logs a command and runs it afterwards. + # + # int run_command_logged ( *cmdv ) + # Logs a command, runs it and logs the result. + # + # void autodie ( *cmdv ), raises die() + # Runs a command. Lets the script die if the command fails. + # + # + # void load_functions ( *filenames, **SHLIB ), raises die() + # Loads additional shell functions file(s) from $SHLIB. + # (Referenced by $lf.) + # + # void dont_run_as_root(), raises die() + # Lets the script die if it is run as root. + # + # int list_has ( word, *list_items ) + # Returns 0 if $word is in the given list, else 1. + # + # int qwhich ( *command ) + # Returns 0 if all listed commands are found by "which", else 1. + # + # + # --- fs --- + # + # int dodir ( *dir ) + # Ensures that zero or more directories exist by creating them if + # necessary. Returns the number of directories that could not be created. + # + # + # --- str --- + # + # int yesno ( word, **YESNO_YES=0, **YESNO_NO=1, **YESNO_EMPTY=2 ) + # Returns $YESNO_YES if $word means "yes", $YESNO_EMPTY if $word is empty + # and $YESNO_NO otherwise (if $word probably means "no"). + # + # ~int str_trim ( *args ) + # Removes whitespace at the beginning and end of a string and replaces + # any whitespace sequence within the string with a single space char. + # Passes the return value of the underlying sed command. + # + # ~int str_upper ( *args ) + # Echoes the uppercase variant of stdin or *args. + # Passes tr's return value. + # + # ~int str_lower ( *args ) + # Echoes the lowercase variant of stdin or *args. + # Passes tr's return value. + # + # ~int str_field ( fieldspec, *args, **FIELD_SEPARATOR=' ' ) + # Echoes the requested fields of stdin or *args. + # Passes cut's return value. + # + # + # --- int --- + # + # int is_int ( word ) + # Returns 0 if $word is an integer, else 1. + # + # int is_natural ( word ) + # Returns 0 if $word is an integer >= 0, else 1. + # + # int is_positive ( word ) + # Returns 0 if $word is an integer >= 1, else 1. + # + # int is_negative ( word ) + # Returns 0 if $word is an integer < 0, else 1. + # + + +------------------------ + Adding a function file +------------------------ + +Function files are shell script files that provide functions and variables. +They should, however, not execute any code directly. + +The template below illustrates how to write function files: + +.. code-block:: sh + + # protect against repeated inclusion of this file + # (replace <name> with a unique identifier) + if [ -z "${__HAVE_<name>__-}" ]; then + readonly __HAVE_<name>__=y + + # function file body + # ... + + fi + +Shell function files should be put into ``<ADDITIONS_DIR>/shlib``. + +--------------------- + Adding a hook event +--------------------- + +Adding a new event has to be done in *roverlay's* source code and is a rather +trivial task. The ``roverlay.hook`` module implements a function for running +the event script: + +.. code-block:: python + + # import hook module + import roverlay.hook + + # ... + # then, somewhere in the code + roverlay.hook.run ( "<event>" ) + # ... + roverlay.hook.run ( "<another event>" ) + + ========================= Configuration Reference ========================= @@ -1936,6 +2300,11 @@ DISTDIR_STRATEGY DISTDIR_VERIFY Alias to OVERLAY_DISTDIR_VERIFY_. +.. _DISTMAP_COMPRESSION: + +DISTMAP_COMPRESSION + Alias to OVERLAY_DISTMAP_COMPRESSION_. + .. _DISTMAP_FILE: DISTMAP_FILE @@ -2051,6 +2420,13 @@ OVERLAY_DISTDIR_VERIFY Defaults to *no* as the verification is normally not needed. +.. _OVERLAY_DISTMAP_COMPRESSION: + +OVERLAY_DISTMAP_COMPRESSION + Compression format for the distmap file. Choices are none, gzip/gz and + bzip2/bz2. + + Defaults to bzip2. .. _OVERLAY_DISTMAP_FILE: @@ -2212,9 +2588,10 @@ USE_EXPAND_RENAME .. _EVENT_HOOK: EVENT_HOOK - A script that is called for handling *events* (see Hooks_). + A script that is called for handling *events* (see `Event Hooks`_). - Defaults to ADDITIONS_DIR_/hooks/mux.sh. + Defaults to <libexec dir>/hooks/mux.sh if roverlay has been installed + and ADDITIONS_DIR_/hooks/mux.sh otherwise. .. _EVENT_HOOK_RESTRICT: @@ -2226,6 +2603,10 @@ EVENT_HOOK_RESTRICT Defaults to <not set>, which is equivalent to *deny all*. + ``EVENT_HOOK_RESTRICT="overlay_success"`` would allow the ``overlay_success`` + event only, whereas ``EVENT_HOOK_RESTRICT="* -overlay_success"`` would + allow any event except for ``overlay_success``. Also see `event policy`_. + .. _FILTER_SHELL_ENV: FILTER_SHELL_ENV @@ -3242,261 +3623,3 @@ Its mode of operation of operation is best described in pseudo-code: The dependency resolver can be **run as thread**, in which case the while loop becomes "loop until resolver closes". - - -======= - undef -======= - -<<TODO; this section has to merged/moved>> - ---------- - distmap ---------- - -The distmap is a file that stores information about the files in the package -mirror directory (`OVERLAY_DISTDIR_ROOT`_). -<<TODO>> - - -------- - hooks -------- - -*roverlay* calls a script when certain events occur, e.g. after successful -overlay creation. - -<<TODO>> - -* real work is offloaded to the called script -* roverlay provides some env vars -* roverlay provides a *$FUNCTIONS* file and a way to load other function files -* ``EVENT_HOOK_RESTRICT="this -not_that *"`` - -<<^^TODO>> - -.. Note:: - - *roverlay* waits until the script terminates. - - -++++++++++++++++++ - Hook environment -++++++++++++++++++ - -.. table:: environment variables provided by *roverlay* - - +----------------+------------------+-----------------------------------------+ - | variable | source | notes / description | - +================+==================+=========================================+ - | PATH | os.environ | | - +----------------+------------------+-----------------------------------------+ - | LOGNAME | os.environ | | - +----------------+------------------+-----------------------------------------+ - | SHLVL | os.environ | | - +----------------+------------------+-----------------------------------------+ - | TERM | os.environ | | - +----------------+------------------+-----------------------------------------+ - | HOME | os.environ | | - +----------------+------------------+-----------------------------------------+ - | ROVERLAY_PHASE | event | event that caused the script to run | - +----------------+------------------+-----------------------------------------+ - | OVERLAY | config | overlay directory (`OVERLAY_DIR`_), | - +----------------+------------------+ initial working directory | - | S | *$OVERLAY* | | - +----------------+------------------+ | - | PWD | *$OVERLAY* | | - +----------------+------------------+-----------------------------------------+ - | DISTROOT | config | package mirror directory | - | | | (`OVERLAY_DISTDIR_ROOT`_) | - +----------------+------------------+-----------------------------------------+ - | TMPDIR | os.environ, | directory for temporary files | - | | *fallback* | | - +----------------+------------------+ | - | T | *$TMPDIR* | | - +----------------+------------------+-----------------------------------------+ - | ADDITIONS_DIR | config | directory with supplementary files | - | | | (`OVERLAY_ADDITIONS_DIR`_) | - +----------------+------------------+ | - | FILESDIR | *$ADDITIONS_DIR* | | - +----------------+------------------+-----------------------------------------+ - | SHLIB | *$ADDITIONS_DIR* | directory with shell function files | - | | | (optional, only set if it exists) | - +----------------+------------------+-----------------------------------------+ - | FUNCTIONS | *$ADDITIONS_DIR* | file with essential shell functions | - | | | (optional, only set if it exists) | - +----------------+------------------+-----------------------------------------+ - | DEBUG | log level | *shbool* (``y`` or ``n``) that | - | | | indicates whether debug messages should | - | | | be printed | - +----------------+------------------+-----------------------------------------+ - | VERBOSE | log level | *shbool* | - +----------------+------------------+-----------------------------------------+ - | QUIET | log level | *shbool* that indicates whether scripts | - | | | should be quiet | - +----------------+------------------+-----------------------------------------+ - | NO_COLOR | *n/a* | *shbool*. Always set to *y* since | - | | | colored output should not be produced | - +----------------+------------------+-----------------------------------------+ - | NOSYNC | config | *shbool* that indicates whether data | - | | | transfers from/to remote machines is | - | | | allowed (NOSYNC_) | - +----------------+------------------+-----------------------------------------+ - | EBUILD | config | the *ebuild* executable | - +----------------+------------------+-----------------------------------------+ - | GIT_EDITOR | *n/a* | set to */bin/false* | - +----------------+------------------+ | - | GIT_ASKPASS | *n/a* | | - +----------------+------------------+-----------------------------------------+ - - -The default *essential shell functions* file (*$FUNCTIONS*) makes, -when included in the hook script, most of the enviroment variables readonly. - - -.. table:: variables provided by *$FUNCTIONS* - - +-----------------+-------------------------------------------------------+ - | variable | description | - +=================+=======================================================+ - | IFS_DEFAULT | default *internal field separator* | - +-----------------+-------------------------------------------------------+ - | IFS_NEWLINE | *IFS* for iterating over text lines | - +-----------------+-------------------------------------------------------+ - | DEVNULL | */dev/null* target (could also be a file) | - +-----------------+-------------------------------------------------------+ - | EX_ERR | default error exit code | - +-----------------+-------------------------------------------------------+ - | EX_ARG_ERR | default exit code for arg errors | - +-----------------+-------------------------------------------------------+ - | SCRIPT_FILENAME | file name of the hook script | - +-----------------+-------------------------------------------------------+ - | SCRIPT_NAME | name of the hook script (without file extension) | - +-----------------+-------------------------------------------------------+ - | lf | reference to a function that loads additional shell | - | | function files | - +-----------------+-------------------------------------------------------+ - - -*$FUNCTIONS* also provides a number of shell functions: - -.. code-block:: sh - - # --- message --- - # - # void veinfo ( message ) - # Prints a message to stdout if $DEBUG is set to 'y'. - # - # void einfo ( message ) - # Prints a message to stdout if $VERBOSE is set to 'y'. - # - # void ewarn ( message ) - # Prints a message to stderr unless $QUIET is set to 'y'. - # - # void eerror ( message ) - # Prints a message to stderr. - # - # - # --- core --- - # - # @noreturn die ( [message], [exit_code] ), raises exit() - # Lets the script die with the given message/exit code. - # - # @noreturn OUT_OF_BOUNDS(), raises die() - # Lets the script die due to insufficient arg count. - # - # int run_command ( *cmdv ) - # Logs a command and runs it afterwards. - # - # int run_command_logged ( *cmdv ) - # Logs a command, runs it and logs the result. - # - # void autodie ( *cmdv ), raises die() - # Runs a command. Lets the script die if the command fails. - # - # - # void load_functions ( *filenames, **SHLIB ), raises die() - # Loads additional shell functions file(s) from $SHLIB. - # (Referenced by $lf.) - # - # void dont_run_as_root(), raises die() - # Lets the script die if it is run as root. - # - # int list_has ( word, *list_items ) - # Returns 0 if $word is in the given list. - # - # int qwhich ( *command ) - # Returns 0 if all listed commands are found by "which". - # - # - # --- fs --- - # - # int dodir ( *dir ) - # Ensures that zero or more directories exist by creating them if - # necessary. Returns the number of directories that could not be created. - # - # - # --- str --- - # - # int yesno ( word, **YESNO_YES=0, **YESNO_NO=1, **YESNO_EMPTY=2 ) - # Returns $YESNO_YES if $word means "yes", $YESNO_EMPTY if $word is empty - # and $YESNO_NO otherwise (if $word probably means "no"). - # - # ~int str_trim ( *args ) - # Removes whitespace at the beginning and end of a string and replaces - # any whitespace sequence within the string with a single space char. - # Passes the return value of the underlying sed command. - # - # ~int str_upper ( *args ) - # Echoes the uppercase variant of stdin or *args. - # Passes tr's return value. - # - # ~int str_lower ( *args ) - # Echoes the lowercase variant of stdin or *args. - # Passes tr's return value. - # - # ~int str_field ( fieldspec, *args, **FIELD_SEPARATOR=' ' ) - # Echoes the requested fields of stdin or *args. - # Passes cut's return value. - # - # - # --- int --- - # - # int is_int ( word ) - # Returns 0 if $word is an integer, else 1. - # - # int is_natural ( word ) - # Returns 0 if $word is an integer >= 0, else 1. - # - # int is_positive ( word ) - # Returns 0 if $word is an integer >= 1, else 1. - # - # int is_negative ( word ) - # Returns 0 if $word is an integer < 0, else 1. - # - - -+++++++++++++++++++++ - Adding a hook event -+++++++++++++++++++++ - -in roverlay - -.. code-block:: python - - # import hook module - import roverlay.hook - - # ... - # then: - roverlay.hook.run ( "<event>" ) - # ... - roverlay.hook.run ( "<another event>" ) - - -in additions dir (assuming that mux.sh is used) - -.. code-block:: text - - <somehow create> ${ADDITIONS_DIR}/<hook>.<event> - <somehow create> ${ADDITIONS_DIR}/<hook>/<event>.sh
WARNING: multiple messages have this Message-ID (diff)
From: "André Erdmann" <dywi@mailerd.de> To: gentoo-commits@lists.gentoo.org Subject: [gentoo-commits] proj/R_overlay:master commit in: doc/rst/ Date: Sun, 30 Jun 2013 15:58:12 +0000 (UTC) [thread overview] Message-ID: <1372267719.0678580564dcf6aa8e2587d3b04d62544265ca74.dywi@gentoo> (raw) Message-ID: <20130630155812.pCKRhRrKKachEo3HhDQeuiETo7dTnI1Nq6-ngjSvmK4@z> (raw) commit: 0678580564dcf6aa8e2587d3b04d62544265ca74 Author: André Erdmann <dywi <AT> mailerd <DOT> de> AuthorDate: Wed Jun 26 17:28:39 2013 +0000 Commit: André Erdmann <dywi <AT> mailerd <DOT> de> CommitDate: Wed Jun 26 17:28:39 2013 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=06785805 doc/rst: "run hooks", "distmap" --- doc/rst/usage.rst | 643 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 383 insertions(+), 260 deletions(-) diff --git a/doc/rst/usage.rst b/doc/rst/usage.rst index 078a7ac..2daedce 100644 --- a/doc/rst/usage.rst +++ b/doc/rst/usage.rst @@ -1010,6 +1010,22 @@ A local directory will never be modified. consider using one of the **websync** repo types, websync_repo_ and websync_pkglist_. +--------- + distmap +--------- + +*roverlay* uses a text file to store information about files in the package +mirror directory (OVERLAY_DISTDIR_ROOT_). This is necessary for comparing +package files from repos with files for which an ebuild has already been +created (in previous runs). + +With the help of the *distmap file*, *roverlay* is able to determine whether +upstream has changed a package file silently and creates a revision-bumped +ebuild for the *new* package file. + +The *distmap file* can optionally be compressed (bzip2 or gzip), which +reduces its size considerably. + ===================== Additions Directory @@ -1790,6 +1806,354 @@ Moving such packages to a "R-package" sub directory would be possible, too: END; +============= + Event Hooks +============= + +*roverlay* is able to call a script when certain events occur, e.g. after +successful overlay creation, which can be used to perform additional actions +without touching *roverlay's* source code. + +To realize this, *roverlay* determines whether a given event is permitted +(`event policy`_) and, if so, creates a `hook environment`_ and runs the +script. Additionally, shell scripts can load *roverlay's* *$FUNCTIONS* file, +which provides extra functionality. + +.. Note:: + + *roverlay* waits until the script terminates and thus possibly waits + forever. + + +---------------------- + Default event script +---------------------- + +The default event script (``mux.sh``) loads *$FUNCTIONS* and then runs the +following script files (by sourcing them), in order: + +#. all files from ADDITIONS_DIR_/hooks/<event> that end with *.sh* + (``<ADDITIONS_DIR>/hooks/<event>/*.sh``) +#. all files ADDITIONS_DIR_/hooks that end with *.<event>* + (``<ADDITIONS_DIR>/hooks/*.<event>``) + +So, there are two naming schemes for *hook scripts*. +Either one is acceptable, but it is advised to stay consistent. +Having the same script at both locations results in executing it twice. + +.. Note:: + + The default event script enables *nounset* behavior, which causes the + shell command interpreter to exit abnormally if an unset variable is + accessed. + + +++++++++++++++++++++++++++ + Activating a hook script +++++++++++++++++++++++++++ + +Activating a hook script can be done by symlinking it: + +.. code-block:: text + + ln -s <real script> ${ADDITIONS_DIR}/hooks/<event>/<name>.sh + # or + ln -s <real script> ${ADDITIONS_DIR}/hooks/<name>.<event> + + +++++++++++++++++++++++++++ + Adding a new hook script +++++++++++++++++++++++++++ + +As hinted before, *hook scripts* are simple shell scripts. The following +template gives an idea of how to write them: + +.. code-block:: sh + + #!/bin/sh + #set -u + + # load essential functions + # (not necessary when using the default event script) + . "${FUNCTIONS?}" || exit + + ## load additional function files, if any + #$lf <name(s)> + + # script body + # + # when redirecting output to $DEVNULL, use ">>" instead of ">" as + # $DEVNULL could be a file + #ls >>${DEVNULL} + # + # ... + + + # the script must not exit if everything went well (return is ok) + return 0 + + +-------------- + Event Policy +-------------- + +The *event policy* controls whether a certain event actually triggers a script +call or not. +It is constructed by parsing the EVENT_HOOK_RESTRICT_ config option: + +* a word prefixed by ``-`` means *deny specific event* (-> blacklist) +* the asterisk char ``*`` (or ``+*``) sets the policy to + *allow unless denied* (blacklist) or *allow all* +* a word prefixed by ``+`` or without a prefix char means + *allow specific event* (-> whitelist) +* the asterisk char with ``-`` as prefix (``-*``) sets the policy to + *deny unless allowed* (whitelist) or *deny all* + + +The policy defaults to *allow all* if ``EVENT_HOOK_RESTRICT`` is not set in +the config file. An empty string sets the policy to *deny all*. + + +------------------ + Hook Environment +------------------ + +.. table:: environment variables provided by *roverlay* + + +----------------+-------------------+-----------------------------------------+ + | variable | source | notes / description | + +================+===================+=========================================+ + | PATH | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | LOGNAME | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | SHLVL | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | TERM | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | HOME | os.environ | | + +----------------+-------------------+-----------------------------------------+ + | ROVERLAY_PHASE | event | event that caused the script to run | + +----------------+-------------------+-----------------------------------------+ + | OVERLAY | config | overlay directory (`OVERLAY_DIR`_), | + +----------------+-------------------+ initial working directory | + | S | *$OVERLAY* | | + +----------------+-------------------+ | + | PWD | *$OVERLAY* | | + +----------------+-------------------+-----------------------------------------+ + | DISTROOT | config | package mirror directory | + | | | (`OVERLAY_DISTDIR_ROOT`_) | + +----------------+-------------------+-----------------------------------------+ + | TMPDIR | os.environ, | directory for temporary files | + | | *fallback* | | + +----------------+-------------------+ | + | T | *$TMPDIR* | | + +----------------+-------------------+-----------------------------------------+ + | ADDITIONS_DIR | config | directory with supplementary files | + | | | (`OVERLAY_ADDITIONS_DIR`_) | + +----------------+-------------------+ | + | FILESDIR | *$ADDITIONS_DIR* | | + +----------------+-------------------+-----------------------------------------+ + | SHLIB | *$ADDITIONS_DIR*, | A list of directories with shell | + | | *installed?* | function files | + | | | (optional, only set if any dir exists) | + +----------------+-------------------+-----------------------------------------+ + | FUNCTIONS | *$ADDITIONS_DIR*, | file with essential shell functions | + | | *installed?* | (optional, only set if it exists) | + +----------------+-------------------+-----------------------------------------+ + | DEBUG | log level | *shbool* (``y`` or ``n``) that | + | | | indicates whether debug messages should | + | | | be printed | + +----------------+-------------------+-----------------------------------------+ + | VERBOSE | log level | *shbool* | + +----------------+-------------------+-----------------------------------------+ + | QUIET | log level | *shbool* that indicates whether scripts | + | | | should be quiet | + +----------------+-------------------+-----------------------------------------+ + | NO_COLOR | *n/a* | *shbool*. Always set to *y* since | + | | | colored output should not be produced | + +----------------+-------------------+-----------------------------------------+ + | NOSYNC | config | *shbool* that indicates whether data | + | | | transfers from/to remote machines is | + | | | allowed (NOSYNC_) | + +----------------+-------------------+-----------------------------------------+ + | EBUILD | config | the *ebuild* executable | + +----------------+-------------------+-----------------------------------------+ + | GIT_EDITOR | *n/a* | set to */bin/false* | + +----------------+-------------------+ | + | GIT_ASKPASS | *n/a* | | + +----------------+-------------------+-----------------------------------------+ + + +The default *essential shell functions* file (*$FUNCTIONS*) makes, +when included in the hook script, most of the enviroment variables readonly. + + +.. table:: variables provided by *$FUNCTIONS* + + +-----------------+-------------------------------------------------------+ + | variable | description | + +=================+=======================================================+ + | IFS_DEFAULT | default *internal field separator* | + +-----------------+-------------------------------------------------------+ + | IFS_NEWLINE | *IFS* for iterating over text lines | + +-----------------+-------------------------------------------------------+ + | DEVNULL | */dev/null* target (could also be a file) | + +-----------------+-------------------------------------------------------+ + | EX_ERR | default error exit code | + +-----------------+-------------------------------------------------------+ + | EX_ARG_ERR | default exit code for arg errors | + +-----------------+-------------------------------------------------------+ + | SCRIPT_FILENAME | file name of the hook script | + +-----------------+-------------------------------------------------------+ + | SCRIPT_NAME | name of the hook script (without file extension) | + +-----------------+-------------------------------------------------------+ + | lf | reference to a function that loads additional shell | + | | function files | + +-----------------+-------------------------------------------------------+ + + +*$FUNCTIONS* also provides a number of shell functions: + +.. code-block:: sh + + # --- message --- + # + # void veinfo ( message ) + # Prints a message to stdout if $DEBUG is set to 'y'. + # + # void einfo ( message ) + # Prints a message to stdout if $VERBOSE is set to 'y'. + # + # void ewarn ( message ) + # Prints a message to stderr unless $QUIET is set to 'y'. + # + # void eerror ( message ) + # Prints a message to stderr. + # + # + # --- core --- + # + # @noreturn die ( [message], [exit_code] ), raises exit() + # Lets the script die with the given message/exit code. + # + # @noreturn OUT_OF_BOUNDS(), raises die() + # Lets the script die due to insufficient arg count. + # + # int run_command ( *cmdv ) + # Logs a command and runs it afterwards. + # + # int run_command_logged ( *cmdv ) + # Logs a command, runs it and logs the result. + # + # void autodie ( *cmdv ), raises die() + # Runs a command. Lets the script die if the command fails. + # + # + # void load_functions ( *filenames, **SHLIB ), raises die() + # Loads additional shell functions file(s) from $SHLIB. + # (Referenced by $lf.) + # + # void dont_run_as_root(), raises die() + # Lets the script die if it is run as root. + # + # int list_has ( word, *list_items ) + # Returns 0 if $word is in the given list, else 1. + # + # int qwhich ( *command ) + # Returns 0 if all listed commands are found by "which", else 1. + # + # + # --- fs --- + # + # int dodir ( *dir ) + # Ensures that zero or more directories exist by creating them if + # necessary. Returns the number of directories that could not be created. + # + # + # --- str --- + # + # int yesno ( word, **YESNO_YES=0, **YESNO_NO=1, **YESNO_EMPTY=2 ) + # Returns $YESNO_YES if $word means "yes", $YESNO_EMPTY if $word is empty + # and $YESNO_NO otherwise (if $word probably means "no"). + # + # ~int str_trim ( *args ) + # Removes whitespace at the beginning and end of a string and replaces + # any whitespace sequence within the string with a single space char. + # Passes the return value of the underlying sed command. + # + # ~int str_upper ( *args ) + # Echoes the uppercase variant of stdin or *args. + # Passes tr's return value. + # + # ~int str_lower ( *args ) + # Echoes the lowercase variant of stdin or *args. + # Passes tr's return value. + # + # ~int str_field ( fieldspec, *args, **FIELD_SEPARATOR=' ' ) + # Echoes the requested fields of stdin or *args. + # Passes cut's return value. + # + # + # --- int --- + # + # int is_int ( word ) + # Returns 0 if $word is an integer, else 1. + # + # int is_natural ( word ) + # Returns 0 if $word is an integer >= 0, else 1. + # + # int is_positive ( word ) + # Returns 0 if $word is an integer >= 1, else 1. + # + # int is_negative ( word ) + # Returns 0 if $word is an integer < 0, else 1. + # + + +------------------------ + Adding a function file +------------------------ + +Function files are shell script files that provide functions and variables. +They should, however, not execute any code directly. + +The template below illustrates how to write function files: + +.. code-block:: sh + + # protect against repeated inclusion of this file + # (replace <name> with a unique identifier) + if [ -z "${__HAVE_<name>__-}" ]; then + readonly __HAVE_<name>__=y + + # function file body + # ... + + fi + +Shell function files should be put into ``<ADDITIONS_DIR>/shlib``. + +--------------------- + Adding a hook event +--------------------- + +Adding a new event has to be done in *roverlay's* source code and is a rather +trivial task. The ``roverlay.hook`` module implements a function for running +the event script: + +.. code-block:: python + + # import hook module + import roverlay.hook + + # ... + # then, somewhere in the code + roverlay.hook.run ( "<event>" ) + # ... + roverlay.hook.run ( "<another event>" ) + + ========================= Configuration Reference ========================= @@ -1936,6 +2300,11 @@ DISTDIR_STRATEGY DISTDIR_VERIFY Alias to OVERLAY_DISTDIR_VERIFY_. +.. _DISTMAP_COMPRESSION: + +DISTMAP_COMPRESSION + Alias to OVERLAY_DISTMAP_COMPRESSION_. + .. _DISTMAP_FILE: DISTMAP_FILE @@ -2051,6 +2420,13 @@ OVERLAY_DISTDIR_VERIFY Defaults to *no* as the verification is normally not needed. +.. _OVERLAY_DISTMAP_COMPRESSION: + +OVERLAY_DISTMAP_COMPRESSION + Compression format for the distmap file. Choices are none, gzip/gz and + bzip2/bz2. + + Defaults to bzip2. .. _OVERLAY_DISTMAP_FILE: @@ -2212,9 +2588,10 @@ USE_EXPAND_RENAME .. _EVENT_HOOK: EVENT_HOOK - A script that is called for handling *events* (see Hooks_). + A script that is called for handling *events* (see `Event Hooks`_). - Defaults to ADDITIONS_DIR_/hooks/mux.sh. + Defaults to <libexec dir>/hooks/mux.sh if roverlay has been installed + and ADDITIONS_DIR_/hooks/mux.sh otherwise. .. _EVENT_HOOK_RESTRICT: @@ -2226,6 +2603,10 @@ EVENT_HOOK_RESTRICT Defaults to <not set>, which is equivalent to *deny all*. + ``EVENT_HOOK_RESTRICT="overlay_success"`` would allow the ``overlay_success`` + event only, whereas ``EVENT_HOOK_RESTRICT="* -overlay_success"`` would + allow any event except for ``overlay_success``. Also see `event policy`_. + .. _FILTER_SHELL_ENV: FILTER_SHELL_ENV @@ -3242,261 +3623,3 @@ Its mode of operation of operation is best described in pseudo-code: The dependency resolver can be **run as thread**, in which case the while loop becomes "loop until resolver closes". - - -======= - undef -======= - -<<TODO; this section has to merged/moved>> - ---------- - distmap ---------- - -The distmap is a file that stores information about the files in the package -mirror directory (`OVERLAY_DISTDIR_ROOT`_). -<<TODO>> - - -------- - hooks -------- - -*roverlay* calls a script when certain events occur, e.g. after successful -overlay creation. - -<<TODO>> - -* real work is offloaded to the called script -* roverlay provides some env vars -* roverlay provides a *$FUNCTIONS* file and a way to load other function files -* ``EVENT_HOOK_RESTRICT="this -not_that *"`` - -<<^^TODO>> - -.. Note:: - - *roverlay* waits until the script terminates. - - -++++++++++++++++++ - Hook environment -++++++++++++++++++ - -.. table:: environment variables provided by *roverlay* - - +----------------+------------------+-----------------------------------------+ - | variable | source | notes / description | - +================+==================+=========================================+ - | PATH | os.environ | | - +----------------+------------------+-----------------------------------------+ - | LOGNAME | os.environ | | - +----------------+------------------+-----------------------------------------+ - | SHLVL | os.environ | | - +----------------+------------------+-----------------------------------------+ - | TERM | os.environ | | - +----------------+------------------+-----------------------------------------+ - | HOME | os.environ | | - +----------------+------------------+-----------------------------------------+ - | ROVERLAY_PHASE | event | event that caused the script to run | - +----------------+------------------+-----------------------------------------+ - | OVERLAY | config | overlay directory (`OVERLAY_DIR`_), | - +----------------+------------------+ initial working directory | - | S | *$OVERLAY* | | - +----------------+------------------+ | - | PWD | *$OVERLAY* | | - +----------------+------------------+-----------------------------------------+ - | DISTROOT | config | package mirror directory | - | | | (`OVERLAY_DISTDIR_ROOT`_) | - +----------------+------------------+-----------------------------------------+ - | TMPDIR | os.environ, | directory for temporary files | - | | *fallback* | | - +----------------+------------------+ | - | T | *$TMPDIR* | | - +----------------+------------------+-----------------------------------------+ - | ADDITIONS_DIR | config | directory with supplementary files | - | | | (`OVERLAY_ADDITIONS_DIR`_) | - +----------------+------------------+ | - | FILESDIR | *$ADDITIONS_DIR* | | - +----------------+------------------+-----------------------------------------+ - | SHLIB | *$ADDITIONS_DIR* | directory with shell function files | - | | | (optional, only set if it exists) | - +----------------+------------------+-----------------------------------------+ - | FUNCTIONS | *$ADDITIONS_DIR* | file with essential shell functions | - | | | (optional, only set if it exists) | - +----------------+------------------+-----------------------------------------+ - | DEBUG | log level | *shbool* (``y`` or ``n``) that | - | | | indicates whether debug messages should | - | | | be printed | - +----------------+------------------+-----------------------------------------+ - | VERBOSE | log level | *shbool* | - +----------------+------------------+-----------------------------------------+ - | QUIET | log level | *shbool* that indicates whether scripts | - | | | should be quiet | - +----------------+------------------+-----------------------------------------+ - | NO_COLOR | *n/a* | *shbool*. Always set to *y* since | - | | | colored output should not be produced | - +----------------+------------------+-----------------------------------------+ - | NOSYNC | config | *shbool* that indicates whether data | - | | | transfers from/to remote machines is | - | | | allowed (NOSYNC_) | - +----------------+------------------+-----------------------------------------+ - | EBUILD | config | the *ebuild* executable | - +----------------+------------------+-----------------------------------------+ - | GIT_EDITOR | *n/a* | set to */bin/false* | - +----------------+------------------+ | - | GIT_ASKPASS | *n/a* | | - +----------------+------------------+-----------------------------------------+ - - -The default *essential shell functions* file (*$FUNCTIONS*) makes, -when included in the hook script, most of the enviroment variables readonly. - - -.. table:: variables provided by *$FUNCTIONS* - - +-----------------+-------------------------------------------------------+ - | variable | description | - +=================+=======================================================+ - | IFS_DEFAULT | default *internal field separator* | - +-----------------+-------------------------------------------------------+ - | IFS_NEWLINE | *IFS* for iterating over text lines | - +-----------------+-------------------------------------------------------+ - | DEVNULL | */dev/null* target (could also be a file) | - +-----------------+-------------------------------------------------------+ - | EX_ERR | default error exit code | - +-----------------+-------------------------------------------------------+ - | EX_ARG_ERR | default exit code for arg errors | - +-----------------+-------------------------------------------------------+ - | SCRIPT_FILENAME | file name of the hook script | - +-----------------+-------------------------------------------------------+ - | SCRIPT_NAME | name of the hook script (without file extension) | - +-----------------+-------------------------------------------------------+ - | lf | reference to a function that loads additional shell | - | | function files | - +-----------------+-------------------------------------------------------+ - - -*$FUNCTIONS* also provides a number of shell functions: - -.. code-block:: sh - - # --- message --- - # - # void veinfo ( message ) - # Prints a message to stdout if $DEBUG is set to 'y'. - # - # void einfo ( message ) - # Prints a message to stdout if $VERBOSE is set to 'y'. - # - # void ewarn ( message ) - # Prints a message to stderr unless $QUIET is set to 'y'. - # - # void eerror ( message ) - # Prints a message to stderr. - # - # - # --- core --- - # - # @noreturn die ( [message], [exit_code] ), raises exit() - # Lets the script die with the given message/exit code. - # - # @noreturn OUT_OF_BOUNDS(), raises die() - # Lets the script die due to insufficient arg count. - # - # int run_command ( *cmdv ) - # Logs a command and runs it afterwards. - # - # int run_command_logged ( *cmdv ) - # Logs a command, runs it and logs the result. - # - # void autodie ( *cmdv ), raises die() - # Runs a command. Lets the script die if the command fails. - # - # - # void load_functions ( *filenames, **SHLIB ), raises die() - # Loads additional shell functions file(s) from $SHLIB. - # (Referenced by $lf.) - # - # void dont_run_as_root(), raises die() - # Lets the script die if it is run as root. - # - # int list_has ( word, *list_items ) - # Returns 0 if $word is in the given list. - # - # int qwhich ( *command ) - # Returns 0 if all listed commands are found by "which". - # - # - # --- fs --- - # - # int dodir ( *dir ) - # Ensures that zero or more directories exist by creating them if - # necessary. Returns the number of directories that could not be created. - # - # - # --- str --- - # - # int yesno ( word, **YESNO_YES=0, **YESNO_NO=1, **YESNO_EMPTY=2 ) - # Returns $YESNO_YES if $word means "yes", $YESNO_EMPTY if $word is empty - # and $YESNO_NO otherwise (if $word probably means "no"). - # - # ~int str_trim ( *args ) - # Removes whitespace at the beginning and end of a string and replaces - # any whitespace sequence within the string with a single space char. - # Passes the return value of the underlying sed command. - # - # ~int str_upper ( *args ) - # Echoes the uppercase variant of stdin or *args. - # Passes tr's return value. - # - # ~int str_lower ( *args ) - # Echoes the lowercase variant of stdin or *args. - # Passes tr's return value. - # - # ~int str_field ( fieldspec, *args, **FIELD_SEPARATOR=' ' ) - # Echoes the requested fields of stdin or *args. - # Passes cut's return value. - # - # - # --- int --- - # - # int is_int ( word ) - # Returns 0 if $word is an integer, else 1. - # - # int is_natural ( word ) - # Returns 0 if $word is an integer >= 0, else 1. - # - # int is_positive ( word ) - # Returns 0 if $word is an integer >= 1, else 1. - # - # int is_negative ( word ) - # Returns 0 if $word is an integer < 0, else 1. - # - - -+++++++++++++++++++++ - Adding a hook event -+++++++++++++++++++++ - -in roverlay - -.. code-block:: python - - # import hook module - import roverlay.hook - - # ... - # then: - roverlay.hook.run ( "<event>" ) - # ... - roverlay.hook.run ( "<another event>" ) - - -in additions dir (assuming that mux.sh is used) - -.. code-block:: text - - <somehow create> ${ADDITIONS_DIR}/<hook>.<event> - <somehow create> ${ADDITIONS_DIR}/<hook>/<event>.sh
next reply other threads:[~2013-06-26 17:29 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2013-06-26 17:29 André Erdmann [this message] 2013-06-30 15:58 ` [gentoo-commits] proj/R_overlay:master commit in: doc/rst/ André Erdmann -- strict thread matches above, loose matches on Subject: below -- 2013-07-23 18:34 [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann 2013-07-23 14:57 André Erdmann 2013-07-16 16:35 André Erdmann 2013-07-15 22:31 André Erdmann 2013-07-03 10:05 [gentoo-commits] proj/R_overlay:master " André Erdmann 2013-06-28 17:18 ` [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann 2013-07-03 10:04 André Erdmann 2013-06-25 21:10 André Erdmann 2013-06-18 14:12 André Erdmann 2013-06-12 21:10 André Erdmann 2013-06-05 18:08 André Erdmann
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1372267719.0678580564dcf6aa8e2587d3b04d62544265ca74.dywi@gentoo \ --to=dywi@mailerd.de \ --cc=gentoo-commits@lists.gentoo.org \ --cc=gentoo-dev@lists.gentoo.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox