From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path:
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
+
+
-
Example: USE_EXPAND_RENAME = ~/roverlay/config/useflag_rename
+Directory for generated files that do not belong to the overlay, e.g. +the distmap file. This option is required.
+Example: CACHEDIR = ~/roverlay/workdir/cache
+This option lists eclass files that should be imported into the overlay (into OVERLAY_DIR/eclass/) and inherited in all ebuilds. @@ -799,6 +818,11 @@ to know in detail what roverlay does before running it.
roverlay uses a text file to store information about files in the package +mirror directory (OVERLAY_DISTDIR_ROOT). This is necessary for comparing +package files from repos with files for which an ebuild has already been +created (in previous runs).
+With the help of the distmap file, roverlay is able to determine whether +upstream has changed a package file silently and creates a revision-bumped +ebuild for the new package file.
+The distmap file can optionally be compressed (bzip2 or gzip), which +reduces its size considerably.
+roverlay is able to call a script when certain events occur, e.g. after +successful overlay creation, which can be used to perform additional actions +without touching roverlay's source code.
+To realize this, roverlay determines whether a given event is permitted +(event policy) and, if so, creates a hook environment and runs the +script. Additionally, shell scripts can load roverlay's $FUNCTIONS file, +which provides extra functionality.
+Note
+roverlay waits until the script terminates and thus possibly waits +forever.
+The default event script (mux.sh) loads $FUNCTIONS and then runs the +following script files (by sourcing them), in order:
+So, there are two naming schemes for hook scripts. +Either one is acceptable, but it is advised to stay consistent. +Having the same script at both locations results in executing it twice.
+Note
+The default event script enables nounset behavior, which causes the +shell command interpreter to exit abnormally if an unset variable is +accessed.
+Activating a hook script can be done by symlinking it:
++ln -s <real script> ${ADDITIONS_DIR}/hooks/<event>/<name>.sh +# or +ln -s <real script> ${ADDITIONS_DIR}/hooks/<name>.<event> ++
As hinted before, hook scripts are simple shell scripts. The following +template gives an idea of how to write them:
++#!/bin/sh +#set -u + +# load essential functions +# (not necessary when using the default event script) +. "${FUNCTIONS?}" || exit + +## load additional function files, if any +#$lf <name(s)> + +# script body +# +# when redirecting output to $DEVNULL, use ">>" instead of ">" as +# $DEVNULL could be a file +#ls >>${DEVNULL} +# +# ... + + +# the script must not exit if everything went well (return is ok) +return 0 ++
The event policy controls whether a certain event actually triggers a script +call or not. +It is constructed by parsing the EVENT_HOOK_RESTRICT config option:
+The policy defaults to allow all if EVENT_HOOK_RESTRICT is not set in +the config file. An empty string sets the policy to deny all.
+variable | +source | +notes / description | +
---|---|---|
PATH | +os.environ | ++ |
LOGNAME | +os.environ | ++ |
SHLVL | +os.environ | ++ |
TERM | +os.environ | ++ |
HOME | +os.environ | ++ |
ROVERLAY_PHASE | +event | +event that caused the script to run | +
OVERLAY | +config | +overlay directory (OVERLAY_DIR), +initial working directory | +
S | +$OVERLAY | +|
PWD | +$OVERLAY | +|
DISTROOT | +config | +package mirror directory +(OVERLAY_DISTDIR_ROOT) | +
TMPDIR | +os.environ, +fallback | +directory for temporary files | +
T | +$TMPDIR | +|
ADDITIONS_DIR | +config | +directory with supplementary files +(OVERLAY_ADDITIONS_DIR) | +
FILESDIR | +$ADDITIONS_DIR | +|
SHLIB | +$ADDITIONS_DIR, +installed? | +A list of directories with shell +function files +(optional, only set if any dir exists) | +
FUNCTIONS | +$ADDITIONS_DIR, +installed? | +file with essential shell functions +(optional, only set if it exists) | +
DEBUG | +log level | +shbool (y or n) that +indicates whether debug messages should +be printed | +
VERBOSE | +log level | +shbool | +
QUIET | +log level | +shbool that indicates whether scripts +should be quiet | +
NO_COLOR | +n/a | +shbool. Always set to y since +colored output should not be produced | +
NOSYNC | +config | +shbool that indicates whether data +transfers from/to remote machines is +allowed (NOSYNC) | +
EBUILD | +config | +the ebuild executable | +
GIT_EDITOR | +n/a | +set to /bin/false | +
GIT_ASKPASS | +n/a | +
The default essential shell functions file ($FUNCTIONS) makes, +when included in the hook script, most of the enviroment variables readonly.
+variable | +description | +
---|---|
IFS_DEFAULT | +default internal field separator | +
IFS_NEWLINE | +IFS for iterating over text lines | +
DEVNULL | +/dev/null target (could also be a file) | +
EX_ERR | +default error exit code | +
EX_ARG_ERR | +default exit code for arg errors | +
SCRIPT_FILENAME | +file name of the hook script | +
SCRIPT_NAME | +name of the hook script (without file extension) | +
lf | +reference to a function that loads additional shell +function files | +
$FUNCTIONS also provides a number of shell functions:
+
+# --- message ---
+#
+# void veinfo ( message )
+# Prints a message to stdout if $DEBUG is set to 'y'.
+#
+# void einfo ( message )
+# Prints a message to stdout if $VERBOSE is set to 'y'.
+#
+# void ewarn ( message )
+# Prints a message to stderr unless $QUIET is set to 'y'.
+#
+# void eerror ( message )
+# Prints a message to stderr.
+#
+#
+# --- core ---
+#
+# @noreturn die ( [message], [exit_code] ), raises exit()
+# Lets the script die with the given message/exit code.
+#
+# @noreturn OUT_OF_BOUNDS(), raises die()
+# Lets the script die due to insufficient arg count.
+#
+# int run_command ( *cmdv )
+# Logs a command and runs it afterwards.
+#
+# int run_command_logged ( *cmdv )
+# Logs a command, runs it and logs the result.
+#
+# void autodie ( *cmdv ), raises die()
+# Runs a command. Lets the script die if the command fails.
+#
+#
+# void load_functions ( *filenames, **SHLIB ), raises die()
+# Loads additional shell functions file(s) from $SHLIB.
+# (Referenced by $lf.)
+#
+# void dont_run_as_root(), raises die()
+# Lets the script die if it is run as root.
+#
+# int list_has ( word, *list_items )
+# Returns 0 if $word is in the given list, else 1.
+#
+# int qwhich ( *command )
+# Returns 0 if all listed commands are found by "which", else 1.
+#
+#
+# --- fs ---
+#
+# int dodir ( *dir )
+# Ensures that zero or more directories exist by creating them if
+# necessary. Returns the number of directories that could not be created.
+#
+#
+# --- str ---
+#
+# int yesno ( word, **YESNO_YES=0, **YESNO_NO=1, **YESNO_EMPTY=2 )
+# Returns $YESNO_YES if $word means "yes", $YESNO_EMPTY if $word is empty
+# and $YESNO_NO otherwise (if $word probably means "no").
+#
+# ~int str_trim ( *args )
+# Removes whitespace at the beginning and end of a string and replaces
+# any whitespace sequence within the string with a single space char.
+# Passes the return value of the underlying sed command.
+#
+# ~int str_upper ( *args )
+# Echoes the uppercase variant of stdin or *args.
+# Passes tr's return value.
+#
+# ~int str_lower ( *args )
+# Echoes the lowercase variant of stdin or *args.
+# Passes tr's return value.
+#
+# ~int str_field ( fieldspec, *args, **FIELD_SEPARATOR=' ' )
+# Echoes the requested fields of stdin or *args.
+# Passes cut's return value.
+#
+#
+# --- int ---
+#
+# int is_int ( word )
+# Returns 0 if $word is an integer, else 1.
+#
+# int is_natural ( word )
+# Returns 0 if $word is an integer >= 0, else 1.
+#
+# int is_positive ( word )
+# Returns 0 if $word is an integer >= 1, else 1.
+#
+# int is_negative ( word )
+# Returns 0 if $word is an integer < 0, else 1.
+#
+
+Function files are shell script files that provide functions and variables. +They should, however, not execute any code directly.
+The template below illustrates how to write function files:
++# protect against repeated inclusion of this file +# (replace <name> with a unique identifier) +if [ -z "${__HAVE_<name>__-}" ]; then +readonly __HAVE_<name>__=y + +# function file body +# ... + +fi ++
Shell function files should be put into <ADDITIONS_DIR>/shlib.
+Adding a new event has to be done in roverlay's source code and is a rather +trivial task. The roverlay.hook module implements a function for running +the event script:
++# import hook module +import roverlay.hook + +# ... +# then, somewhere in the code +roverlay.hook.run ( "<event>" ) +# ... +roverlay.hook.run ( "<another event>" ) ++
The main config file uses '<option> = <value>' syntax, comment lines start with #. Variable substitution ("${X}") is not supported. Quotes around the value are optional and allow to span long values over multiple lines. @@ -2191,7 +2611,15 @@ restrictions. Commenting it out is safe.
The following sections will list all config entries.
Directory for persistent files that don't belong to the overlay, e.g. +the distmap file.
+This option is required.
+<<TODO: default value!>>
+Defaults to ebuild, which should be fine in most cases.
A bool that controls whether syncing, i.e. data transfers from/to +remote machines, is allowed or forbidden.
+Defaults to no.
+Set a max. average bandwidth usage in kilobytes per second. @@ -2226,7 +2661,7 @@ This will pass '--bwlimit=<value>' to all rsync commands.
A bool that controls whether file integrity of OVERLAY_DISTDIR_ROOT +should be checked on startup. This is an expensive operation since each +file have to be read once.
+Defaults to no as the verification is normally not needed.
+Compression format for the distmap file. Choices are none, gzip/gz and +bzip2/bz2.
+Defaults to bzip2.
+File path to the distmap file.
+Defaults to <not set>, which results in CACHEDIR/distmap.db.
+A list of eclass files that will be imported into the overlay and inherited @@ -2398,7 +2866,7 @@ writing.
Some config config options are split from the main config file for various reasons:
A script that is called for handling events (see Event Hooks).
+Defaults to <libexec dir>/hooks/mux.sh if roverlay has been installed +and ADDITIONS_DIR/hooks/mux.sh otherwise.
+A list of events that are allowed (<event>, +<event>) or +forbidden (-<event>). The asterisk wildcard character can be used to +set the default policy to allow unless forbidden (*) or +deny unless allowed (-*).
+Defaults to <not set>, which is equivalent to deny all.
+EVENT_HOOK_RESTRICT="overlay_success" would allow the overlay_success +event only, whereas EVENT_HOOK_RESTRICT="* -overlay_success" would +allow any event except for overlay_success. Also see event policy.
+A bool that controls whether the hook environment should be filtered +or not.
+Defaults to true.
+The date format (ISO8601) used in log messages.
@@ -2502,7 +3007,7 @@ have their own log level.Enables/Disables logging to console. The value has to be a bool.
@@ -2523,7 +3028,7 @@ have their own log level.Sets the log file. File logging will be disabled if this option does not @@ -2582,7 +3087,7 @@ files will be kept.
A directory where all description data read from an R package will be @@ -2601,9 +3106,9 @@ on roverlay exit. Primarily useful for creating new rules.
The USE_EXPAND_RENAME file contains dictionary-like entries that assign effective flag names to flag names generated at runtime.
The syntax is as follows:
@@ -2637,7 +3142,7 @@ unpredictable results.The field definition file uses ConfigParser syntax and defines how an R package's DESCRIPTION file is read. See the next section, default field definition file, for an example.
@@ -2719,7 +3224,7 @@ such a field is found.It is not checked whether a flag is known or not.
This is the default field definition file (without any ignored fields):
[Description]
@@ -2756,7 +3261,7 @@ such a field is found.
As previously stated, the DepRes Console is only meant for testing. It is an interactive console with the following features:
This chapter gives an in-depth overview of how roverlay works. Code documentation is also available and html pages for it can be created with make pydoc in the R Overlay src directory or by using pydoc directly.
PackageInfo is the data object that contains all information about an R package and is created by the owning repository.
After initialization it contains data like
@@ -2941,7 +3446,7 @@ for it. Otherwise, p is now part of the overlay and has to pass ebuild creation.Repositories are managed in a list-like object, RepoList. Its task is to provide R package input for overlay creation and implements the following functionality:
@@ -2952,7 +3457,7 @@ functionality:The functionality described above is an abstraction layer that calls the respective function for each repository and collects the result. So, while the RepoList object knows what to do for all repositories, @@ -2993,7 +3498,7 @@ subclass-specifc _sync()/_nosync() functions if available. repository types, rsync, websync_repo and websync_pkglist derive from BasicRepo.
Adding new repository types is best done by creating a new repo class that inherits BasicRepo. The table below shows BasicRepo's subclass awareness concerning sync() and what may be changed if required. @@ -3080,7 +3585,7 @@ to be accessible.
The overlay is roverlay's central data structure that represents a portage overlay. It is organized in a tree structure (overlay root, categories, package directories) and implements all overlay-related functionality:
@@ -3111,7 +3616,7 @@ levelmetadata.xml files are created with a tree structure that contains metadata nodes, e.g. '<pkgmetadata>...</pkgmetadata>' and '<use>...</use>' are nodes. The current implementation writes the R package's full description @@ -3120,7 +3625,7 @@ Metadata creation uses the latest package, i.e. highest version, for which an ebuild has been created.
Manifest files are created by calling the ebuild executable for each package directory in a filtered environment where FETCHCOMMAND and RESUMECOMMAND are set to no-operation. The directories that contain the R @@ -3129,7 +3634,7 @@ is set to DISTFILES_ROOT
Ebuild creation is the process centered around one PackageInfo instance p that tries to create an ebuild for it.
It does the following steps:
@@ -3148,7 +3653,7 @@ order to save memory etc. as ebuild successfully createdEach ebuild variable is an object whose class is derived from the EbuildVar class. An EbuildVar defines its position in the ebuild and how its text output should look like. Ebuild text is created by adding ebuild variables @@ -3156,7 +3661,7 @@ to an Ebuilder that automatically sorts them and creates the ebuild.
Overlay creation is the process of creating an overlay for many R packages and roverlay's main task. More specifically, OverlayCreation is an R packages -> Overlay (-> overlay in filesystem) interface. @@ -3170,7 +3675,7 @@ instead the event is logged. Running more than one OverlayWorker in par is possible.
Each ebuild creation process has access to the dependency resolver that accepts dependency strings, tries to resolve them and returns the result, either "unresolvable" or the resolving dependency as @@ -3192,7 +3697,7 @@ all required dependencies. No ebuild can be created in that case.
Details about dependency resolution like how channels work can be found in the following subsections.
Every dependency string has a dependency type that declares how a dependency should be resolved. It has one or more of these properties:
The DESCRIPTION file of an R package contains several fields that list dependencies on R packages or other software like scientific libraries. This section describes which dependency fields are used and how.
@@ -3280,7 +3785,7 @@ but optional dependencies during the pkg_postinst() ebuild phase.A dependency environment is an object that reflects the whole dependency resolution process of a single dependency string. It usually contains the dependency string, its dependency type, information about its @@ -3290,7 +3795,7 @@ resolving resolving dependency, if any.
dependency resolver.The EbuildJob Channel is used by the ebuild creation to communicate with the dependency resolver. It is initialized by an ebuild creation process and realizes a greedy string to string dependency resolution.
@@ -3327,7 +3832,7 @@ dependencies are unresolvable.The dependency resolver does not know how to resolve a dependency string. Instead, it searches through a list of dependency rule pools that may be able to do this.
@@ -3350,7 +3855,7 @@ R package dependencies. By convention, it will never resolve any system dependencies.The dependency resolver can be extended by modules. Two base types are available, channels and listeners.
The dependency resolver puts all parts of dependency resolution together, rule pools, listeners and channels. Its main task is a loop that processes all queued dependency environments and sends the result back to @@ -3428,7 +3933,7 @@ becomes "loop until resolver closes".