[gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 880 bytes --]

bin/isolated-functions.sh contains at least 1 useful function, which could be exposed for external consumers
(without __ prefix), but must have private name (with __ prefix) when bin/isolated-functions.sh is used in
ebuild environment.

Possible solutions:
1. Make /usr/lib/portage/bin/isolated-functions.sh magically define non-private variants of useful functions
   when run in non-ebuild environment.
2. Provide /usr/bin/portage.bash, which would source isolated-functions.sh (and maybe other files) and define
   non-private variants of useful functions. /usr/bin/portage.bash would be easier sourceable due to PATH.
3. Provide /usr/lib/portage.bash, which would source isolated-functions.sh (and maybe other files) and define
   non-private variants of useful functions.
4. Another location...

(I would probably prefer solution #2.)

--
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

On Sat, Feb 1, 2014 at 12:08 PM, Arfrever Frehtes Taifersar Arahesis <
arfrever.fta@gmail.com> wrote:

> bin/isolated-functions.sh contains at least 1 useful function, which could
> be exposed for external consumers
> (without __ prefix), but must have private name (with __ prefix) when
> bin/isolated-functions.sh is used in
> ebuild environment.
>
> Possible solutions:
> 1. Make /usr/lib/portage/bin/isolated-functions.sh magically define
> non-private variants of useful functions
>    when run in non-ebuild environment.
> 2. Provide /usr/bin/portage.bash, which would source isolated-functions.sh
> (and maybe other files) and define
>    non-private variants of useful functions. /usr/bin/portage.bash would
> be easier sourceable due to PATH.
> 3. Provide /usr/lib/portage.bash, which would source isolated-functions.sh
> (and maybe other files) and define
>    non-private variants of useful functions.
> 4. Another location...
>
> (I would probably prefer solution #2.)
>
> I think 1 isn't very unix-y, 2-3 are fine. If we expect users to run it,
it goes in bin, if not, then lib.

-A



> --
> Arfrever Frehtes Taifersar Arahesis
>

[-- Attachment #2: Type: text/html, Size: 1668 bytes --]

Re: [gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

On Saturday, February 01, 2014 21:08:11 Arfrever Frehtes Taifersar Arahesis 
wrote:
> bin/isolated-functions.sh contains at least 1 useful function, which could
> be exposed for external consumers (without __ prefix), but must have
> private name (with __ prefix) when bin/isolated-functions.sh is used in
> ebuild environment.

who are these mysterious external consumers ?

what you propose means all funcs in there now become exported API and we now 
have to live with that.  w/out any real background, i'm very hesitant to allow 
that.

> Possible solutions:
> 1. Make /usr/lib/portage/bin/isolated-functions.sh magically define
> non-private variants of useful functions when run in non-ebuild
> environment.

this is a no-go.  we should be cutting down on internal overhead.

> 2. Provide /usr/bin/portage.bash, which would source isolated-functions.sh
> (and maybe other files) and define non-private variants of useful
> functions. /usr/bin/portage.bash would be easier sourceable due to PATH.
> 3. Provide /usr/lib/portage.bash, which would source isolated-functions.sh
> (and maybe other files) and define non-private variants of useful
> functions.
> 4. Another location...
> 
> (I would probably prefer solution #2.)

sounds like a file that should be sourced only which imo bans it from $PATH.  
i'm aware of the magic shell behavior where `source` searches PATH, but that's 
not an acceptable reason imo.

easy to add something like `portageq helpers-dir` that'd give you a path and 
then you use that to load the various scripts directly.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: text/plain, Size: 3346 bytes --]

2014-02-05 09:08 Mike Frysinger napisał(a):
> On Saturday, February 01, 2014 21:08:11 Arfrever Frehtes Taifersar Arahesis 
> wrote:
> > bin/isolated-functions.sh contains at least 1 useful function, which could
> > be exposed for external consumers (without __ prefix), but must have
> > private name (with __ prefix) when bin/isolated-functions.sh is used in
> > ebuild environment.
> 
> who are these mysterious external consumers ?

E.g. euse, gentoo-bashcomp and potentially other tools written in bash, which need
some data from configuration of repositories.

I would like to provide 3 functions:
- repos_read_config()
  This function would use portageq to set PORTAGE_REPOSITORIES in environment.

  repos_read_config() {
    eval "$(portageq envvar -v PORTAGE_REPOSITORIES)"
  }

  If external consumer needs also other variables, then it can directly use:
  eval "$(portageq envvar -v PORTAGE_REPOSITORIES VAR1 VAR2 VAR3)"

- repos_list()
  This function would print list of names of repositories.

- repo_attr()
  This function would print value of specified attribute (${2}) for specified repository (${1}).
  This function would call __repo_attr() function currently defined in bin/isolated-functions.sh.

Example code:
  . portage.bash
  repos_read_config
  for repo in $(repos_list); do
    location=$(repo_attr ${repo} location)
    …
  done

Calling portageq multiple times would be too slow for some tools.

gentoo-bashcomp contains a custom implementation of parser of repos.conf, which could be replaced by
calls to functions provided by Portage:
http://git.overlays.gentoo.org/gitweb/?p=proj/gentoo-bashcomp.git;a=commitdiff;h=bf91db09a102544196b21cd792ac423262656a9a

> what you propose means all funcs in there now become exported API and we now 
> have to live with that.  w/out any real background, i'm very hesitant to allow 
> that.
> 
> > Possible solutions:
> > 1. Make /usr/lib/portage/bin/isolated-functions.sh magically define
> > non-private variants of useful functions when run in non-ebuild
> > environment.
> 
> this is a no-go.  we should be cutting down on internal overhead.
> 
> > 2. Provide /usr/bin/portage.bash, which would source isolated-functions.sh
> > (and maybe other files) and define non-private variants of useful
> > functions. /usr/bin/portage.bash would be easier sourceable due to PATH.
> > 3. Provide /usr/lib/portage.bash, which would source isolated-functions.sh
> > (and maybe other files) and define non-private variants of useful
> > functions.
> > 4. Another location...
> > 
> > (I would probably prefer solution #2.)
> 
> sounds like a file that should be sourced only which imo bans it from $PATH.  
> i'm aware of the magic shell behavior where `source` searches PATH, but that's 
> not an acceptable reason imo.

${EPREFIX}/usr/bin/portage.bash could be installed without +x permission.

${EPREFIX}/usr/bin is probably in ${PATH} in prefix environment, so ${EPREFIX}/usr/bin/portage.bash
might be easier to use than e.g. ${EPREFIX}/usr/lib/portage.bash.

> easy to add something like `portageq helpers-dir` that'd give you a path and 
> then you use that to load the various scripts directly.

It would result in calling portageq twice instead of once.

--
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-portage-dev] Re: [RFC] Location of Portage bash API (outside of ebuilds)

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 2232 bytes --]

On Wed, 5 Feb 2014 20:57:24 +0100
Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com> wrote:

> 2014-02-05 09:08 Mike Frysinger napisał(a):
> > On Saturday, February 01, 2014 21:08:11 Arfrever Frehtes Taifersar Arahesis 
> > wrote:
> > > bin/isolated-functions.sh contains at least 1 useful function, which could
> > > be exposed for external consumers (without __ prefix), but must have
> > > private name (with __ prefix) when bin/isolated-functions.sh is used in
> > > ebuild environment.
> > 
> > who are these mysterious external consumers ?
> 
> E.g. euse, gentoo-bashcomp and potentially other tools written in bash, which
> need some data from configuration of repositories.
> 
> I would like to provide 3 functions:
> - repos_read_config()
>   This function would use portageq to set PORTAGE_REPOSITORIES in environment.
> 
>   repos_read_config() {
>     eval "$(portageq envvar -v PORTAGE_REPOSITORIES)"
>   }
> 
>   If external consumer needs also other variables, then it can directly use:
>   eval "$(portageq envvar -v PORTAGE_REPOSITORIES VAR1 VAR2 VAR3)"
> 
> - repos_list()
>   This function would print list of names of repositories.
> 
> - repo_attr()
>   This function would print value of specified attribute (${2}) for specified
> repository (${1}). This function would call __repo_attr() function currently
> defined in bin/isolated-functions.sh.
> 
> Example code:
>   . portage.bash
>   repos_read_config
>   for repo in $(repos_list); do
>     location=$(repo_attr ${repo} location)
>     …
>   done
> 
> Calling portageq multiple times would be too slow for some tools.
> 
> gentoo-bashcomp contains a custom implementation of parser of repos.conf,
> which could be replaced by calls to functions provided by Portage:
> http://git.overlays.gentoo.org/gitweb/?p=proj/gentoo-bashcomp.git;a=commitdiff;h=bf91db09a102544196b21cd792ac423262656a9a

Calling portageq once is too slow for gentoo-bashcomp. :/  That parser is
terrible, but it's 1000x faster than going through portage.


-- 
Ryan Hill                        psn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-portage-dev] [RFC] Location of Portage bash API (outside of ebuilds)

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

On Wednesday, February 05, 2014 20:57:24 Arfrever wrote:
> Calling portageq multiple times would be too slow for some tools.

i think this is the fundamental point you're trying to side step/ignore.  
we've done work in the past to mitigate this.  e.g. `portageq envvar -v` 
allows you to list multiple env vars at once and eval the result so you only 
have to query it once.

if you think the existing portageq API is insufficient, then propose 
extensions/new helpers.

if portageq is still too slow, then we should be figuring out how to speed 
that up.  that would help out everyone.

i don't think exporting bash funcs is the answer to any of the issues you're 
looking at here.  we should not be writing two sets of APIs, especially when 
one is a speed hack to get around limitations in the other implementation.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]