[gentoo-science] sage queues

[gentoo-science] sage queues

[VulK]

Dear all,
this is my first post to gentoo-science and I am writing because I have some
problems running experimental code from the sage project.
My issue is the following:
I have sci-mathematics/sage-4.7-r2 installed from the  sage-on-gentoo overlay
and I would like to install the combinat queue; I am following these
instructions: http://wiki.sagemath.org/combinat/MercurialStepByStep
The command I am supposed to run is 
# sage -combinat install
unfortunately -combinat is not recognized by sage as a valid option. I
browsed a little bit around the filesystem and I noticed that $SAGE_ROOT is
empty (except for some documentation) while on other installations of sage
(not using the ebuilds) there is plenty of stuff including a devel/combinat
folder.
Is there an option I can use when installing sage to allow for experimental
sources? or is there any other way I can use queues without installing sage
not using portage?
Thanks
VulK

PS: some weird behaviour:

% sage -h
----------------------------------------------------------------------
| Sage Version 4.7, Release Date: 2011-05-23                         |
----------------------------------------------------------------------

Optional arguments:
  file.<sage|py|spyx> -- run given .sage, .py or .spyx files
  -advanced           -- list all command line options
  -c <cmd>            -- Evaluates cmd as sage code
  -experimental       -- list all experimental packages that can be installed
  -gap [...]          -- run Sage's Gap with given arguments
  -gp [...]           -- run Sage's PARI/GP calculator with given arguments
  -h, -?              -- print this help message
  -i [packages]       -- install the given Sage packages
  -inotebook [...]    -- start the *insecure* Sage notebook
  -maxima [...]       -- run Sage's Maxima with given arguments
  -mwrank [...]       -- run Sage's mwrank with given arguments
  -n, -notebook [...] -- start the Sage notebook (options are the same
                         as for the notebook command in Sage)
  -optional           -- list all optional packages that can be installed
  -python [...]       -- run the Python interpreter
  -R [...]            -- run Sage's R with given arguments
  -singular [...]     -- run Sage's singular with given arguments
  -root               -- print the Sage root directory
  -t [options] <files|dir>
                      -- test examples in .py, .pyx, .sage or .tex files
                         options:
                           -long  -- include lines with the phrase 'long time'
                           -verbose  -- print debugging output during the test
                           -optional  -- also test all #optional examples
                           -only-optional <tag1,...,tagn>  -- only run tests
                            including one of the #optional tags
                           -randorder[=seed]  -- randomize order of tests
  -v, -version        -- print the Sage version

% sage -experimental
sage-run received unknown option: -experimental 
usage: sage [options]
Try 'sage -h' for more information.

Re: [gentoo-science] sage queues

[fbissey]

Quoting VulK <etn45p4m@gmail.com>:

> Dear all,
> this is my first post to gentoo-science and I am writing because I have some
> problems running experimental code from the sage project.
> My issue is the following:
> I have sci-mathematics/sage-4.7-r2 installed from the  sage-on-gentoo overlay
> and I would like to install the combinat queue; I am following these
> instructions: http://wiki.sagemath.org/combinat/MercurialStepByStep
> The command I am supposed to run is
> # sage -combinat install
> unfortunately -combinat is not recognized by sage as a valid option. I
> browsed a little bit around the filesystem and I noticed that $SAGE_ROOT is
> empty (except for some documentation) while on other installations of sage
> (not using the ebuilds) there is plenty of stuff including a devel/combinat
> folder.
> Is there an option I can use when installing sage to allow for experimental
> sources? or is there any other way I can use queues without installing sage
> not using portage?
> Thanks
> VulK
>
> PS: some weird behaviour:
>
> % sage -h
> ----------------------------------------------------------------------
> | Sage Version 4.7, Release Date: 2011-05-23                         |
> ----------------------------------------------------------------------
>
> Optional arguments:
>   file.<sage|py|spyx> -- run given .sage, .py or .spyx files
>   -advanced           -- list all command line options
>   -c <cmd>            -- Evaluates cmd as sage code
>   -experimental       -- list all experimental packages that can be installed
>   -gap [...]          -- run Sage's Gap with given arguments
>   -gp [...]           -- run Sage's PARI/GP calculator with given arguments
>   -h, -?              -- print this help message
>   -i [packages]       -- install the given Sage packages
>   -inotebook [...]    -- start the *insecure* Sage notebook
>   -maxima [...]       -- run Sage's Maxima with given arguments
>   -mwrank [...]       -- run Sage's mwrank with given arguments
>   -n, -notebook [...] -- start the Sage notebook (options are the same
>                          as for the notebook command in Sage)
>   -optional           -- list all optional packages that can be installed
>   -python [...]       -- run the Python interpreter
>   -R [...]            -- run Sage's R with given arguments
>   -singular [...]     -- run Sage's singular with given arguments
>   -root               -- print the Sage root directory
>   -t [options] <files|dir>
>                       -- test examples in .py, .pyx, .sage or .tex files
>                          options:
>                            -long  -- include lines with the phrase 
> 'long time'
>                            -verbose  -- print debugging output during 
> the test
>                            -optional  -- also test all #optional examples
>                            -only-optional <tag1,...,tagn>  -- only run tests
>                             including one of the #optional tags
>                            -randorder[=seed]  -- randomize order of tests
>   -v, -version        -- print the Sage version
>
> % sage -experimental
> sage-run received unknown option: -experimental
> usage: sage [options]
> Try 'sage -h' for more information.

Hi VuLK,

unfortunately at this stage we do not support that in sage-on-gentoo.
Actually the version we ship is stripped down in some ways.
Let me explain:
sage has its own upgrade system, it wouldn't work in the kind of 
installation we
do and that would mean changing, adding and deleting files in the 
system outside
the control of the package manager. We definitely don't want to do that. So we
removed the options for sage upgrade. The only option to upgrade is
portage/package-core etc...

There are options to help you create spkg, install spkg and so on, we could
probably give back the one to create spkg but we otherwise completely
circumvent  the sage build system so the corresponding options are gone.

The main problem is that sage's normal distribution model is trying to be
developer friendly but isn't distro friendly. We coerced it into a 
distro which
makes it more appealing for an end user to try but it is stripped of 
some of the
dev-friendly features.

There are advantages and disadvantages for both models. We can be/are 
more up to
date than sage with some packages. If I patch something I literally have to
reinstall the whole of of the sage spkg from portage, the equivalent of sage
-ba while from vanilla sage you could use sage -b and only rebuild the
necessary bits.

Now you are the first person making this kind of request about using something
like the combinat queue. We probably can give you an ebuild pulling the
combinat queue. There are just two caveats here:
1) it may take a bit of time for us to come up with something.
2) because I expect the queue to be somewhat in flux it would have to be a hot
ebuild of some kind. If you can live with that we can probably work something
out.

Francois

Re: [gentoo-science] sage queues

[VulK]

Hi,
Thank you for the explanation: I kind of guessed that some part of sage were
omitted to adapt the two packaging system but your explanation gave me the
details I was missing. 
As you said the combinat queue is/should be a real mess of continuous
updates (at least this is what I was told) so I am not entirely sure how well
an e-build would perform, in case you decide to spend some time on it I will
gladly be a guinea pig for testing it out.
I do not understand sage package system in details so my request may just be
stupid but is it possible to produce separate ebuilds for the different part 
of sage that are now stripped? If not for all of those can this be done for the
various packages in $SAGE_ROOT/devel ? If an e-build is not feasible, can
USE flags be used to select which extensions to include at compile time?
Thank you
S.


* fbissey@slingshot.co.nz <fbissey@slingshot.co.nz> [2011-08-09 16:29:52]:

> Quoting VulK <etn45p4m@gmail.com>:
> 
> > Dear all,
> > this is my first post to gentoo-science and I am writing because I have some
> > problems running experimental code from the sage project.
> > My issue is the following:
> > I have sci-mathematics/sage-4.7-r2 installed from the  sage-on-gentoo overlay
> > and I would like to install the combinat queue; I am following these
> > instructions: http://wiki.sagemath.org/combinat/MercurialStepByStep
> > The command I am supposed to run is
> > # sage -combinat install
> > unfortunately -combinat is not recognized by sage as a valid option. I
> > browsed a little bit around the filesystem and I noticed that $SAGE_ROOT is
> > empty (except for some documentation) while on other installations of sage
> > (not using the ebuilds) there is plenty of stuff including a devel/combinat
> > folder.
> > Is there an option I can use when installing sage to allow for experimental
> > sources? or is there any other way I can use queues without installing sage
> > not using portage?
> > Thanks
> > VulK
> >
> > PS: some weird behaviour:
> >
> > % sage -h
> > ----------------------------------------------------------------------
> > | Sage Version 4.7, Release Date: 2011-05-23                         |
> > ----------------------------------------------------------------------
> >
> > Optional arguments:
> >   file.<sage|py|spyx> -- run given .sage, .py or .spyx files
> >   -advanced           -- list all command line options
> >   -c <cmd>            -- Evaluates cmd as sage code
> >   -experimental       -- list all experimental packages that can be installed
> >   -gap [...]          -- run Sage's Gap with given arguments
> >   -gp [...]           -- run Sage's PARI/GP calculator with given arguments
> >   -h, -?              -- print this help message
> >   -i [packages]       -- install the given Sage packages
> >   -inotebook [...]    -- start the *insecure* Sage notebook
> >   -maxima [...]       -- run Sage's Maxima with given arguments
> >   -mwrank [...]       -- run Sage's mwrank with given arguments
> >   -n, -notebook [...] -- start the Sage notebook (options are the same
> >                          as for the notebook command in Sage)
> >   -optional           -- list all optional packages that can be installed
> >   -python [...]       -- run the Python interpreter
> >   -R [...]            -- run Sage's R with given arguments
> >   -singular [...]     -- run Sage's singular with given arguments
> >   -root               -- print the Sage root directory
> >   -t [options] <files|dir>
> >                       -- test examples in .py, .pyx, .sage or .tex files
> >                          options:
> >                            -long  -- include lines with the phrase 
> > 'long time'
> >                            -verbose  -- print debugging output during 
> > the test
> >                            -optional  -- also test all #optional examples
> >                            -only-optional <tag1,...,tagn>  -- only run tests
> >                             including one of the #optional tags
> >                            -randorder[=seed]  -- randomize order of tests
> >   -v, -version        -- print the Sage version
> >
> > % sage -experimental
> > sage-run received unknown option: -experimental
> > usage: sage [options]
> > Try 'sage -h' for more information.
> 
> Hi VuLK,
> 
> unfortunately at this stage we do not support that in sage-on-gentoo.
> Actually the version we ship is stripped down in some ways.
> Let me explain:
> sage has its own upgrade system, it wouldn't work in the kind of 
> installation we
> do and that would mean changing, adding and deleting files in the 
> system outside
> the control of the package manager. We definitely don't want to do that. So we
> removed the options for sage upgrade. The only option to upgrade is
> portage/package-core etc...
> 
> There are options to help you create spkg, install spkg and so on, we could
> probably give back the one to create spkg but we otherwise completely
> circumvent  the sage build system so the corresponding options are gone.
> 
> The main problem is that sage's normal distribution model is trying to be
> developer friendly but isn't distro friendly. We coerced it into a 
> distro which
> makes it more appealing for an end user to try but it is stripped of 
> some of the
> dev-friendly features.
> 
> There are advantages and disadvantages for both models. We can be/are 
> more up to
> date than sage with some packages. If I patch something I literally have to
> reinstall the whole of of the sage spkg from portage, the equivalent of sage
> -ba while from vanilla sage you could use sage -b and only rebuild the
> necessary bits.
> 
> Now you are the first person making this kind of request about using something
> like the combinat queue. We probably can give you an ebuild pulling the
> combinat queue. There are just two caveats here:
> 1) it may take a bit of time for us to come up with something.
> 2) because I expect the queue to be somewhat in flux it would have to be a hot
> ebuild of some kind. If you can live with that we can probably work something
> out.
> 
> Francois
> 
> 
> 
> 

Re: [gentoo-science] sage queues

[fbissey]

Quoting VulK <etn45p4m@gmail.com>:

> Hi,
> Thank you for the explanation: I kind of guessed that some part of sage were
> omitted to adapt the two packaging system but your explanation gave me the
> details I was missing.
> As you said the combinat queue is/should be a real mess of continuous
> updates (at least this is what I was told) so I am not entirely sure how well
> an e-build would perform, in case you decide to spend some time on it I will
> gladly be a guinea pig for testing it out.
> I do not understand sage package system in details so my request may just be
> stupid but is it possible to produce separate ebuilds for the different part
> of sage that are now stripped? If not for all of those can this be 
> done for the
> various packages in $SAGE_ROOT/devel ? If an e-build is not feasible, can
> USE flags be used to select which extensions to include at compile time?
>
The details are a bit long to explain but everything provided by sage is
currently split. Technically what is missing is some scripts from the spkg
sage_scripts (provided by our sage-baselayout ebuild). Most of the files in
$SAGE_LOCAL/bin of a vanilla install that starts with sage are provided 
by this
spkg. And we omit a lot of them, some are already installed only on use flag
request. We could add more if it was useful and feasible from a package
management perspective.

I must say that talking with sage-developers interested in sage installed with
portage there is a possibility that some stuff may come back in some form once
we figured it out. Something like sage -combinat creates a new sage branch.
There is a possibility that we could allow such a branch to be created 
inside a
user account (not system wide) and allow its use. But that's still some 
way off
on my map. In fact it may come as a surprise to my fellow sage-on-gentoo devs.

Francois

Re: [gentoo-science] sage queues

[Christopher Schwan]

On Tuesday 09 August 2011 01:07:44 you wrote:
> Hi,
> Thank you for the explanation: I kind of guessed that some part of sage were
> omitted to adapt the two packaging system but your explanation gave me the
> details I was missing.
> As you said the combinat queue is/should be a real mess of continuous
> updates (at least this is what I was told) so I am not entirely sure how
> well an e-build would perform, in case you decide to spend some time on it
> I will gladly be a guinea pig for testing it out.
> I do not understand sage package system in details so my request may just be
> stupid but is it possible to produce separate ebuilds for the different
> part of sage that are now stripped? If not for all of those can this be
> done for the various packages in $SAGE_ROOT/devel ? If an e-build is not
> feasible, can USE flags be used to select which extensions to include at
> compile time? Thank you
> S.

If I understand you correctly, you are asking if it is possible to further 
split up the sage ebuild, so that we have individual ebuilds for 
sage.combinat, sage.interfaces and so on. That would make it possible to 
replace individual parts by development versions (which is your original aim) 
- very interesting idea.

Lets assume its done, then I see the following pros and cons:

Pro:
- The option to leave out certain packages and thereby the possiblity to lower 
the number of dependencies (that was my long-term goal)
- Recompiliation of Sage (because of the addition of some patches) would take 
less time

Con:
- Upgrading is much more difficult, because the number of packages that need a 
bump would be much higher (...site-packages/sage currently has 42 
subdirectories). Maybe we will then need some automatization
- The number of patches we apply to Sage is quite high, also the number of 
lines of code (additional sed patches). By splitting up the big Sage ebuild we 
decentralize this and I fear the overall number of patches will rise - 
although the new ebuilds will be relatively small, I think.

I don't know how difficult this split is, but I am tempted to do this. What do 
you think of it, Francois? DId you already tried something similar with sage-
prefix?

Cheers,
Christopher

Re: [gentoo-science] sage queues

[VulK]

* fbissey@slingshot.co.nz <fbissey@slingshot.co.nz> [2011-08-09 17:21:10]:

> Quoting VulK <etn45p4m@gmail.com>:
> 
> > Hi,
> > Thank you for the explanation: I kind of guessed that some part of sage were
> > omitted to adapt the two packaging system but your explanation gave me the
> > details I was missing.
> > As you said the combinat queue is/should be a real mess of continuous
> > updates (at least this is what I was told) so I am not entirely sure how well
> > an e-build would perform, in case you decide to spend some time on it I will
> > gladly be a guinea pig for testing it out.
> > I do not understand sage package system in details so my request may just be
> > stupid but is it possible to produce separate ebuilds for the different part
> > of sage that are now stripped? If not for all of those can this be 
> > done for the
> > various packages in $SAGE_ROOT/devel ? If an e-build is not feasible, can
> > USE flags be used to select which extensions to include at compile time?
> >
> The details are a bit long to explain but everything provided by sage is
> currently split. Technically what is missing is some scripts from the spkg
> sage_scripts (provided by our sage-baselayout ebuild). Most of the files in
> $SAGE_LOCAL/bin of a vanilla install that starts with sage are provided 
> by this
> spkg. And we omit a lot of them, some are already installed only on use flag
> request. We could add more if it was useful and feasible from a package
> management perspective.
> 
> I must say that talking with sage-developers interested in sage installed with
> portage there is a possibility that some stuff may come back in some form once
> we figured it out. 

Do you mean that in a distant future sage package system *might* become
portage?

> Something like sage -combinat creates a new sage branch.
> There is a possibility that we could allow such a branch to be created 
> inside a
> user account (not system wide) and allow its use. But that's still some 
> way off
> on my map. In fact it may come as a surprise to my fellow sage-on-gentoo devs.
> 
> Francois
> 
> 
VulK

Re: [gentoo-science] sage queues

[fbissey]

Quoting VulK <etn45p4m@gmail.com>:
>
> Do you mean that in a distant future sage package system *might* become
> portage?
>

Hi VuLK,

there is a distinct possibility. A number of developers are
thinking about it and we have a proof of concept that kind of run
on linux. It still has rough edges and we don't spend as much
time on it as necessary to give it some polish at the moment.

https://bitbucket.org/burcin/sage-prefix/wiki/Home

Francois