[gentoo-soc] Interest in Improve packaging of scientific software

[gentoo-soc] Interest in Improve packaging of scientific software

[Federico Lodovici]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

Hi,
I'm Federico Lodovici an undergraduate student of Computer Science at
Milan-Bicocca University.

 I've been using mainly Gentoo for an year and a half and linux since I was
14. I'm interested in Machine Learning and Data Analysis and Visualization
tools applied to science (since I want to continue my studies in Data
Science for Physics) and I really want to been able to give anyone the
opportunity to use this tools on gentoo without many hassle so I found this
idea very interesting to me but actually I've not contacted the mentors.

 I'm helping to mantain an unofficial repo of ebuild on github so I'm
somehow familiar in writing them, I've got a Bugzilla account with the same
email but I haven't done any bugfix or PR yet. I'm very familiar with
python (even if actually I'm using it with pandas to do simple data
analysis projects) and with Bash/Zsh/Sh (On my github there are simple
scripts I've done using it)

My github profile is https://github.com/fedelodo, my freenode and telegram
usernames are fede_run.

Thanks for your attention, I look forward to hearing back from you.
Federico Lodovici

[-- Attachment #2: Type: text/html, Size: 1320 bytes --]

Re: [gentoo-soc] Interest in Improve packaging of scientific software

[Benda Xu]

Hi Federico,

Federico Lodovici <f.lodox@gmail.com> writes:

> I'm Federico Lodovici an undergraduate student of Computer Science at
> Milan-Bicocca University.
>
> I've been using mainly Gentoo for an year and a half and linux since I
> was 14. I'm interested in Machine Learning and Data Analysis and
> Visualization tools applied to science (since I want to continue my
> studies in Data Science for Physics) and I really want to been able to
> give anyone the opportunity to use this tools on gentoo without many
> hassle so I found this idea very interesting to me 

> I'm helping to mantain an unofficial repo of ebuild on github so I'm
> somehow familiar in writing them, 

> I'm very familiar with python (even if actually I'm using it with
> pandas to do simple data analysis projects) and with Bash/Zsh/Sh (On
> my github there are simple scripts I've done using it)
>
> My github profile is https://github.com/fedelodo, my freenode and
> telegram usernames are fede_run.

Welcome and thank you for your interest!

> but actually I've not contacted the mentors.

> I've got a Bugzilla account with the same email but I haven't done any
> bugfix or PR yet. 

You have already figured out what to do.  So just go for it, discuss
with the mentors and find bugs assigned to sci _AT_ gentoo _DOT_ org on
the bugzilla and help resolve them.

If you are interested in data science and physics, look into the related
packages both in gentoo repository and science overlay, and figure out
what's missing.  Feel free to ask on this list along your preparation.
Then you can come back with a feasible plan for the summer.

Yours,
Benda

Re: [gentoo-soc] Interest in Improve packaging of scientific software

[Federico Lodovici]

[-- Attachment #1: Type: text/plain, Size: 2026 bytes --]

Hi,
during this week I've updated my gsoc proposal document, you can find it
here:
https://docs.google.com/document/d/1bN3ZiEGweg1t2SmPztIu6aveE5jQIZFGm722L31_tiM/edit?usp=sharing
What do you think? How can I improve it?
Tanks in advance.
Federico Lodovici

2018-02-19 16:20 GMT+01:00 Benda Xu <heroxbd@gmail.com>:

> Hi Federico,
>
> Federico Lodovici <f.lodox@gmail.com> writes:
>
> > I'm Federico Lodovici an undergraduate student of Computer Science at
> > Milan-Bicocca University.
> >
> > I've been using mainly Gentoo for an year and a half and linux since I
> > was 14. I'm interested in Machine Learning and Data Analysis and
> > Visualization tools applied to science (since I want to continue my
> > studies in Data Science for Physics) and I really want to been able to
> > give anyone the opportunity to use this tools on gentoo without many
> > hassle so I found this idea very interesting to me
>
> > I'm helping to mantain an unofficial repo of ebuild on github so I'm
> > somehow familiar in writing them,
>
> > I'm very familiar with python (even if actually I'm using it with
> > pandas to do simple data analysis projects) and with Bash/Zsh/Sh (On
> > my github there are simple scripts I've done using it)
> >
> > My github profile is https://github.com/fedelodo, my freenode and
> > telegram usernames are fede_run.
>
> Welcome and thank you for your interest!
>
> > but actually I've not contacted the mentors.
>
> > I've got a Bugzilla account with the same email but I haven't done any
> > bugfix or PR yet.
>
> You have already figured out what to do.  So just go for it, discuss
> with the mentors and find bugs assigned to sci _AT_ gentoo _DOT_ org on
> the bugzilla and help resolve them.
>
> If you are interested in data science and physics, look into the related
> packages both in gentoo repository and science overlay, and figure out
> what's missing.  Feel free to ask on this list along your preparation.
> Then you can come back with a feasible plan for the summer.
>
> Yours,
> Benda
>
>

[-- Attachment #2: Type: text/html, Size: 2975 bytes --]

Re: [gentoo-soc] Interest in Improve packaging of scientific software

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 4152 bytes --]

Hi,

On Sun, 18 Mar 2018 20:29:49 +0100 Federico Lodovici wrote:
> Hi,
> during this week I've updated my gsoc proposal document, you can find it
> here:
> https://docs.google.com/document/d/1bN3ZiEGweg1t2SmPztIu6aveE5jQIZFGm722L31_tiM/edit?usp=sharing
> What do you think? How can I improve it?

Most important: provide detailed per-week plan of your work. This
is very important. Listing deliverables is good, but we must see
per-week progress on how you are going to obtain them. In case on
unexpected problems the plan may be amended later if mutual mentors
and student agreement is reached.

The detailed plan is not just formality: it help a lot to see how
student understands their task and how realistic are objectives
within given time frame. Of course it should be reasonable without
extremes: not too simple to be reasonable for 3 months of work and
not overcomplicated, because student should do work timely.

Please note that according to the GSoC program rules, mentors must
evaluated student based on the final submitted application.
Amendments after the deadline are not possible.

> TF needs a lot of python dependencies and also optional cuda
> toolkit and cuda libraries

We already have CUDA toolkit and libraries in the sci overlay. They
may need some update, but it shouldn't be a problem.

> I’ll then start the most challenge part, porting and packaging
> Intel Tools.

While Intel stuff may rightfully be a part of your project, I do not
recommend to focus on them too much, since this is a proprietary
software and GSoC is all about Free/Libre software.

> I’m also interested to port Intel’s python distribution

I've discussed this project with Intel devs on one of the
conferences. There is nothing special about it: it is a normal
Python linked with Intel libraries and with some math libs replaced
with more optimized free software solutions. So everyone can do the
same with Intel MKL without need to obtain Intel Python. They
created this project mostly due to marketing issues, since python
is a popular language and management want to establish Intel's
presence in this area.

If you want to pursue this task, I recommend to build on FLOSS
solutions as described above, packaging Intel Python itself is
quite useless.

> I’d like if it is possible to bring into the sci-gentoo overlay
> an “official” matlab ebuild

Devoting a whole month to the proprietary piece of software is
questionable again. Devoting some time to improve
proprietary software packaging in Gentoo is OK, but devoting half of
your time for them is questionable at least.

> I’ve fixed a compilation bug on sci-physics/rivet and I’m
> finding the best strategy to fix sci-lib/scipy.

Please provide a links or bug numbers for bugs you have fixed in
the main repo. I found only your version bump of rivet (commit
23ff9c6da1eb892b12e7a0b1edbf4d87ae4f6b26). If you have made other
changes, please provide all the links in your proposal.

Do not forget to run repoman on your packages or changes, your
overlay clearly have some QA issues right now (e.g. packages lack
metadata.xml).

Bottom notes:

1. Please provide your scheduled absence during the coding period if
any: you may have exams or other events when your availability will
be limited

2. We need to know your availability hours and time zone.

3. You must provide more contact information, see top of the
https://wiki.gentoo.org/wiki/Google_Summer_of_Code/2018/Ideas
page for the list of our requirements:

«Give us your contact info and working hours. Please provide your
email address, home mailing address, and phone number. This is a
requirement and provides for accountability on both your side and
ours. Also, please tell us what hours you will be working and
responsive to contact via email and IRC; these should sum to at
least 35 hours a week.»

There is no need to provide your personal data in publicly visible
draft, however be sure to include it in the final proposal.

Thanks and good luck!

P.S. Please, do not top-post.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-soc] Interest in Improve packaging of scientific software

[Benda Xu]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

Hi Andrew,

Andrew Savchenko <bircoph@gentoo.org> writes:

>> I’ll then start the most challenge part, porting and packaging
>> Intel Tools.
>
> While Intel stuff may rightfully be a part of your project, I do not
> recommend to focus on them too much, since this is a proprietary
> software and GSoC is all about Free/Libre software.
>
>> I’m also interested to port Intel’s python distribution
>
> I've discussed this project with Intel devs on one of the
> conferences. There is nothing special about it: it is a normal
> Python linked with Intel libraries and with some math libs replaced
> with more optimized free software solutions. So everyone can do the
> same with Intel MKL without need to obtain Intel Python. They
> created this project mostly due to marketing issues, since python
> is a popular language and management want to establish Intel's
> presence in this area.
>
> If you want to pursue this task, I recommend to build on FLOSS
> solutions as described above, packaging Intel Python itself is
> quite useless.

+1

With a more rubost blas/lapack framework/eclass, an optimized
scipy/numpy linked with OpenBLAS or Intel MKL will be on par if not
overtake the Intel python binaries.

>> I’d like if it is possible to bring into the sci-gentoo overlay
>> an “official” matlab ebuild
>
> Devoting a whole month to the proprietary piece of software is
> questionable again. Devoting some time to improve
> proprietary software packaging in Gentoo is OK, but devoting half of
> your time for them is questionable at least.

+1

I don't think a gigantic piece of proprietary software is very
interesting to us.

Benda

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [gentoo-soc] Interest in Improve packaging of scientific software

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1651 bytes --]

On Mon, 26 Mar 2018 08:03:32 +0900 Benda Xu wrote:
> Hi Andrew,
> 
> Andrew Savchenko <bircoph@gentoo.org> writes:
> 
> >> I’ll then start the most challenge part, porting and packaging
> >> Intel Tools.
> >
> > While Intel stuff may rightfully be a part of your project, I do not
> > recommend to focus on them too much, since this is a proprietary
> > software and GSoC is all about Free/Libre software.
> >
> >> I’m also interested to port Intel’s python distribution
> >
> > I've discussed this project with Intel devs on one of the
> > conferences. There is nothing special about it: it is a normal
> > Python linked with Intel libraries and with some math libs replaced
> > with more optimized free software solutions. So everyone can do the
> > same with Intel MKL without need to obtain Intel Python. They
> > created this project mostly due to marketing issues, since python
> > is a popular language and management want to establish Intel's
> > presence in this area.
> >
> > If you want to pursue this task, I recommend to build on FLOSS
> > solutions as described above, packaging Intel Python itself is
> > quite useless.
> 
> +1
> 
> With a more rubost blas/lapack framework/eclass, an optimized
> scipy/numpy linked with OpenBLAS or Intel MKL will be on par if not
> overtake the Intel python binaries.

[OT]
I made some tests with linpack (HPL) with MKL vs OpenBLAS on
Gentoo-based HPC cluster at my previous job. OpenBLAS was beating
MKL by 0.5 ± 0.1%. Not large, but statistically significant result.
But of course MKL is not just BLAS/LAPACK...
[/OT]

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]