[gentoo-soc] Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[gentoo-soc] Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Yuan Liao (Leo)]

Hi folks,

I have been quite busy creating the ebuilds that build the Kotlin
libraries from source during the past week, and I am pleased to report
that the work is now complete and the ebuilds are installable and
usable.  Every Kotlin library listed here [1] can be built from source
using those ebuilds.  All ebuilds have been pushed to my fork of the
Spark overlay [2].

To use the ebuilds to build the Kotlin libraries from source, binary
JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
must be installed first for bootstrapping.  Then, after the Kotlin
compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
can be built from source.  The exact commands to build and install the
libraries from source are:

    env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
    emerge --ask dev-lang/kotlin-bin
    emerge --ask --update --newuse --deep @world

The ebuilds only use kotlinc and javac to build the Kotlin libraries.
Although Gradle is used by the upstream to build Kotlin, it is not
used in those ebuilds.  I extracted the commands Gradle would run to
build each library component to create those ebuilds.

I am currently in the process of writing a blog post describing how I
discovered the way to extract those commands from Gradle and
surmounted various challenges in creating those ebuilds, and I have
also planned to provide more documentation regarding installing and
maintaining those packages.  The blog post is supposed to be completed
today, but I really cannot make it because there are really lots of
things to talk about and my brain is slowing down as bedtime is coming
closer and closer.  I am sorry for the delay.  Once it is published
(hopefully tomorrow), I will post the link to it in a reply to this
email.  The link to the documentation will also be posted after it is
done.

Best regards,
Leo

[1]: https://github.com/JetBrains/kotlin/tree/v1.5.20/libraries
[2]: https://github.com/Leo3418/spark-overlay/tree/master/dev-java#:~:text=kotlin-annotations-jvm

[gentoo-soc] Re: Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Yuan Liao (Leo)]

Hi folks,

The blog post discussing how I created the Kotlin library ebuilds has
been published at:
https://leo3418.github.io/2021/07/05/gentoo-build-kt-src.html

Thanks,
Leo

On Sun, Jul 4, 2021 at 10:03 PM Yuan Liao (Leo) <liaoyuan@gmail.com> wrote:
>
> Hi folks,
>
> I have been quite busy creating the ebuilds that build the Kotlin
> libraries from source during the past week, and I am pleased to report
> that the work is now complete and the ebuilds are installable and
> usable.  Every Kotlin library listed here [1] can be built from source
> using those ebuilds.  All ebuilds have been pushed to my fork of the
> Spark overlay [2].
>
> To use the ebuilds to build the Kotlin libraries from source, binary
> JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
> must be installed first for bootstrapping.  Then, after the Kotlin
> compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
> can be built from source.  The exact commands to build and install the
> libraries from source are:
>
>     env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
>     emerge --ask dev-lang/kotlin-bin
>     emerge --ask --update --newuse --deep @world
>
> The ebuilds only use kotlinc and javac to build the Kotlin libraries.
> Although Gradle is used by the upstream to build Kotlin, it is not
> used in those ebuilds.  I extracted the commands Gradle would run to
> build each library component to create those ebuilds.
>
> I am currently in the process of writing a blog post describing how I
> discovered the way to extract those commands from Gradle and
> surmounted various challenges in creating those ebuilds, and I have
> also planned to provide more documentation regarding installing and
> maintaining those packages.  The blog post is supposed to be completed
> today, but I really cannot make it because there are really lots of
> things to talk about and my brain is slowing down as bedtime is coming
> closer and closer.  I am sorry for the delay.  Once it is published
> (hopefully tomorrow), I will post the link to it in a reply to this
> email.  The link to the documentation will also be posted after it is
> done.
>
> Best regards,
> Leo
>
> [1]: https://github.com/JetBrains/kotlin/tree/v1.5.20/libraries
> [2]: https://github.com/Leo3418/spark-overlay/tree/master/dev-java#:~:text=kotlin-annotations-jvm

[gentoo-soc] Re: Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Yuan Liao (Leo)]

Hi folks,

Second update: the user documentation of the Kotlin packages is now
complete.  It is available as a subpage under my user page on Gentoo
Wiki:
https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin

Thanks,
Leo

On Mon, Jul 5, 2021 at 6:30 PM Yuan Liao (Leo) <liaoyuan@gmail.com> wrote:
>
> Hi folks,
>
> The blog post discussing how I created the Kotlin library ebuilds has
> been published at:
> https://leo3418.github.io/2021/07/05/gentoo-build-kt-src.html
>
> Thanks,
> Leo
>
> On Sun, Jul 4, 2021 at 10:03 PM Yuan Liao (Leo) <liaoyuan@gmail.com> wrote:
> >
> > Hi folks,
> >
> > I have been quite busy creating the ebuilds that build the Kotlin
> > libraries from source during the past week, and I am pleased to report
> > that the work is now complete and the ebuilds are installable and
> > usable.  Every Kotlin library listed here [1] can be built from source
> > using those ebuilds.  All ebuilds have been pushed to my fork of the
> > Spark overlay [2].
> >
> > To use the ebuilds to build the Kotlin libraries from source, binary
> > JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
> > must be installed first for bootstrapping.  Then, after the Kotlin
> > compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
> > can be built from source.  The exact commands to build and install the
> > libraries from source are:
> >
> >     env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
> >     emerge --ask dev-lang/kotlin-bin
> >     emerge --ask --update --newuse --deep @world
> >
> > The ebuilds only use kotlinc and javac to build the Kotlin libraries.
> > Although Gradle is used by the upstream to build Kotlin, it is not
> > used in those ebuilds.  I extracted the commands Gradle would run to
> > build each library component to create those ebuilds.
> >
> > I am currently in the process of writing a blog post describing how I
> > discovered the way to extract those commands from Gradle and
> > surmounted various challenges in creating those ebuilds, and I have
> > also planned to provide more documentation regarding installing and
> > maintaining those packages.  The blog post is supposed to be completed
> > today, but I really cannot make it because there are really lots of
> > things to talk about and my brain is slowing down as bedtime is coming
> > closer and closer.  I am sorry for the delay.  Once it is published
> > (hopefully tomorrow), I will post the link to it in a reply to this
> > email.  The link to the documentation will also be posted after it is
> > done.
> >
> > Best regards,
> > Leo
> >
> > [1]: https://github.com/JetBrains/kotlin/tree/v1.5.20/libraries
> > [2]: https://github.com/Leo3418/spark-overlay/tree/master/dev-java#:~:text=kotlin-annotations-jvm

Re: [gentoo-soc] Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Benda Xu]

Hi Leo,

"Yuan Liao (Leo)" <liaoyuan@gmail.com> writes:

> I have been quite busy creating the ebuilds that build the Kotlin
> libraries from source during the past week, and I am pleased to report
> that the work is now complete and the ebuilds are installable and
> usable.  Every Kotlin library listed here [1] can be built from source
> using those ebuilds.  All ebuilds have been pushed to my fork of the
> Spark overlay [2].
>
> To use the ebuilds to build the Kotlin libraries from source, binary
> JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
> must be installed first for bootstrapping.  Then, after the Kotlin
> compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
> can be built from source.  The exact commands to build and install the
> libraries from source are:
>
>     env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
>     emerge --ask dev-lang/kotlin-bin
>     emerge --ask --update --newuse --deep @world
>
> The ebuilds only use kotlinc and javac to build the Kotlin libraries.
> Although Gradle is used by the upstream to build Kotlin, it is not
> used in those ebuilds.  I extracted the commands Gradle would run to
> build each library component to create those ebuilds.
>
> I am currently in the process of writing a blog post describing how I
> discovered the way to extract those commands from Gradle and
> surmounted various challenges in creating those ebuilds, and I have
> also planned to provide more documentation regarding installing and
> maintaining those packages.  The blog post is supposed to be completed
> today, but I really cannot make it because there are really lots of
> things to talk about and my brain is slowing down as bedtime is coming
> closer and closer.  I am sorry for the delay.  Once it is published
> (hopefully tomorrow), I will post the link to it in a reply to this
> email.  The link to the documentation will also be posted after it is
> done.

Congratulation for such a major triumph as a by-product of your GSoC
project!  The idea for Gentoo to be the first GNU/Linux distribution
supporting kotlin toolchain from source code is the sexiest.

@fordfrog, IMHO this achievement deserves wider exposure.  I would like
to encourage Leo to call for early testers of Gentoo Kotlin ebuilds on
-dev and start the procedure to land
https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin somewhere aligned with
https://wiki.gentoo.org/wiki/Java.  How do you think?

Cheers,
Benda

Re: [gentoo-soc] Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Miroslav Šulc]

Dne 17. 07. 21 v 5:52 Benda Xu napsal(a):
> Hi Leo,
>
> "Yuan Liao (Leo)" <liaoyuan@gmail.com> writes:
>
>> I have been quite busy creating the ebuilds that build the Kotlin
>> libraries from source during the past week, and I am pleased to report
>> that the work is now complete and the ebuilds are installable and
>> usable.  Every Kotlin library listed here [1] can be built from source
>> using those ebuilds.  All ebuilds have been pushed to my fork of the
>> Spark overlay [2].
>>
>> To use the ebuilds to build the Kotlin libraries from source, binary
>> JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
>> must be installed first for bootstrapping.  Then, after the Kotlin
>> compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
>> can be built from source.  The exact commands to build and install the
>> libraries from source are:
>>
>>      env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
>>      emerge --ask dev-lang/kotlin-bin
>>      emerge --ask --update --newuse --deep @world
>>
>> The ebuilds only use kotlinc and javac to build the Kotlin libraries.
>> Although Gradle is used by the upstream to build Kotlin, it is not
>> used in those ebuilds.  I extracted the commands Gradle would run to
>> build each library component to create those ebuilds.
>>
>> I am currently in the process of writing a blog post describing how I
>> discovered the way to extract those commands from Gradle and
>> surmounted various challenges in creating those ebuilds, and I have
>> also planned to provide more documentation regarding installing and
>> maintaining those packages.  The blog post is supposed to be completed
>> today, but I really cannot make it because there are really lots of
>> things to talk about and my brain is slowing down as bedtime is coming
>> closer and closer.  I am sorry for the delay.  Once it is published
>> (hopefully tomorrow), I will post the link to it in a reply to this
>> email.  The link to the documentation will also be posted after it is
>> done.
> Congratulation for such a major triumph as a by-product of your GSoC
> project!  The idea for Gentoo to be the first GNU/Linux distribution
> supporting kotlin toolchain from source code is the sexiest.
>
> @fordfrog, IMHO this achievement deserves wider exposure.  I would like
> to encourage Leo to call for early testers of Gentoo Kotlin ebuilds on
> -dev and start the procedure to land
> https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin somewhere aligned with
> https://wiki.gentoo.org/wiki/Java.  How do you think?

i would like to move kotlin to the main tree once it is ready according 
to Leo. just one thing comes to my mind that should be resolved prior to 
that, and that is support for multiple kotlin compilers being installed 
at the same time and a way to choose among them, like we have for java 
(eselect java-vm). i think this is needed both for users and for 
packaging packages that use kotlin compiler. one other thing that comes 
to my mind is launching kotlin compiled apps with kotlin deps, but i 
suppose that our java launcher should be able to handle the deps so it 
probably won't be an issue at all.

i would also like to merge java-ebuilder improvements from Leo's fork, 
but not sure whether i can/should merge everything from the fork or just 
cherry-pick some features atm.

> Cheers,
> Benda
>

Re: [gentoo-soc] Week 4 Report for Big Data Infrastructure and H2O ebuilds Project

[Yuan Liao (Leo)]

Hi Benda and fordfrog,

> Congratulation for such a major triumph as a by-product of your GSoC
> project!  The idea for Gentoo to be the first GNU/Linux distribution
> supporting kotlin toolchain from source code is the sexiest.

Thanks!  I am quite excited about this too.

> i would like to move kotlin to the main tree once it is ready according
> to Leo. just one thing comes to my mind that should be resolved prior to
> that, and that is support for multiple kotlin compilers being installed
> at the same time and a way to choose among them, like we have for java
> (eselect java-vm). i think this is needed both for users and for
> packaging packages that use kotlin compiler. one other thing that comes
> to my mind is launching kotlin compiled apps with kotlin deps, but i
> suppose that our java launcher should be able to handle the deps so it
> probably won't be an issue at all.

I agree.  An eselect module for the Kotlin compiler would be
beneficial.  To make the Kotlin packages practically useful, I am glad
to allot some time to creating such a module.

Speaking of launching Kotlin apps, at least in the domain of
Kotlin/JVM, Kotlin packages are compiled into JARs, and the Kotlin
Standard Library is also a JAR, so they should integrate with any
existing Java utilities well.  The kotlin-libs.eclass I created for
Kotlin library packages follows most of the conventions of existing
Java eclasses in terms of classpath registration, JAR installation
target path, etc.; it was designed with interoperability with Java
packages in mind.

> i would also like to merge java-ebuilder improvements from Leo's fork,
> but not sure whether i can/should merge everything from the fork or just
> cherry-pick some features atm.

Once I finish up my current work on ebuild testing (which should be
close and done next week), I will revisit and review the commits in my
fork, and discuss with you which improvements can be merged into the
main tree and what additional changes are desired.

Thanks,
Leo

On Sat, Jul 17, 2021 at 12:19 AM Miroslav Šulc <fordfrog@gentoo.org> wrote:
>
> Dne 17. 07. 21 v 5:52 Benda Xu napsal(a):
> > Hi Leo,
> >
> > "Yuan Liao (Leo)" <liaoyuan@gmail.com> writes:
> >
> >> I have been quite busy creating the ebuilds that build the Kotlin
> >> libraries from source during the past week, and I am pleased to report
> >> that the work is now complete and the ebuilds are installable and
> >> usable.  Every Kotlin library listed here [1] can be built from source
> >> using those ebuilds.  All ebuilds have been pushed to my fork of the
> >> Spark overlay [2].
> >>
> >> To use the ebuilds to build the Kotlin libraries from source, binary
> >> JARs for kotlin-stdlib and kotlin-reflect pre-built by the upstream
> >> must be installed first for bootstrapping.  Then, after the Kotlin
> >> compiler package -- dev-lang/kotlin-bin -- is installed, the libraries
> >> can be built from source.  The exact commands to build and install the
> >> libraries from source are:
> >>
> >>      env USE="binary" emerge -a1 dev-java/kotlin-stdlib dev-java/kotlin-reflect
> >>      emerge --ask dev-lang/kotlin-bin
> >>      emerge --ask --update --newuse --deep @world
> >>
> >> The ebuilds only use kotlinc and javac to build the Kotlin libraries.
> >> Although Gradle is used by the upstream to build Kotlin, it is not
> >> used in those ebuilds.  I extracted the commands Gradle would run to
> >> build each library component to create those ebuilds.
> >>
> >> I am currently in the process of writing a blog post describing how I
> >> discovered the way to extract those commands from Gradle and
> >> surmounted various challenges in creating those ebuilds, and I have
> >> also planned to provide more documentation regarding installing and
> >> maintaining those packages.  The blog post is supposed to be completed
> >> today, but I really cannot make it because there are really lots of
> >> things to talk about and my brain is slowing down as bedtime is coming
> >> closer and closer.  I am sorry for the delay.  Once it is published
> >> (hopefully tomorrow), I will post the link to it in a reply to this
> >> email.  The link to the documentation will also be posted after it is
> >> done.
> > Congratulation for such a major triumph as a by-product of your GSoC
> > project!  The idea for Gentoo to be the first GNU/Linux distribution
> > supporting kotlin toolchain from source code is the sexiest.
> >
> > @fordfrog, IMHO this achievement deserves wider exposure.  I would like
> > to encourage Leo to call for early testers of Gentoo Kotlin ebuilds on
> > -dev and start the procedure to land
> > https://wiki.gentoo.org/wiki/User:Leo3418/Kotlin somewhere aligned with
> > https://wiki.gentoo.org/wiki/Java.  How do you think?
>
> i would like to move kotlin to the main tree once it is ready according
> to Leo. just one thing comes to my mind that should be resolved prior to
> that, and that is support for multiple kotlin compilers being installed
> at the same time and a way to choose among them, like we have for java
> (eselect java-vm). i think this is needed both for users and for
> packaging packages that use kotlin compiler. one other thing that comes
> to my mind is launching kotlin compiled apps with kotlin deps, but i
> suppose that our java launcher should be able to handle the deps so it
> probably won't be an issue at all.
>
> i would also like to merge java-ebuilder improvements from Leo's fork,
> but not sure whether i can/should merge everything from the fork or just
> cherry-pick some features atm.
>
> > Cheers,
> > Benda
> >