Re: [gentoo-soc] Weekly Report: Big Data Infrastructure and Maven Overlay in Week 5

[EBo <ebo@sandien.com>]

Ahhh... thank you for your time and explanation.  This all makes sense 
now.  I can see how having something that would pom.xml <==> 
package.ebuild would be extremely handy.  Best of luck to you you and 
your team.

   Thanks again,

   EBo --

On Jul 5 2020 11:23 AM, Zhang Zongyu wrote:
> Hello,
>
> Here is the proposal of this project.[1]
> In a nutshell, the goal of this project is to intergrate Portage with
> Maven, or to translate Maven pom.xml into ebuild files.
> And the most related project should be java-ebuilder, which aims at
> processing pom.xml and forming an equivalent ebuild.[2]
> The deliverable of this project will be an overlay resolving all the
> dependencies of spark, which is widely used for big data processing, 
> so my
> mentor named the project "big data infrastructure".
>
> Actually it is not related to big data processing or any "big data".
>
> I am sorry if it confused you, and I am glad that you are interested 
> in
> this project.
>
> Regards,
> Zhang Zongyu
>
> [1] https://summerofcode.withgoogle.com/projects/#4994566568017920
> [2] https://github.com/6-6-6/java-ebuilder
>
> EBo <ebo@sandien.com> 于 2020年7月6日周一 上午12:18写道：
>
>> Out of curiosity, can you point me to any project write up that was
>> probably part of the proposal?  One of my last big-data projects 
>> started
>> out with a 3.5 petabyte repository and filtered on down to cover 10
>> million contiguous kilometers.  We did not use Maven, and I would be
>> curious to see where the project is going.  Now that I know about 
>> this I
>> will try to follow it a bit more.
>>
>>    Best of success!
>>
>> On Jul 5 2020 10:05 AM, Zhang Zongyu wrote:
>> > Hello,
>> >
>> > This week I have done a lot of repetitive work. That is, to go
>> > through
>> > all of the ebuild files in Gentoo Repository and to get them the
>> > equivalent GroupId and ArtifactId in Maven Central. There are more
>> > than 500 packages in dev-java/ and java-virtual/ category, and
>> > sometimes one Gentoo package may correspond to more than one Maven
>> > artifact, so it took me a lot of time to complete them. And here 
>> is
>> > my
>> > lookup up table which covers all packages in dev-java/ and maps
>> > "category/package:slot" into "groupId:artifactId".[1] As a result,
>> > spark-core can make use of more than 100 Gentoo native packages 
>> now.
>> >
>> > Besides, I have also added java-ebuilder some features. It can 
>> check
>> > the metadata of a package and get the correct SLOT, and it can 
>> make
>> > ebuild unpack "Maven Central distributed source files" into proper
>> > ${JAVA_SRC_DIR} now.
>> >
>> > Next week I am going to test and add jaxb (javax.xml.bind:jaxb-api
>> > and
>> > javax.xml.bind:jaxb-impl) and some newer version packages which 
>> are
>> > required by spark-core, and I hope the newer version ebuild files
>> > could be accepted by Gentoo-Java team. The prototype of the ebuild
>> > files that are just generated by java-ebuilder are listed in [2].
>> >
>> > Regards,
>> > Zhang Zongyu
>> >
>> > [1] lookup table containing all packages from dev-java/
>> >
>> >
>> 
>> https://github.com/6-6-6/JEbuilder/blob/master/app-portage/java-ebuilder/files/scripts/LUT/LUT
>> > [2] a prototype overlay
>> > https://github.com/6-6-6/spark-overlay/tree/master/dev-java
>>
>>
>>