[gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Mishal Roy]

[-- Attachment #1.1: Type: text/plain, Size: 272 bytes --]

Hi,

As a proof of my proposed concept I have written a kernel ebuild which
successfully builds x86 linux kernel from source code present in
/usr/src/linux directory of Gentoo liveDVD. I have attached the ebuild and
images related to it.


Thanks and Regards,
Mishal Roy.

[-- Attachment #1.2: Type: text/html, Size: 370 bytes --]

[-- Attachment #2: x86kernelbuild-1.0.ebuild --]
[-- Type: application/octet-stream, Size: 300 bytes --]

# Copyright 1999-2015 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=0

DESCRIPTION="This is an ebuild that compiles x86 linux kernel"

SLOT="0"
KEYWORDS="~x86"

pkg_preinst()
{
cd /usr/src/linux
make ARCH=x86 i386_defconfig
make && make modules_install
}

[-- Attachment #3: ebuild.png --]
[-- Type: image/png, Size: 145063 bytes --]

[-- Attachment #4: ebuild2.png --]
[-- Type: image/png, Size: 698115 bytes --]

[-- Attachment #5: ebuild3.png --]
[-- Type: image/png, Size: 619964 bytes --]

[-- Attachment #6: ebuild4.png --]
[-- Type: image/png, Size: 646940 bytes --]

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Benda Xu]

Hi Mishal,

Mishal Roy <roymishal210@gmail.com> writes:

> As a proof of my proposed concept I have written a kernel ebuild which
> successfully builds x86 linux kernel from source code present in
> /usr/src/linux directory of Gentoo liveDVD. I have attached the ebuild
> and images related to it.

Thank you for trying this out.  

You still have a lot to learn about the basics of ebuild.  Alice is on
the Kernel Team.  She might have more to comment.

> x86kernelbuild-1.0.ebuild

The version number should be the same as upstream.  No
architecture-specific name, like 'x86, should be used in the name.

> EAPI=0

EAPI=0 is no longer encourage in Gentoo.  Check out EAPI-6

> DESCRIPTION="This is an ebuild that compiles x86 linux kernel"

> SLOT="0"

> KEYWORDS="~x86"

> pkg_preinst()

pkg_preinst is not used like this.  Check out src_compile.

> 
> {
> cd /usr/src/linux
> make ARCH=x86 i386_defconfig
> make && make modules_install
> }

> builds x86 linux kernel from source code present in /usr/src/linux
> directory of Gentoo liveDVD

This is not how Gentoo ebuild works, the build should prepare the source
code.


Kernel is a specific piece of software.  Traditionally users take care
of the final installation.  But there is a genkernel helper to automate
the process.  Please check out

  https://wiki.gentoo.org/wiki/Genkernel

for inspirations.


Cheers,
Benda

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Mishal Roy]

[-- Attachment #1: Type: text/plain, Size: 3250 bytes --]

Hi,

Yes I will download the Xiaomi boot image by ebuild.

I had proposed this small project so that it can be successfully completed
within the time frame of three months.
Eliminating the concept of boot image is feasible.
If you recommend, I would like to extend my proposal by adding it.
Aboot the application bootloader of Redmi Note 4 reads boot image and loads
kernel+ramdisk into RAM.
On the other hand, bootloader on computer reads kernel and ramdisk ,and
loads those into RAM.
We can make Aboot function like a computer bootloader by editing its source
code [1].
However we will require Xiaomi's permission  to allow us install the newly
built Aboot because Aboot gets checked by SBL(Secondary boot loader) [2].

I recently got familiar with basics of Gentoo.
I have been using Ubuntu since 2 years.
I have Ubuntu on my desktop computer and laptop.
I use Ubuntu for compiling ASM, C and MATLAB codes.
I also use it for running Guitarix,Rakarrack ( virtual guitar amplifiers)
and Audacity sound recorder.
Through Ubuntu I got to know about initramfs as well as list of services
that are started by sysvinit and upstart.
i.e. udev, pulseaudio , x-sever, lightdm, dbus, cron etc.
I have experimented with these sevices by starting and stopping.
I have not contributed to any community or solved bugs so far.
I have Dell Venue 7 3740 which is an Intel powered Android device.This
device is my main device for Android development.
https://opensource.dell.com/releases/Venue_7_3740_Merrifield/developer-edition/A195/

1. https://www.tldp.org/HOWTO/SRM-HOWTO/aboot.html
2.
https://forum.xda-developers.com/android/general/info-android-device-partitions-basic-t3586565/amp/

Thanks and Regards,
Mishal Roy.

On Thu, Mar 22, 2018 at 9:37 AM, Benda Xu <heroxbd@gentoo.org> wrote:

> Hi Mishal,
>
> Mishal Roy <roymishal210@gmail.com> writes:
>
> > As a proof of my proposed concept I have written a kernel ebuild which
> > successfully builds x86 linux kernel from source code present in
> > /usr/src/linux directory of Gentoo liveDVD. I have attached the ebuild
> > and images related to it.
>
> Thank you for trying this out.
>
> You still have a lot to learn about the basics of ebuild.  Alice is on
> the Kernel Team.  She might have more to comment.
>
> > x86kernelbuild-1.0.ebuild
>
> The version number should be the same as upstream.  No
> architecture-specific name, like 'x86, should be used in the name.
>
> > EAPI=0
>
> EAPI=0 is no longer encourage in Gentoo.  Check out EAPI-6
>
> > DESCRIPTION="This is an ebuild that compiles x86 linux kernel"
>
> > SLOT="0"
>
> > KEYWORDS="~x86"
>
> > pkg_preinst()
>
> pkg_preinst is not used like this.  Check out src_compile.
>
> >
> > {
> > cd /usr/src/linux
> > make ARCH=x86 i386_defconfig
> > make && make modules_install
> > }
>
> > builds x86 linux kernel from source code present in /usr/src/linux
> > directory of Gentoo liveDVD
>
> This is not how Gentoo ebuild works, the build should prepare the source
> code.
>
>
> Kernel is a specific piece of software.  Traditionally users take care
> of the final installation.  But there is a genkernel helper to automate
> the process.  Please check out
>
>   https://wiki.gentoo.org/wiki/Genkernel
>
> for inspirations.
>
>
> Cheers,
> Benda
>

[-- Attachment #2: Type: text/html, Size: 4769 bytes --]

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Benda Xu]

Hi Mishal,

Mishal Roy <roymishal210@gmail.com> writes:

> Yes I will download the Xiaomi boot image by ebuild.
>
> I had proposed this small project so that it can be successfully
> completed within the time frame of three months.

> Eliminating the concept of boot image is feasible.

> If you recommend, I would like to extend my proposal by adding it.

Yes, it is my recommendation to extend the reach of this project. I
understand your position to play it safe to guarantee success.  But a
single ebuild is not of much interest to me personally for 3 months
work.

You have the final word, as this is your project application.

> Aboot the application bootloader of Redmi Note 4 reads boot image and
> loads kernel+ramdisk into RAM.

> On the other hand, bootloader on computer reads kernel and ramdisk
> ,and loads those into RAM.

> We can make Aboot function like a computer bootloader by editing its
> source code [1].

> However we will require Xiaomi's permission to allow us install the
> newly built Aboot because Aboot gets checked by SBL(Secondary boot
> loader) [2].

By "boot image" I think you mean the Android userland image for /system.
I don't see the need to modify application bootloader if we build our
own "boot image" and application bootloader is unlocked to accept
unsigned "boot image".

Did I miss something?


> I recently got familiar with basics of Gentoo.
> I have been using Ubuntu since 2 years.
> I have Ubuntu on my desktop computer and laptop.
> I use Ubuntu for compiling ASM, C and MATLAB codes.
> I also use it for running Guitarix,Rakarrack ( virtual guitar amplifiers) and Audacity sound recorder.
> Through Ubuntu I got to know about initramfs as well as list of services that are started by sysvinit and upstart.
> i.e. udev, pulseaudio , x-sever, lightdm, dbus, cron etc.
> I have experimented with these sevices by starting and stopping.
> I have not contributed to any community or solved bugs so far.
> I have Dell Venue 7 3740 which is an Intel powered Android device.This device is my main device for Android development.
> https://opensource.dell.com/releases/Venue_7_3740_Merrifield/developer-edition/A195/

Very impressive, I see your experience and potential.  It would help if
you summarize your strong points into the application in the Biography
section, after sorting out the major target of your project.

> 1. https://www.tldp.org/HOWTO/SRM-HOWTO/aboot.html
> 2. https://forum.xda-developers.com/android/general/info-android-device-partitions-basic-t3586565/amp/

Thank you for sharing,
Benda

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Rich Freeman]

On Thu, Mar 22, 2018 at 12:07 AM, Benda Xu <heroxbd@gentoo.org> wrote:
>
>> builds x86 linux kernel from source code present in /usr/src/linux
>> directory of Gentoo liveDVD
>
> This is not how Gentoo ebuild works, the build should prepare the source
> code.
>

I don't want to derail whatever your goals are here, but IMO an ebuild
that actually builds a kernel would be useful.

However, such an ebuild shouldn't touch /usr/src.  If I were doing a
kernel build ebuild I'd have it fetch kernel sources like any other
package, then build it in the normal location, with configuration
options provided via USE flags or some default config, and then
install the final kernel in /boot.

I realize that isn't how Gentoo has done it for the last 20 years, but
I've always thought that it would make sense to have a way to actually
maintain up-do-date kernels in the same manner as any other package,
with the /usr/src approach being an also-supported alternative.

That said, there are complications.  Some kernel modules want prepared
(or even built) sources and this approach would clean those after
install.  Also, we'd probably need to use subslots for initramfs
rebuilds or something like that.  I suspect this is why it hasn't been
done yet.

Again, I don't want to derail SoC with this unless there was an intent
to actually build a kernel this way...

-- 
Rich

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Mishal Roy]

[-- Attachment #1.1: Type: text/plain, Size: 1774 bytes --]

Hi,

I agree with you. That was  not the right way of writing an ebuild. I had
posted it as a proof of concept. I don't intend to do so while writing an
ebuild for soc. The way I want to do has been given in my proposal. Please
go through it.

Thanks and Regards,
Mishal Roy.

On Thu, Mar 22, 2018, 8:55 PM Rich Freeman <rich0@gentoo.org> wrote:

> On Thu, Mar 22, 2018 at 12:07 AM, Benda Xu <heroxbd@gentoo.org> wrote:
> >
> >> builds x86 linux kernel from source code present in /usr/src/linux
> >> directory of Gentoo liveDVD
> >
> > This is not how Gentoo ebuild works, the build should prepare the source
> > code.
> >
>
> I don't want to derail whatever your goals are here, but IMO an ebuild
> that actually builds a kernel would be useful.
>
> However, such an ebuild shouldn't touch /usr/src.  If I were doing a
> kernel build ebuild I'd have it fetch kernel sources like any other
> package, then build it in the normal location, with configuration
> options provided via USE flags or some default config, and then
> install the final kernel in /boot.
>
> I realize that isn't how Gentoo has done it for the last 20 years, but
> I've always thought that it would make sense to have a way to actually
> maintain up-do-date kernels in the same manner as any other package,
> with the /usr/src approach being an also-supported alternative.
>
> That said, there are complications.  Some kernel modules want prepared
> (or even built) sources and this approach would clean those after
> install.  Also, we'd probably need to use subslots for initramfs
> rebuilds or something like that.  I suspect this is why it hasn't been
> done yet.
>
> Again, I don't want to derail SoC with this unless there was an intent
> to actually build a kernel this way...
>
> --
> Rich
>

[-- Attachment #1.2: Type: text/html, Size: 2379 bytes --]

[-- Attachment #2: gsoc.pdf --]
[-- Type: application/pdf, Size: 57460 bytes --]

[gentoo-soc] kernel ebuild (Was: Application for Google Summer of Code 2018-Mishal Roy)

[Benda Xu]

Hi Rich,

Thank you for the input.  I had the same puzzle, too.

Rich Freeman <rich0@gentoo.org> writes:

> On Thu, Mar 22, 2018 at 12:07 AM, Benda Xu <heroxbd@gentoo.org> wrote:
>>
>>> builds x86 linux kernel from source code present in /usr/src/linux
>>> directory of Gentoo liveDVD
>>
>> This is not how Gentoo ebuild works, the build should prepare the source
>> code.
>
> I don't want to derail whatever your goals are here, but IMO an ebuild
> that actually builds a kernel would be useful.
>
> However, such an ebuild shouldn't touch /usr/src.  If I were doing a
> kernel build ebuild I'd have it fetch kernel sources like any other
> package, then build it in the normal location, with configuration
> options provided via USE flags or some default config, and then
> install the final kernel in /boot.

+1

> I realize that isn't how Gentoo has done it for the last 20 years, but
> I've always thought that it would make sense to have a way to actually
> maintain up-do-date kernels in the same manner as any other package,
> with the /usr/src approach being an also-supported alternative.

It's worth to know why we made such a choice.  I took the status quo as
is and have never thought carefully about it.

> That said, there are complications.  Some kernel modules want prepared
> (or even built) sources and this approach would clean those after
> install.

We can have a USE flag to control whether the source should be installed
along with the image.  And those kernel module can claim their
dependence on the USE flag.

> Also, we'd probably need to use subslots for initramfs rebuilds or
> something like that.  I suspect this is why it hasn't been done yet.

@Alice, any hints from the kernel team?

> Again, I don't want to derail SoC with this unless there was an intent
> to actually build a kernel this way...

Well, thank you!  That's why I wanted to keep the technical discussions
related to GSOC public.  It's always good to have our students to
communicate with and learn from experts of Gentoo.  Ultimately, the
projects should be useful to the Gentoo community.

Yours,
Benda

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Patrick Lauer]

On 03/22/2018 04:25 PM, Rich Freeman wrote:
> On Thu, Mar 22, 2018 at 12:07 AM, Benda Xu <heroxbd@gentoo.org> wrote:
>>
>>> builds x86 linux kernel from source code present in /usr/src/linux
>>> directory of Gentoo liveDVD
>>
>> This is not how Gentoo ebuild works, the build should prepare the source
>> code.
>>
> 
> I don't want to derail whatever your goals are here, but IMO an ebuild
> that actually builds a kernel would be useful.
> 

https://github.com/adjust/gentoo-overlay/tree/master/sys-kernel

This just needs to be cleaned up a bit to be upstreamed, and so far I've
not had the time for it.

It's a solved problem (I'm aware of at least two other ebuilds to do the
same) :)

> However, such an ebuild shouldn't touch /usr/src.  If I were doing a
> kernel build ebuild I'd have it fetch kernel sources like any other
> package, then build it in the normal location, with configuration
> options provided via USE flags or some default config, and then
> install the final kernel in /boot.
> 
> I realize that isn't how Gentoo has done it for the last 20 years, but
> I've always thought that it would make sense to have a way to actually
> maintain up-do-date kernels in the same manner as any other package,
> with the /usr/src approach being an also-supported alternative.
> 
> That said, there are complications.  Some kernel modules want prepared
> (or even built) sources and this approach would clean those after
> install.  Also, we'd probably need to use subslots for initramfs
> rebuilds or something like that.  I suspect this is why it hasn't been
> done yet.
> 
> Again, I don't want to derail SoC with this unless there was an intent
> to actually build a kernel this way...
> 

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[Rich Freeman]

On Fri, Mar 23, 2018 at 2:50 AM, Patrick Lauer <patrick@gentoo.org> wrote:
>
> https://github.com/adjust/gentoo-overlay/tree/master/sys-kernel
>
> This just needs to be cleaned up a bit to be upstreamed, and so far I've
> not had the time for it.
>
> It's a solved problem (I'm aware of at least two other ebuilds to do the
> same) :)
>

Yeah, that is certainly one approach.  Part of me thinks that
modularizing the initramfs portion would be ideal, and as Benda
suggested making it possible to install the built sources might be
useful (if doing so one would need to patch the file that points to
the build location so that future module builds can find them).  This
particular ebuild uses sources in /usr/src but I think it would be
cleaner to just fetch these into the build directory.

The package I find most frustrating is zfs-kmod, because it needs
fully-built sources and not just prepared sources.  I build to a
tmpfs, and re-creating prepared sources using that approach is
trivial, but doing a full rebuild just to update zfs modules is a bit
painful (granted, less so now that I have 12 cores).  I've been too
lazy to figure out why zfs needs them...

-- 
Rich

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy

[EBo]

On Mar 23 2018 7:37 AM, Rich Freeman wrote:
> On Fri, Mar 23, 2018 at 2:50 AM, Patrick Lauer <patrick@gentoo.org> 
> wrote:
>>
>> https://github.com/adjust/gentoo-overlay/tree/master/sys-kernel
>>
>> This just needs to be cleaned up a bit to be upstreamed, and so far 
>> I've
>> not had the time for it.
>>
>> It's a solved problem (I'm aware of at least two other ebuilds to do 
>> the
>> same) :)
>>
>
> Yeah, that is certainly one approach.  Part of me thinks that
> modularizing the initramfs portion would be ideal, and as Benda
> suggested making it possible to install the built sources might be
> useful (if doing so one would need to patch the file that points to
> the build location so that future module builds can find them).  This
> particular ebuild uses sources in /usr/src but I think it would be
> cleaner to just fetch these into the build directory.
>
> The package I find most frustrating is zfs-kmod, because it needs
> fully-built sources and not just prepared sources.  I build to a
> tmpfs, and re-creating prepared sources using that approach is
> trivial, but doing a full rebuild just to update zfs modules is a bit
> painful (granted, less so now that I have 12 cores).  I've been too
> lazy to figure out why zfs needs them...

In the past I have had zfs-kmod seriously break my kernel builds.  
Cleaning that up to be consistent would probably be useful to the 
general community as a whole IMNSHO.  Isn't there a way to set the build 
and install directories from some env variable?  If so, maybe verifying 
that the core emuilds are consistent would be helpful.  Hmmm... in the 
past I have written utility scripts to check for things like this.  
Would it be reasonable to add those checks to repoman (or whatever it is 
called today)?

Hope I am not out of line asking this on this forum.

   EBo --

[gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy.

[Mishal Roy]

[-- Attachment #1.1: Type: text/plain, Size: 435 bytes --]

Hello,

I installed 32 bit tarball 2016.10.26 on my Xiaomi Redmi Note 4 and the
Gentoo prefix is running nicely.I have attached some images relating to the
Gentoo prefix on Android.When Gentoo environment is launched by the
startprefix there is change in values of SHELL and PATH variables. Is the
root directory  changed too for successful working of Gentoo prefix?
Kindly guide me for further steps.

Thanks and Regards,
Mishal Roy.

[-- Attachment #1.2: Type: text/html, Size: 593 bytes --]

[-- Attachment #2: 2.png --]
[-- Type: image/png, Size: 72910 bytes --]

[-- Attachment #3: 5.png --]
[-- Type: image/png, Size: 287140 bytes --]

[-- Attachment #4: 7.png --]
[-- Type: image/png, Size: 258562 bytes --]

[-- Attachment #5: 6.png --]
[-- Type: image/png, Size: 308193 bytes --]

[-- Attachment #6: 3.png --]
[-- Type: image/png, Size: 261413 bytes --]

[-- Attachment #7: 1.png --]
[-- Type: image/png, Size: 234904 bytes --]

[-- Attachment #8: 4.png --]
[-- Type: image/png, Size: 251733 bytes --]

Re: [gentoo-soc] Application for Google Summer of Code 2018-Mishal Roy.

[Benda Xu]

Hi Mishal,

Thank you for testing out Gentoo on Android.

Mishal Roy <roymishal210@gmail.com> writes:

> I installed 32 bit tarball 2016.10.26 on my Xiaomi Redmi Note 4 and
> the Gentoo prefix is running nicely.  

> I installed 32 bit tarball 2016.10.26 on my Xiaomi Redmi Note 4 and
> ran the startprefix script. I was amazed to see the prefix launching a
> Gentoo environment on the Android device natively.

Is there any compelling reason for not using tarball-2018.02.26?

Snapdragon 625 in Redmi Note 4 is 64bit.  Is the linux kernel on it
64bit?  If so, you will also be able to install a 64bit Gentoo tarball.

> I have attached some images relating to the Gentoo prefix on Android.

Thanks, they look well indeed.

> When Gentoo environment is launched by the startprefix there is change
> in values of SHELL and PATH variables.

> I typed printenv to get the enivornment variable values.PATH ,SHELL
> values were changed after running startprefix.

Yes, that's expected.

> Is the root directory changed too for successful working of Gentoo
> prefix?

> Is the ROOT location changed by prefix to run a Gentoo environment ?

No, the root directory does not change.  That is for chroot[1] or linux
containers or filesystem namespaces[3].  Gentoo Prefix runs from within
a directory prefix.[4]  In our case, it is /data/gentoo.

> Unlike, Gentoo minimal installation CD where user has to mount
> proc,sys,dev filesystems and edit configuration files,user doesn't
> have to do the same for Gentoo prefix.

Well, we don't have control over the filesystem layout by Gentoo Prefix
on an Android device.  It is Android init system that takes care of
/proc, /sys, /dev, etc.  This might change when the control is
transferred from Android to Gentoo in the future.

> I ran emerge --sync and it worked successfully. Also the emerge --ask
> app-office/gnumeric gave a list of dependencies and USE flags.

Did it build successfully?

> I see Gentoo Prefix as a powerful tool for Android. For example
> Samsung Galaxy Grand 2 didn't have OTG support and inorder to gain
> support I had to compile an OTG enabled Android kernel using
> computer. By using Portage and Ebuild, Android device can itself
> compile kernel source code, build boot image and install that boot
> image. Thus making it easy for Android users.

Nice! As a proof of concept, could you please write an ebuild for the
linux kernel of Samsung Galaxy Grand 2?

> Kindly guide me for further steps.

How about a kernel ebuild as above?

Please start drafting your application if you intend to get involved.
And circulate on this mailing list for discussion.

  https://developers.google.com/open-source/gsoc/timeline

Cheers,
Benda

1. https://en.wikipedia.org/wiki/Chroot
2. https://en.wikipedia.org/wiki/Linux_containers
3. https://en.wikipedia.org/wiki/Linux_namespaces
4. https://wiki.gentoo.org/wiki/Project:Prefix