[gentoo-soc] Week 8 Report for Refining ROCm Packages in Gentoo

[wuyy <xgreenlandforwyy@gmail.com>]

Hello all,

I'm back on GSoC after a two-week temporary leave.

This week there are two major progress: dev-util/rocprofiler and
rocm.eclass.

I have implemented all the functions I think necessary for rocm.eclass.
It was just send to rocm.eclass draft to gentoo-dev mailing list (also
with a Github PR at [1]), please have a review. In the following weeks,
I will collect feedbacks and continue to polish it.

In summary, I have implemented those functions which is listed in my
proposal:
- USE_EXPNAD of amdgpu_targets_, and ROCM_USEDEP to make the use flag
  coherent among dependencies;
- rocm_src_configure contains common arguments in src_prepare;
- rocm_src_test which checks the permission on /dev/kfd and
  /dev/dri/render*

There are also something listed in proposal but I decided not to
implement now:
- rocm_src_prepare: although there are some similarities among ebuilds,
  src_prepare are highly customized to each ROCm components. Unifying
  would take extra work.
- SRC_URI: currently all SRC_URI is already specified in each ebuilds.
  It does not hurt to keep the status quo.

Moreover, during implementation I found another feature necessary 
- rocm_src_test: correctly handles different scenarios. ROCm packages
  may have cmake test, which can be run using cmake_src_test, or only
  compiled some testing binaries which requires execution from
  command-line. I made rocm_src_test automatically detect the method, so
  ROCm packages just have to call this function directly without doing
  anything.

Actually I have never imagined rocm.eclass could be in this shape
eventually. Initially I just thought it would provide some utilities,
mainly src_test and USE_EXPAND. But when implementing I found all these
feature requires careful treatment. The comments (mainly examples) also
takes half of the length. It ends up in 278 lines, which is a
middle-sized among current eclasses. Maybe it can be further trimmed
down after  polishing, because there could be awkward implementations or
re-inventions in it.

Based on my draft rocm.eclass, I have prepared sci-libs/roc*=5.1.3,
sci-lib/hip-*-5.1.3 and dev-python/cupy making use of it. It feels great
to simplify the ebuilds, and portage can handles the USE_EXPAND and
dependencies just as expected. Once the rocm.eclass get in tree, I'll
push those ROCm-5.1.3 ebuilds.


Anther thing to mention is that ROCm-5.1.3 toolchains finally get merged
[5], with the fixed dev-util/rocprofiler-{4.3.0,5.0.2,5.1.3}.
rocprofiler is actually buggy before, because I thought I committed the
patch which stripped the libhsa-amd-aqlprofile.so loading (I even
claimed it in the commit message), but it was not committed and lost in
history. So I reproduced the patch. Also, I did some research about this
proprietary lib. By default, not loading it means tracing hsa/hip is not
possible -- you only get basic information like name and time of each
GPU kernel execution, but do not know the pipeline of kernel execution
(which one has spawned which kernel). AQL should be HSA architected
queuing language (HSA AQL), where
https://llvm.org/docs/AMDGPUUsage.html#hsa-aql-queue documented. It did
sound related to the pipeline of kernel dispatching. By the description,
libhsa-amd-aqlprofile.so is an extension API of AQL Profile. But
actually, patching the source code to let rocprofiler not loading
libhsa-amd-aqlprofile.so does not breaks the tracing of hsa/hip. So, I'm
not sure why libhsa-amd-aqlprofile.so is needed, and raised a question
at [2].  So I complete the fix in [3,4].

According to the renewed proposal (I have been leaving for two weeks, so
there are changes in plan), I should collect feedback and refine
rocm.eclass, and prepare dev-python/cupy and sci-libs/rocWMMA. I'll
investigate ROCgdb, too. Also, rocm-device-libs is a major package
because many users relies on it to provide opencl. I'll work on bumping
its version, too. What's more, with hip-5.1.3 against vanilla clang,
rocm for blender can land in ::gentoo.

[1] https://github.com/gentoo/gentoo/pull/26784
[2] https://github.com/RadeonOpenCompute/ROCm/issues/1781
[3] https://github.com/gentoo/gentoo/pull/26755
[4] https://github.com/gentoo/gentoo/pull/26771
[5] https://github.com/gentoo/gentoo/pull/26441

Best wishes,
-- 
Yiyang Wu