Newbie friendly, DRM-only opengles, /init tutorial

[habemus-papadum]

Hi Miouyouyou!

I was wondering if you would consider the following idea for a tutorial at some point; it is a followup from my comments at your gist:

Create a wiki page detailing the steps required to build a minimal system image that boots and runs your opengles example as an /init script. From some of your writings, I think you have an interest in android; this tutorial would have a slightly different target audience: people interested creating absolute minimal devices in which they can control and understand nearly every part of the process and nevertheless can access the full power of the cpu/gpu (e.g. have proper Mali drivers). I'm not sure how interesting this would be for you, but if you have questions I can try to flesh out more details and provide better explanations. (and as you pointed out in your comment on the gist, the remaining black box/proprietary stuff would be the Mali drivers themselves, which for me is something I can live with....)

For the time being, I was thinking: Since most of the complicated details are worked out (RockMyy, the opengl gist, etc), it might be good to make the tutorial newbie friendly:

• starting with some basic advise on selecting a board
• assembling a tool chain (keep this as simple as possible...)
• building RockMyy -- menuconfig options, laying out root file system
• getting mali drivers from rockchip, creating symlinks
• building opengl app
• creating image and creating sd card

If you are interested we could do a collaboration where you document the hard parts and I document the newbie parts.

Thanks!

[Miouyouyou]

Greetings @habemus-papadum ! 😃

Yeah, I'll try to put in place several tutorials concerning the installation, and the use of the Mali user-space binary drivers with :

• Raw OpenGL ES with the EGL + DRM/KMS/GBM backend
• SDL 2.0.6 + DRM backend, if I can make it work
  And a few scripts to install these drivers quickly, since installing them is quite a pain.

I also have to document how to use the cross-compilation script. Though, the script is now set up to use the default cross-compiler installed on Debian/Ubuntu systems when you do :
apt install crossbuild-essential-armhf
So, if you do that, you can just grab the script, run it with bash : bash ./GetPatchAndCompileKernel.sh, wait for its completion and you should have a /tmp/RockMyy-Build folder with a freshly cross-compiled kernel in it, that you can run on your board.

The hardest part being how to copy the files on your system and boot the new kernel. Generally this involves creating a tarball and send it to your board through SSH, using scp :

export YOUR_BOARD_IP=10.100.0.55
cd /tmp/RockMyy-Build
tar cJpvf kernel.tar.xz *
scp kernel.tar.xz $YOUR_BOARD_IP:/tmp
ssh $YOUR_BOARD_IP

And then install the files in the tarball, from the board :

# On the board, through SSH
cd /tmp
tar Jxpvf kernel.tar.xz
cp -r lib/* /lib/
cp -r usr/* /usr/

Then, for the boot/ files, you'll have to understand how your bootloader is setup. With my setup, I only do :

cp boot/* /boot

But that only works because I have an extlinux.conf that boots /boot/zImage and uses the dtb /boot/rk3288-miqi.dtb . If you look at the thread I linked, on the second page, I've documented my setup. That should be a good base for a documentation.

Now my setup is generally incompatible with U-boot scripts, so the user looking to install and test that new kernel will have to tinker the boot files installations depending on the bootloader configuration of its distribution, or forget about kernel updates provided by its distribution.
The best way to document this is to understand how the most popular distributions running on RK3288 (Armbian, TinkerOS, Elar Systems images, ...) are setup and provide a few appropriate commands to install the boot files in each case.
A better way would to provide an installation script, or prepackaged files, but that requires a lot of trial and error.

Now, any user wishing to do install a new custom kernel will have to understand how to get a serial connection to the board, and use it to debug issues when the board does not boot anymore. Because that will happen and is generally a source of panic for many users. So better understand how to do that BEFORE attempting to install new kernels that way.

For the board, at the moment, I can only recommend using a MiQi, since it's the only board I own and can test thoroughly. I'll try to get my hands on other boards. The Armbian team tried to get some ASUS Tinkerboard samples for me, but ASUS never replied to their requests 😕 . Also, this board requires patches to reboot the system correctly so... yeah...
If I can get my hands on a Firefly-RK3288, I'll try to test it, as that seems to be a quite solid system.

Meanwhile, I've never tested the SDCard boot. It might work on a MiQi but I frankly don't how it will fare.

Anyway, yeah, I'm interested in building a documentation and good for collaboration, as long as the documentation stays CC0 or CC-BY 🐱

[habemus-papadum]

@Miouyouyou
thanks for your comments.

I was thinking a little more about this, and here are my latest thoughts
toolchain: my linux box runs centos which never has any useful packages, but in any case I was thinking of using the linaro aarch64 tool chain

linaro tool chain comes with a sysroot, and works well enough on my centos box, and for this project I don't mind being lazy...

so the steps would be:

• userspace
  • copy mali userspace drivers into linaro sysroot
  • build opengl app using linaro tool chain
  • collect app and dependent shared libs into a root file system for the device (around 4-6 files total: app, ld-linux, libc, mali libs)
• kernel
  build with rockmyy script and linaro toolchain, but use menuconfig to properly configure initrd using the root file system from above

this bzImage is fully self contained and ready to go

some questions I'm not sure about --

• simplest way to boot kernel? as you said this seems hard. current choices are:
  • for tinkerboard, modify the uboot scripts used by tinkeros.
  • using your suggestions, scp into armian/tinkeros device and add another kernel -- i'm not quite sure about all the details
• how to know for sure that 3d graphics will go to hdmi port (highly likely, but not positive)? also, as you said, work out details for serial port during boot

I plan to play around with things on my end and keep posting notes here. on my todo:

• check that using linaro toolchain to build kernel and user space app works fine (hopefully this is easy)
• using existing distros, fig
  ure out details of uboot and also how to debug kernel boot over serial port
• combine everything together.
  (I have a tinkerboard)
  Sp maybe we will both just try things out until we figure out a workflow that is "easy" and then try to right up the results?

[Miouyouyou]

Toolchain

As long as the aarch64 toolchain can compile armv7 (32 bits) executables, it's fine. Else, you'll have to also get the usual armv7 (generally named "arm") toolchain from linaro too.

Compilation of user-space programs

Now, you don't really need to copy the Mali libraries to compile projects. This is nice. But you can also use the Mesa ones, if these are easier to obtain. You could even put placeholders libraries containing only the "right" symbols names with no actual code in them.

Note that when you compile, the main purpose of specifying the libraries names is to generate a list of files that will be put in the ELF executable.
Very roughly, when executed the ELF binary will be put into memory and the dynamic linker will be loaded. The address of the dynamic linker's dynamic symbol lookup function will be written as the address of every dynamic symbol in the in-memory ELF file so that, when the address of a dynamic symbol is currently unknown and must be looked up (dynamic symbol being generally the function name available in a dynamic library. i.e. : glCreateShader), the dynamic linker dynamic symbol lookup function will be executed (generally it's stored in /lib/ld.so.something on linux systems and another one on Android systems).
When executed, the dynamic linker dynamic symbol lookup function will load in memory the dynamic libraries put in the ELF executable libraries list, if not done already, and search for the asked symbol in each loaded library until it finds it. Then it will write the address back into in-memory ELF file so that future calls of the same dynamic symbol lead to a direct jump to the now "resolved" address.

The whole point of that diatribe is to understand that, the libraries during the compilation and the libraries during the execution can be entirely different, as long as they share the same symbols names.
What really matter is that the system executing the binary has the same libraries names with the same symbols names, and appropriate executable code in these libraries..
The ELF file only contains libraries "names". It doesn't care about the content of these libraries during the compilation.
During the compilation, the library can full of empty functions. As long that it has the right symbols (because these will be checked by GCC, just to be sure you're talking about the right library), this will compile.

So, you can compile against Mesa libraries and execute with Mali libraries and everything should be fine. Of course, if you can use the Mali ones for both, this will avoid any potential symbol mismatch issue.

Kernel compilation

Now, I wonder if you can't specify an initrd file to use directly through environment variables interpreted during the kernel compilation itself (Meaning, through environment variables setup by the kernel developers themselves).
Anyway, you'll need to execute the script like this :

MAKE_CONFIG=menuconfig bash ./GetPatchAndCompileKernel.sh

To get access to menuconfig when compiling with my script.

Booting

Now, for the boot part... it's always an issue when beginning.
I'll have to play with different U-Boot configurations on different distributions to provide useful hints.
I'll also repaste my extlinux.conf setup tomorrow.

Documentation

Anyway, yeah, this will require some trial and error and a documentation, or at least some notes, of these "trials", "errors" and "solutions". Documentation of the potential errors is required to avoid letting new comers in a broken state, which lead people to panic and/or disgust.

[habemus-papadum]

aarch64 -> uh, yeah, I was spacing out, sorry. Will use 32 bit toolchain

the main purpose of specifying the libraries names

fun fact, Apple no longer provides any binaries in the sysroots for all its devices it bundles with Xcode. instead there is a yaml file for each shared lib listing symbols names etc that the system linker knows how to process (each sysroot is still several 100MB...)

I thought to use the libmali drivers because it seemed like a pain to to install mesa! -- too big, too many pieces that I don't need (X support, proper opengl, etc) -- Copying over a few libmali libs and headers and wrangling a few symlinks felt much easier... Funny how people see the same thing in almost opposite terms... 😄

initrd -> I don't know about env variables, but I was going to menuconfig because from there you can form the initrd in two parts -- 1) a 'script' which can do things like properly layout /dev devices (/dev/null, etc) without having to call sudo, and 2) path to a local dir (containing app and shared libs) which will be cpio'd into one nice package

booting -- ah, booting -- that might be a pain...

I should start recording steps sometime next week.

[Miouyouyou]

fun fact, Apple no longer provides any binaries in the sysroots for all its devices it bundles with Xcode. instead there is a yaml file for each shared lib listing symbols names etc that the system linker knows how to process (each sysroot is still several 100MB...)

Well, that makes things clearer and easier to maintain 😃.

Yeah, using the mali drivers and just creating symlinks can be way faster than getting the whole Mesa artillery, indeed. Just wanted to clear some potential misconceptions.

Concerning the /boot/extlinux/extlinux.conf, it's defined like this :

label kernel-4.4
    kernel /boot/zImage
    fdt /boot/rk3288-miqi.dtb
    append  earlyprintk console=ttyS2,115200n8 rw root=/dev/mmcblk1p1 rootfstype=ext4 init=/sbin/init

It's a pretty standard extlinux.conf. As long as the right zImage and dtb files are copied in /boot, it should boot just fine.
Note that I do not have any boot.cmd or boot.scr file in the /boot folder. I think U-boot will parse these files first THEN the extlinux/extlinux.conf.
So, yeah, I guess that adding an initrd line with the path to the initrd image will do the trick.

I'll give it a try this week-end.

[habemus-papadum]

Cool -- So I have very little knowledge about uboot/extlinux.conf at this point.

but my basic plan for tinkerboard was to use a variant of the steps here to create a uboot image ont which I would add your extlinux.conf from above (with the tinker.dtb and kernel from this repo)

hopefully it will work!

[habemus-papadum]

also -- to be clear, my intention is to combine the initrd into the zImage as well as compile all modules statically into the kernel, so no initrd entry on the boot command line -- I guess this would be "classic, embedded style" as opposed to what distros do to provide the possibility for varied hardware and easy upgrade of the kernel... I don't know if this is possible with the dtb as well, but if it is, I will bundle that into the zImage as well

[Miouyouyou]

It should be possible to include everything inside the kernel. That basically what I was doing at first, until a few people asked me to include support for various hardware. Beside a few modules (mali_kbase and some Rockchip HDMI CEC modules), you can remove most the modules without any problem.

Note that some "staging" modules actually include the WiFi driver of the Tinkerboard though.

[habemus-papadum]

Note that some "staging" modules actually include the WiFi driver of the Tinkerboard though.

I'm not following... what are "staging modules"? In any case I was thinking of leaving all hardware support enabled, but just statically built. that way the final work flow is that you build the kernel once. and then when you iterate your app, you build it and then have kbuild redo the ziImage bunding, and deploy to device (initially via flash, but via tftp asap)

[habemus-papadum]

What happens when a touch screen is added to the board? (I don't have one). Does it show up as a second monitor or is video mirrored between the hdmi and touchscreen? Does the egl demo need to be updated to support users that may have different video configurations (touch screen only, etc)?