Bug#1078871: installer: reserve first 16 MiB space in default recipes for ARM devices?

[Holger Wansing]

Package: partman-auto



Date: Thu, 15 Aug 2024 00:47:22 +0200
From: "Diederik de Haas" <***@cknow.org>
To: "Pascal Hambourg" <***@plouf.fr.eu.org>, <***@bugs.debian.org>, "Philip Hands" <***@hands.com>, "Holger Wansing" <***@mailbox.org>
Cc: José Ángel Pastrana <***@gmail.com>, "Vagrant Cascadian" <***@debian.org>
Subject: Re: Bug#1076952: [RFD] partman-auto: Update guided partitioning size limits for current and future needs

Guided partitioning with LVM already provides a feature to reserve space
in the VG. Maybe it could be extended to guided partitioning with plain
partitions.

I'm not 100% sure if this fits into this subject/discussion, but ...

On ARM devices it would be very useful if the first 16MB would be
(automatically) reserved.
The U-Boot bootloader is normally put in the first part of the boot
device and for Rockchip based devices that can extend to the 16MB
'mark'. AFAIK bootloaders for other SoCs are before that.

If you use the current recipes you end up with an unbootable system as
the U-Boot bootloader get overwritten with the / (root) partition and
the data on it.
There should be a bug for it, but I didn't manage to find it.

Right now, the instruction is to choose manual partitioning and create
a 16MB partition ([1] says 32MB, but it should be 16MB [2]) and then the
normal partitions and after that you could remove that partition again.
And if you type in 16MB, then you need to 'hope' that it is actually
16MB and not something (a bit) smaller.
I like to think that I'm more advanced then most users and I found it
quite difficult to do.

So it would be very helpful if the recipe(s) for ARM devices would
reserve the first 16MB automatically with plain partitioning.

[1] https://wiki.pine64.org/wiki/ROCK64_Software_Releases#Debian
[2] https://opensource.rock-chips.com/wiki_Partitions

[Holger Wansing]

A summary from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1076952#65

Looks like another incarnation of
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=+770666>
is this still (or again) an issue?

----------------------------------------------------------------------------

Post by Holger Wansing
I'm not 100% sure if this fits into this subject/discussion, but ...

It is beyond the original scope (partition size limits) and I believe it
would deserve its own discussion involving people who are familiar with
ARM platforms.
Disclaimer: I have no experience nor knowledge about ARM (or any other
architectures than x86) and its boot process.

Post by Holger Wansing
The U-Boot bootloader is normally put in the first part of the boot
device and for Rockchip based devices that can extend to the 16MB
'mark'. AFAIK bootloaders for other SoCs are before that.
If you use the current recipes you end up with an unbootable system as
the U-Boot bootloader get overwritten with the / (root) partition and
the data on it.

AFAICS, the first partition in all non-EFI ARM recipes is /boot, not /.

Post by Holger Wansing
Right now, the instruction is to choose manual partitioning and create
a 16MB partition ([1] says 32MB, but it should be 16MB [2]) and then the
normal partitions and after that you could remove that partition again.
And if you type in 16MB, then you need to 'hope' that it is actually
16MB and not something (a bit) smaller.

16 MB (~15.3 MiB) or 16 MiB (~16.8 MB) ?
In partman, 1 MB really means 10^6 bytes, not 1 MiB (2^20 bytes).

Post by Holger Wansing
So it would be very helpful if the recipe(s) for ARM devices would
reserve the first 16MB automatically with plain partitioning.

What do you mean exactly by "plain partitioning" ? Manual partitioning ?
Guiding partitioning with all files in one filesystem ? Other ?

Post by Holger Wansing
[1] https://wiki.pine64.org/wiki/ROCK64_Software_Releases#Debian
[2] https://opensource.rock-chips.com/wiki_Partitions

I do not know any way to reserve unallocated space in recipes. The
recipes could create a 16-MiB unused partition but the table in [2]
lists a lot of special partitions within the first 16 MiB. Are these
actual partitions ? If yes, how are they supposed to be created ?

Looks like another incarnation of
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=+770666>

It looks like a different issue to me. IIUC these bug reports are about
parted_server erasing the boot loader location when creating a new
partition table, not the first partition overlapping with the boot
loader location.

----------------------------------------------------------------------------

Disclaimer: I have no experience nor knowledge about ARM (or any other
architectures than x86) and its boot process.

For partitioning there's nothing special ... apart that the first 16MiB
should not be used.

Post by Holger Wansing

Post by Holger Wansing
The U-Boot bootloader is normally put in the first part of the boot
device and for Rockchip based devices that can extend to the 16MB
'mark'. AFAIK bootloaders for other SoCs are before that.
If you use the current recipes you end up with an unbootable system as
the U-Boot bootloader get overwritten with the / (root) partition and
the data on it.

AFAICS, the first partition in all non-EFI ARM recipes is /boot, not /.

I shouldn't have written which partition, but the important thing is
that the first 'real' partition starts at 16 MiB.

Post by Holger Wansing

Post by Holger Wansing
Right now, the instruction is to choose manual partitioning and create
a 16MB partition ([1] says 32MB, but it should be 16MB [2]) and then the
normal partitions and after that you could remove that partition again.
And if you type in 16MB, then you need to 'hope' that it is actually
16MB and not something (a bit) smaller.

16 MB (~15.3 MiB) or 16 MiB (~16.8 MB) ?
In partman, 1 MB really means 10^6 bytes, not 1 MiB (2^20 bytes).

I think I've actually tried both and IIRC that didn't make a difference.

Post by Holger Wansing

Post by Holger Wansing
So it would be very helpful if the recipe(s) for ARM devices would
reserve the first 16MB automatically with plain partitioning.

What do you mean exactly by "plain partitioning" ? Manual partitioning ?
Guiding partitioning with all files in one filesystem ? Other ?

Partitioning NOT involving LVM.
I 'jumped in' when the subject of creating blank/reserved partitions
with LVM and then the question arose if that should also be used for
"plain" partitioning, which I interpreted as not using LVM.

Post by Holger Wansing

Post by Holger Wansing
[1] https://wiki.pine64.org/wiki/ROCK64_Software_Releases#Debian
[2] https://opensource.rock-chips.com/wiki_Partitions

I do not know any way to reserve unallocated space in recipes. The
recipes could create a 16-MiB unused partition but the table in [2]
lists a lot of special partitions within the first 16 MiB. Are these
actual partitions ? If yes, how are they supposed to be created ?

I think you technically could create those partitions, but those are not
relevant. All that matter is that the first 16 MiB stay unused so that
U-Boot can be put there.

Post by Holger Wansing
Looks like another incarnation of
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=+770666>

It looks like a different issue to me. IIUC these bug reports are about
parted_server erasing the boot loader location when creating a new
partition table, not the first partition overlapping with the boot
loader location.

AIUI bug #770666 is the same/similar as this 'Rockchip' issue.
Bug #751704 OTOH is related, but does deal with problems wrt partition
table. But I don't know/understand which of parted* sub programs is
responsible for which part.

----------------------------------------------------------------------------

Post by Holger Wansing
I do not know any way to reserve unallocated space in recipes. The
recipes could create a 16-MiB unused partition but the table in [2]
lists a lot of special partitions within the first 16 MiB. Are these
actual partitions ? If yes, how are they supposed to be created ?

I think you technically could create those partitions, but those are not
relevant. All that matter is that the first 16 MiB stay unused so that
U-Boot can be put there.

It is still unclear to me if it can be an unused partition or if it must
be unallocated space which can be used to create new partitions.
AFAIK, only the former can be done in recipes.

Post by Holger Wansing

Post by Holger Wansing
Looks like another incarnation of
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=770666>

It looks like a different issue to me. IIUC these bug reports are about
parted_server erasing the boot loader location when creating a new
partition table, not the first partition overlapping with the boot
loader location.

AIUI bug #770666 is the same/similar as this 'Rockchip' issue.
Bug #751704 OTOH is related, but does deal with problems wrt partition
table.

AIUI, #751704 and #770666 are the same bug for different platforms and
are unrelated with the first partition position. They are caused by
parted erasing the first 9 KiB when creating a new partition table. The
fix for #770666 only extended the fix for #751704 to handle more
platforms. The 'Rockchip' issue you mention is unrelated as the boot
loader is located after 32 KiB, beyond the erased area.

----------------------------------------------------------------------------

It is still unclear to me if it can be an unused partition or if it must
be unallocated space which can be used to create new partitions.
AFAIK, only the former can be done in recipes.

Either would work.

----------------------------------------------------------------------------

Post by Holger Wansing

Post by Holger Wansing
All that matter is that the first 16 MiB stay unused so that
U-Boot can be put there.

It is still unclear to me if it can be an unused partition or if it must
be unallocated space which can be used to create new partitions.
AFAIK, only the former can be done in recipes.

Either would work.

Then I guess a 16 MiB unused partition could be added to relevant
recipes. Now, which are the relevant recipes ? In other words, which
arch/subarch need it ?
recipes-armel-kirkwood
recipes-armel-orion5x
recipes-armhf-efikasb
recipes-armhf-efi (=recipes-amd64-efi)
recipes-armhf
recipes-arm64-efi (=recipes-amd64-efi)
recipes-arm64 (=recipes-armhf)

----------------------------------------------------------------------------

Then I guess a 16 MiB unused partition could be added to relevant
recipes. Now, which are the relevant recipes ? In other words, which
arch/subarch need it ?
recipes-armel-kirkwood
recipes-armel-orion5x
recipes-armhf-efikasb

These ones are NOT relevant for this.

recipes-armhf-efi (=recipes-amd64-efi)
recipes-armhf
recipes-arm64-efi (=recipes-amd64-efi)
recipes-arm64 (=recipes-armhf)

Rockchip makes both 32bit as 64bit ARM SoCs, so `recipes-armhf` is
relevant.
I don't know if it's common, but AFAIK you can use U-Boot with EFI, but
it sounds 'weird' to add it to a recipe with AMD64 in its name...

----------------------------------------------------------------------------

Post by Holger Wansing
Then I guess a 16 MiB unused partition could be added to relevant
recipes. Now, which are the relevant recipes ? In other words, which
arch/subarch need it ?
recipes-armhf-efi (= recipes-amd64-efi)
recipes-armhf
recipes-arm64-efi (= recipes-amd64-efi)
recipes-arm64 (= recipes-armhf)

Rockchip makes both 32bit as 64bit ARM SoCs, so `recipes-armhf` is
relevant.

If only Rockchip SoCs need the reserved partition and are detected as a
specific subarchitecture by archdetect, new specific recipes for this
subarchitecture could be added.

I don't know if it's common, but AFAIK you can use U-Boot with EFI, but
it sounds 'weird' to add it to a recipe with AMD64 in its name...

Indeed. If some ARM EFI platforms need the reserved partition, then one
of the recipes-arm*-efi symlinks could be replaced with a directory
containing new specific recipes and the other could be changed to point
to it.

[Diederik de Haas]

Copying my latest response on this subject into this bug (too) ...

A summary from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug76952#65
----------------------------------------------------------------------------

Then I guess a 16 MiB unused partition could be added to relevant
recipes. Now, which are the relevant recipes ? In other words, which
arch/subarch need it ?
recipes-armhf-efi (= recipes-amd64-efi)
recipes-armhf
recipes-arm64-efi (= recipes-amd64-efi)
recipes-arm64 (= recipes-armhf)

Rockchip makes both 32bit as 64bit ARM SoCs, so `recipes-armhf` is
relevant.

If only Rockchip SoCs need the reserved partition and are detected as a
specific subarchitecture by archdetect, new specific recipes for this
subarchitecture could be added.

I used Rockchip as an example as I'm familiar with that AND *afaik* they
expect the U-Boot stuff to start at the furthest offset.

But when U-Boot is used, every platform uses *some* offset and then needs
some space/bytes for the U-Boot binary/bits.
If that goes above the 2MB mark, then the current recipes overwrite part
of whole of the U-Boot binary ... making the system unbootable.

So you could special case Rockchip and only use the 16MB offset for the
Operating System partitions and data for Rockchip SoCs.
Or you could keep the first 16MB free for all ARM SoCs and then it
should work for all ARM SoCs as long as they stay below the 16MB 'mark'.
Which AFAIK are all of them.

I don't know if it's common, but AFAIK you can use U-Boot with EFI, but
it sounds 'weird' to add it to a recipe with AMD64 in its name...

Indeed. If some ARM EFI platforms need the reserved partition, then one
of the recipes-arm*-efi symlinks could be replaced with a directory
containing new specific recipes and the other could be changed to point
to it.

And the above principle also applies when EFI is being used with U-Boot
as it's specific to U-Boot.

[Johannes Schauer Marin Rodrigues]

16 MB (~15.3 MiB) or 16 MiB (~16.8 MB) ?
In partman, 1 MB really means 10^6 bytes, not 1 MiB (2^20 bytes).

I think I've actually tried both and IIRC that didn't make a difference.

for the MNT Reform we recently bumped the start of the first partition to be at
16777216 (16 MiB), i.e. in our scripts we run:

/sbin/parted -s "$SYSIMAGE.img" "mkpart primary ext4 16MiB ..."

For the longest time, our offset for the first partition was at 4194304 bytes
but since there is now the rk3588 module that we want to support, this had to
be bumped quite a bit. Initially we had it at 12 MiB (which worked fine) but
after having read https://opensource.rock-chips.com/wiki_Partitions this now
became 16 MiB just to be on the safe-side (no idea how the user will be
repurposing their system) and because 4 MiB extra do not add much in the grand
scheme of things...

Thanks!

cheers, josch

[Pascal Hambourg]

Shouldn't this bug be advertised in debian-***@lists.debian.org so that
people familiar with ARM platforms are aware of it ?

[Holger Wansing]

Indeed, I intended to do so, but forgot then.
Thanks

[Debian Bug Tracking System]

tags -1 patch

Bug #1078871 [partman-auto] installer: reserve first 16 MiB space in default recipes for ARM devices?
Added tag(s) patch.

[Pascal Hambourg]

Control: tags -1 patch

Hi arm people,
could you please comment on this bug, please?
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1078871>
Is there a need for some change?
https://salsa.debian.org/installer-team/partman-auto/-/merge_requests/18>

Feedback welcome.
I cannot test it as I do not have any ARM hardware.

[Holger Wansing]

Control: tags -1 + pending

[Debian Bug Tracking System]

Post by Holger Wansing
tags -1 + pending

Bug #1078871 [partman-auto] installer: reserve first 16 MiB space in default recipes for ARM devices?
Added tag(s) pending.

[Debian Bug Tracking System]

Your message dated Tue, 08 Oct 2024 21:23:58 +0000
with message-id <E1syHgU-006vdB-***@fasolo.debian.org>
and subject line Bug#1078871: fixed in partman-auto 168
has caused the Debian Bug report #1078871,
regarding installer: reserve first 16 MiB space in default recipes for ARM devices?
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ***@bugs.debian.org
immediately.)

[Holger Wansing]

Hi,

Post by Holger Wansing
Date: Thu, 15 Aug 2024 00:47:22 +0200
Subject: Re: Bug#1076952: [RFD] partman-auto: Update guided partitioning size limits for current and future needs

Guided partitioning with LVM already provides a feature to reserve space
in the VG. Maybe it could be extended to guided partitioning with plain
partitions.

I'm not 100% sure if this fits into this subject/discussion, but ...
On ARM devices it would be very useful if the first 16MB would be
(automatically) reserved.
The U-Boot bootloader is normally put in the first part of the boot
device and for Rockchip based devices that can extend to the 16MB
'mark'. AFAIK bootloaders for other SoCs are before that.
If you use the current recipes you end up with an unbootable system as
the U-Boot bootloader get overwritten with the / (root) partition and
the data on it.
There should be a bug for it, but I didn't manage to find it.
Right now, the instruction is to choose manual partitioning and create
a 16MB partition ([1] says 32MB, but it should be 16MB [2]) and then the
normal partitions and after that you could remove that partition again.
And if you type in 16MB, then you need to 'hope' that it is actually
16MB and not something (a bit) smaller.
I like to think that I'm more advanced then most users and I found it
quite difficult to do.
So it would be very helpful if the recipe(s) for ARM devices would
reserve the first 16MB automatically with plain partitioning.
[1] https://wiki.pine64.org/wiki/ROCK64_Software_Releases#Debian
[2] https://opensource.rock-chips.com/wiki_Partitions

This is now included in testing.
@Diederik: could you test, if it works for you now?
Or anyone else, who has the possibility/the hardware for this?


Thanks
Holger

[Pascal Hambourg]

Hi Diederik,

First, thank you for the detailed test report !

Post by Holger Wansing
@Diederik: could you test, if it works for you now?
Or anyone else, who has the possibility/the hardware for this?

(...)

I forgot to write down which partition option I choose, but I _think_ it
was 'use the entire disk' for 1 partition. More on that later as I want
to verify that.
1.0 MB FREE SPACE
#1 16.8 MB
#2 896.5 MB f ext2 /boot
#3 14.0 GB f ext4 /
#4 825.2 MB f swap swap
1.0 MB FREE SPACE

It looks as expected with the "atomic" recipe.

Disk /dev/sdb: 14.64 GiB, 15720251392 bytes, 30703616 sectors
Disk model: SDDR-389
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: DE22E911-3163-4844-A95C-56175B710651
Device Start End Sectors Size Type
/dev/sdb1 2048 34815 32768 16M Linux filesystem
/dev/sdb2 34816 1785855 1751040 855M Linux filesystem
/dev/sdb3 1785856 29089791 27303936 13G Linux filesystem
/dev/sdb4 29089792 30701567 1611776 787M Linux swap

It looks as expected too.

According to [2] (wiki_Partitions) the boot partition start sector
should be 32768, but AFAIK it's fine if it's later, so that's good.

Automatic partitioning has a granularity of 1MB (decimal) which is then
combined with parted's default alignment on 1MiB (binary) boundaries, so
it is not easy to get exact results.

I already had a hunch why that may be and that's when I examined the
SDcard with `lsblk` and later with `fdisk`. Went into `fdisk`'s expert
menu and enabled the boot flag on partition 2 (`/boot`).
Plugged the SDcard into my Rock64 again and booted up and then it
succeeded \o/
IOW: It was so, so close from working ... but it needs the 'boot' flag.

Do you mean the "legacy BIOS bootable flag" (legacy_boot in parted) ?
Unfortunately partman does not manage it. IIRC the "$bootable{ }" flag
in recipes is translated into parted "boot" flag, but on GPT the "boot"
flag means the same as "esp" (this is a huge mess) and partman clears
them if the partition method is not "efi". It should not be hard to
write a patch to translate the "$bootable{ }" flag into the
"legacy_boot" flag instead of the "boot" flag on GPT.

I'll (later) do another install attempt to verify whether it actually
should NOT have created a boot partition or that I just mis-remembered.

All existing arm* recipes always created a separated ext2 /boot
partition, so I kept it in the new arm64 recipes. If this is not needed,
the arm64 recipes can be changed so that they create a separate /boot
partition only when partitioning with LVM, just like recipes for other
architectures.

[Diederik de Haas]

Hi Pascal,

Post by Pascal Hambourg
First, thank you for the detailed test report !

You're welcome.

Post by Pascal Hambourg

Post by Holger Wansing
@Diederik: could you test, if it works for you now?
Or anyone else, who has the possibility/the hardware for this?

(...)

I forgot to write down which partition option I choose, but I _think_ it
was 'use the entire disk' for 1 partition. More on that later as I want
to verify that.
1.0 MB FREE SPACE
#1 16.8 MB
#2 896.5 MB f ext2 /boot
#3 14.0 GB f ext4 /
#4 825.2 MB f swap swap
1.0 MB FREE SPACE

It looks as expected with the "atomic" recipe.

Disk /dev/sdb: 14.64 GiB, 15720251392 bytes, 30703616 sectors
Disk model: SDDR-389
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: DE22E911-3163-4844-A95C-56175B710651
Device Start End Sectors Size Type
/dev/sdb1 2048 34815 32768 16M Linux filesystem
/dev/sdb2 34816 1785855 1751040 855M Linux filesystem
/dev/sdb3 1785856 29089791 27303936 13G Linux filesystem
/dev/sdb4 29089792 30701567 1611776 787M Linux swap

It looks as expected too.

According to [2] (wiki_Partitions) the boot partition start sector
should be 32768, but AFAIK it's fine if it's later, so that's good.

Automatic partitioning has a granularity of 1MB (decimal) which is then
combined with parted's default alignment on 1MiB (binary) boundaries, so
it is not easy to get exact results.

If it helps, 32768 is exactly 16MiB. So with a partition size of 15MiB
combined with the 1MiB (2048 sectors) offset parted uses, that should be
right on the mark. But AFAIC that's not needed.

Post by Pascal Hambourg

I already had a hunch why that may be and that's when I examined the
SDcard with `lsblk` and later with `fdisk`. Went into `fdisk`'s expert
menu and enabled the boot flag on partition 2 (`/boot`).
Plugged the SDcard into my Rock64 again and booted up and then it
succeeded \o/
IOW: It was so, so close from working ... but it needs the 'boot' flag.

Do you mean the "legacy BIOS bootable flag" (legacy_boot in parted) ?

I believe so, yes.

Post by Pascal Hambourg
Unfortunately partman does not manage it. IIRC the "$bootable{ }" flag
in recipes is translated into parted "boot" flag, but on GPT the "boot"
flag means the same as "esp" (this is a huge mess) and partman clears
them if the partition method is not "efi". It should not be hard to
write a patch to translate the "$bootable{ }" flag into the
"legacy_boot" flag instead of the "boot" flag on GPT.

I'll (later) do another install attempt to verify whether it actually
should NOT have created a boot partition or that I just mis-remembered.

All existing arm* recipes always created a separated ext2 /boot
partition, so I kept it in the new arm64 recipes. If this is not needed,
the arm64 recipes can be changed so that they create a separate /boot
partition only when partitioning with LVM, just like recipes for other
architectures.

I never create a separate /boot partition, which has several advantages
IMO. But the ARM ecosystem is ... let's say ... diverse.
So I don't know if other systems would/could need a separate /boot
partition. But Rockchip devices don't need it.

And I'm hereby going to *assume* that my observation was correct and
then it is confusing if you choose to NOT create a separate boot
partition, that you still get one.
I think there is another choice in the installer which does create a
separate /boot partition? If so, then the 'all in one partition' should
do exactly that and if ppl want a separate /boot partition, they should
choose that other option. IOW: give ppl what they ask for ;-)

Cheers,
Diederik

[Pascal Hambourg]

Post by Diederik de Haas

Post by Pascal Hambourg

Device Start End Sectors Size Type
/dev/sdb1 2048 34815 32768 16M Linux filesystem
/dev/sdb2 34816 1785855 1751040 855M Linux filesystem
/dev/sdb3 1785856 29089791 27303936 13G Linux filesystem
/dev/sdb4 29089792 30701567 1611776 787M Linux swap

It looks as expected too.

According to [2] (wiki_Partitions) the boot partition start sector
should be 32768, but AFAIK it's fine if it's later, so that's good.

Automatic partitioning has a granularity of 1MB (decimal) which is then
combined with parted's default alignment on 1MiB (binary) boundaries, so
it is not easy to get exact results.

If it helps, 32768 is exactly 16MiB. So with a partition size of 15MiB
combined with the 1MiB (2048 sectors) offset parted uses, that should be
right on the mark. But AFAIC that's not needed.

The requested size in the recipe is 18MB and it results in an actual
size of 16MiB. If I change the requested size to 17MB, then the
resulting size is 15MiB and it ends at sector 32767. But I'd rather be
on the safe side and keep some margin.

Post by Diederik de Haas
And I'm hereby going to *assume* that my observation was correct and
then it is confusing if you choose to NOT create a separate boot
partition, that you still get one.
I think there is another choice in the installer which does create a
separate /boot partition? If so, then the 'all in one partition' should
do exactly that and if ppl want a separate /boot partition, they should
choose that other option. IOW: give ppl what they ask for ;-)

No, there is no user choice to create a separate /boot partition or not.
Depending on the architecture, all recipes either:
- never create a separate /boot partition
- always create a separate /boot partition
- create a separate /boot partition only when partitioning with LVM.

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Diederik de Haas (2024-10-19 21:08:37)

Post by Diederik de Haas

All existing arm* recipes always created a separated ext2 /boot partition,
so I kept it in the new arm64 recipes. If this is not needed, the arm64
recipes can be changed so that they create a separate /boot partition only
when partitioning with LVM, just like recipes for other architectures.

I never create a separate /boot partition, which has several advantages
IMO. But the ARM ecosystem is ... let's say ... diverse.
So I don't know if other systems would/could need a separate /boot partition.
But Rockchip devices don't need it.

just some thoughts on the topic of having a separate /boot partition.

For the MNT Reform system images we use a separate `/boot` partition by default
because:

- support encrypted rootfs (luks)
- allow bit-by-bit identical rootfs for all supported boards (`/boot` differs
because /boot/dtb and /boot/dtb-$(uname -r) symlink contents differ)
- kernel, initrd and dtb being at the root of the partition allows for simply
mounting it on /boot from another system -- with everything in a single
partition, one'd get /boot/boot

Thanks!

cheers, josch

P.S.: eagerly awaiting my own rk3588 to arrive

[Diederik de Haas]

Hi,

Post by Johannes Schauer Marin Rodrigues
Quoting Diederik de Haas (2024-10-19 21:08:37)

Post by Diederik de Haas
I never create a separate /boot partition, which has several advantages
IMO. But the ARM ecosystem is ... let's say ... diverse.
So I don't know if other systems would/could need a separate /boot partition.
But Rockchip devices don't need it.

just some thoughts on the topic of having a separate /boot partition.
For the MNT Reform system images we use a separate `/boot` partition by default
- support encrypted rootfs (luks)
- allow bit-by-bit identical rootfs for all supported boards (`/boot` differs
because /boot/dtb and /boot/dtb-$(uname -r) symlink contents differ)
- kernel, initrd and dtb being at the root of the partition allows for simply
mounting it on /boot from another system -- with everything in a single
partition, one'd get /boot/boot

IIRC having a separate `/boot` means that the dtb files need to be
copied over (via a kernel hook?), while that isn't needed when there is
no separate `/boot`.

But I haven't had much time to fully work those parts out; my image
creation project hasn't seen any activity for 6 months now.
Maybe during the Forky cycle?

Post by Johannes Schauer Marin Rodrigues
P.S.: eagerly awaiting my own rk3588 to arrive

I (still) don't have an rk3588 device, but I did recently receive a
PineNote \o/ and I'm *really* excited about that :-D
Hopefully in Forky with full upstream and Debian support for it :)

Cheers,
Diederik

[Holger Wansing]

Hi all,

Post by Pascal Hambourg
Hi Diederik,
First, thank you for the detailed test report !

IOW: It was so, so close from working ... but it needs the 'boot' flag.

Do you mean the "legacy BIOS bootable flag" (legacy_boot in parted) ?
Unfortunately partman does not manage it. IIRC the "$bootable{ }" flag
in recipes is translated into parted "boot" flag, but on GPT the "boot"
flag means the same as "esp" (this is a huge mess) and partman clears
them if the partition method is not "efi". It should not be hard to
write a patch to translate the "$bootable{ }" flag into the
"legacy_boot" flag instead of the "boot" flag on GPT.

I'll (later) do another install attempt to verify whether it actually
should NOT have created a boot partition or that I just mis-remembered.

All existing arm* recipes always created a separated ext2 /boot
partition, so I kept it in the new arm64 recipes. If this is not needed,
the arm64 recipes can be changed so that they create a separate /boot
partition only when partitioning with LVM, just like recipes for other
architectures.

Sorry guys, I cannot follow you here.
This Rockchip system uses the legacy msdos partition table, right? (No UEFI,
thus no GPT).
With this partition table it should be possible and is confirmed to be
working, to set the bootable flag for a partition AFAIK (like an ext2 /boot
partition).

So, what's the problem here? What went wrong?
It should have worked, I think...


Holger

[Pascal Hambourg]

Post by Holger Wansing
This Rockchip system uses the legacy msdos partition table, right? (No UEFI,
thus no GPT).
With this partition table it should be possible and is confirmed to be
working, to set the bootable flag for a partition AFAIK (like an ext2 /boot
partition).

In Diederik's report the partition table is GPT, not MSDOS. Indeed the
default disk label (partition table) for the arm64 architecture is GPT,
regardless of whether the sub-architecture is EFI or not (see
<https://salsa.debian.org/installer-team/partman-partitioning/-/blob/master/lib/disk-label.sh>).

[Diederik de Haas]

Post by Pascal Hambourg

Post by Holger Wansing
This Rockchip system uses the legacy msdos partition table, right? (No UEFI,
thus no GPT).
With this partition table it should be possible and is confirmed to be
working, to set the bootable flag for a partition AFAIK (like an ext2 /boot
partition).

In Diederik's report the partition table is GPT, not MSDOS. Indeed the
default disk label (partition table) for the arm64 architecture is GPT,
regardless of whether the sub-architecture is EFI or not (see
<https://salsa.debian.org/installer-team/partman-partitioning/-/blob/master/lib/disk-label.sh>).

Correct, I always use GPT partitioning.
AFAIUI, UEFI requires GPT, but GPT does not require UEFI.

And as it happens to be the case, I have one device which actually does
use UEFI (I think; and GRUB) and another one which doesn't.
This was likely the result of some experiment just to see what would
happen. I do not recall any details about what or why I did (it).

The 'cs21' device is a (different) Rock64 (rk3328):

```
***@cs21:~$ ls -lh /boot/
total 242M
...
-rw-r--r-- 1 root root 285K sep 30 21:08 config-6.1.0-26-arm64
drwxr-xr-x 4 root root 16K jan 1 1970 efi
drwxr-xr-x 2 root root 4,0K okt 9 18:02 extlinux
drwxr-xr-x 5 root root 4,0K okt 9 18:03 grub
-rw-r--r-- 1 root root 30M okt 9 18:02 initrd.img-6.1.0-26-arm64
-rw-r--r-- 1 root root 83 sep 30 21:08 System.map-6.1.0-26-arm64
-rw-r--r-- 1 root root 32M sep 30 21:08 vmlinuz-6.1.0-26-arm64
```

I have 4 kernels installed, but listing them all isn't useful.

And now the partition stuff:

```
***@cs21:~# parted -s /dev/mmcblk1 print
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary

***@cs21:~# lsblk -o +PARTFLAGS,FSTYPE /dev/mmcblk1
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```

And 'quartz64a' is a Pine64 Quartz64 Model A (rk3566):

```
***@quartz64a:~# parted -s /dev/mmcblk1 print
Model: MMC A3A562 (sd/mmc)
Disk /dev/mmcblk1: 124GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 32.8kB 3670kB 3637kB loader1_part
2 8389kB 12.6MB 4194kB loader2_part
3 12.6MB 16.8MB 4194kB trust_part
4 16.8MB 134MB 117MB efi_part
5 134MB 124GB 124GB ext4 root_part legacy_boot

***@quartz64a:~# lsblk -o +PARTFLAGS,FSTYPE /dev/mmcblk1
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 115.2G 0 disk
├─mmcblk1p1 179:1 0 3.5M 0 part
├─mmcblk1p2 179:2 0 4M 0 part
├─mmcblk1p3 179:3 0 4M 0 part
├─mmcblk1p4 179:4 0 112M 0 part
└─mmcblk1p5 179:5 0 115.1G 0 part / 0x4 ext4
```

FTR: I cannot think of any reason why the device type would matter.
And the Quartz64-A has those other partitions, but doesn't use them.
Just some other experiment with the 'official' Rockchip partition
layout.

And I just changed the 'legacy_boot' flag to 'boot' on the SDcard with
which I did my test, which then (apparently) also triggers the 'esp'
flag and it booted with that too.
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Cheers,
Diederik

[Pascal Hambourg]

Post by Diederik de Haas
AFAIUI, UEFI requires GPT, but GPT does not require UEFI.

No, UEFI does not require GPT, at least not on my amd64 machines
(including QEMU+OVMF). It also supports booting on MSDOS.

Post by Diederik de Haas
And as it happens to be the case, I have one device which actually does
use UEFI (I think; and GRUB) and another one which doesn't.

(...)

Post by Diederik de Haas
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```

Partition 2 looks like a regular EFI partition mounted on /boot/efi as
expected by GRUB. With a size of 239MB it should be FAT32 instead of
FAT16, but maybe the UEFI firmware expects only FAT16 or does not care.
No legacy_boot flag on any partition.

Post by Diederik de Haas
```
Model: MMC A3A562 (sd/mmc)
Disk /dev/mmcblk1: 124GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 3670kB 3637kB loader1_part
2 8389kB 12.6MB 4194kB loader2_part
3 12.6MB 16.8MB 4194kB trust_part
4 16.8MB 134MB 117MB efi_part
5 134MB 124GB 124GB ext4 root_part legacy_boot
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 115.2G 0 disk
├─mmcblk1p1 179:1 0 3.5M 0 part
├─mmcblk1p2 179:2 0 4M 0 part
├─mmcblk1p3 179:3 0 4M 0 part
├─mmcblk1p4 179:4 0 112M 0 part
└─mmcblk1p5 179:5 0 115.1G 0 part / 0x4 ext4
```

Partitions 1-3 cover the first 16MiB, like cs21's partition 1 does.
Partition 4 has the name 'efi_part' but it does not have the 'esp' flag
and parted does not detect any filesystem, so I guess it is just empty
and unused too. The root partition has the 'legacy_boot' flag like the
Rock64 with which you did your test.

Post by Diederik de Haas
And I just changed the 'legacy_boot' flag to 'boot' on the SDcard with
which I did my test, which then (apparently) also triggers the 'esp'
flag and it booted with that too.

On GPT, parted 'boot' flag has the same meaning as 'esp' and both
represent the 'EFI System Partition' type GUID. 'legacy_boot' is a real
flag in a bit field.

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

Note: partman allows to create EFI partitions only when the installer
was booted in EFI mode. I would like to change this, at least on
EFI-capable architectures such as x86 or ARM.

[Diederik de Haas]

Post by Pascal Hambourg

Post by Diederik de Haas
AFAIUI, UEFI requires GPT, but GPT does not require UEFI.

No, UEFI does not require GPT, at least not on my amd64 machines
(including QEMU+OVMF). It also supports booting on MSDOS.

Ok, good to know.

Post by Pascal Hambourg

Post by Diederik de Haas
And as it happens to be the case, I have one device which actually does
use UEFI (I think; and GRUB) and another one which doesn't.

(...)

Post by Diederik de Haas
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```

Partition 2 looks like a regular EFI partition mounted on /boot/efi as
expected by GRUB. With a size of 239MB it should be FAT32 instead of
FAT16, but maybe the UEFI firmware expects only FAT16 or does not care.

I don't care enough (about FAT) to look into it, but I think 'vfat' can
mean either FAT16 or FAT32?
And looking at the size of partition 1, I'm quite sure I installed that
system with d-i, but went through the hoops to leave it blank.

Post by Pascal Hambourg
No legacy_boot flag on any partition.

Indeed, but it does have 'boot' (and thus 'esp') and that's why I tried
whether setting 'boot' instead of 'legacy_boot' would work.

Post by Pascal Hambourg

Post by Diederik de Haas
```
Model: MMC A3A562 (sd/mmc)
Disk /dev/mmcblk1: 124GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 3670kB 3637kB loader1_part
2 8389kB 12.6MB 4194kB loader2_part
3 12.6MB 16.8MB 4194kB trust_part
4 16.8MB 134MB 117MB efi_part
5 134MB 124GB 124GB ext4 root_part legacy_boot
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 115.2G 0 disk
├─mmcblk1p1 179:1 0 3.5M 0 part
├─mmcblk1p2 179:2 0 4M 0 part
├─mmcblk1p3 179:3 0 4M 0 part
├─mmcblk1p4 179:4 0 112M 0 part
└─mmcblk1p5 179:5 0 115.1G 0 part / 0x4 ext4
```

Partitions 1-3 cover the first 16MiB, like cs21's partition 1 does.
Partition 4 has the name 'efi_part' but it does not have the 'esp' flag
and parted does not detect any filesystem, so I guess it is just empty
and unused too. The root partition has the 'legacy_boot' flag like the
Rock64 with which you did your test.

FWIW: I set those partition names explicitly myself and created the
partitions either with directly calling parted or indirectly via
`debos`.
I (highly) likely did that to test whether those partitions were
relevant at all as I'd `dd` a 9.1MB `u-boot-rockchip.bin` file:
``dd if=u-boot-rockchip.bin of=/dev/mmcblk0 bs=32k seek=1 conv=fsync``

Conclusion: those partitions are NOT relevant

Post by Pascal Hambourg

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

It appears to not matter if it's 'boot' or 'legacy_boot', but it (ofc?)
needs to be set on the partition with the kernel as that needs to be
started for the rest of the boot process.
My guess is that on the tested system it tried to do something with TFTP
as it couldn't find a kernel as the '[legacy_]boot' flag was not set,
thus couldn't find a kernel thus tried another option (tftp).

Cheers,
Diederik

[Pascal Hambourg]

Post by Diederik de Haas

Post by Pascal Hambourg

Post by Diederik de Haas
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```

Partition 2 looks like a regular EFI partition mounted on /boot/efi as
expected by GRUB. With a size of 239MB it should be FAT32 instead of
FAT16, but maybe the UEFI firmware expects only FAT16 or does not care.

I don't care enough (about FAT) to look into it, but I think 'vfat' can
mean either FAT16 or FAT32?

Yes, but some UEFI firmware are picky about the combinations of FAT type
and partition size they accept.

Post by Diederik de Haas

Post by Pascal Hambourg

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

It appears to not matter if it's 'boot' or 'legacy_boot', but it (ofc?)
needs to be set on the partition with the kernel as that needs to be
started for the rest of the boot process.

But in cs21's partition table, the boot/esp flag is on the EFI
partition, not the root partition. Does it mean that the kernel is
installed in the EFI partition instead of the root partition ? Or is the
"kernel" a secondary EFI boot loader such as GRUB ?

[Diederik de Haas]

Post by Pascal Hambourg

Post by Diederik de Haas

Post by Pascal Hambourg

Post by Diederik de Haas
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```

Partition 2 looks like a regular EFI partition mounted on /boot/efi as
expected by GRUB. With a size of 239MB it should be FAT32 instead of
FAT16, but maybe the UEFI firmware expects only FAT16 or does not care.

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

It appears to not matter if it's 'boot' or 'legacy_boot', but it (ofc?)
needs to be set on the partition with the kernel as that needs to be
started for the rest of the boot process.

But in cs21's partition table, the boot/esp flag is on the EFI
partition, not the root partition. Does it mean that the kernel is
installed in the EFI partition instead of the root partition ? Or is the
"kernel" a secondary EFI boot loader such as GRUB ?

Good point, my use of 'kernel' was inaccurate.

After u-boot does its thing, it handles control over to 'something'
which continues the boot process.
In most of my cases that is the kernel, but on cs21 I _think_ that it
indeed handles things over to GRUB which then starts the requested
system/kernel.
On cs21 I do have `grub-efi-arm64` installed, but I don't know much
about EFI things, but it works so I had no reason to change it.

Cheers,
Diederik

[Holger Wansing]

Hi,

Post by Diederik de Haas
```
total 242M
...
-rw-r--r-- 1 root root 285K sep 30 21:08 config-6.1.0-26-arm64
drwxr-xr-x 4 root root 16K jan 1 1970 efi
drwxr-xr-x 2 root root 4,0K okt 9 18:02 extlinux
drwxr-xr-x 5 root root 4,0K okt 9 18:03 grub
-rw-r--r-- 1 root root 30M okt 9 18:02 initrd.img-6.1.0-26-arm64
-rw-r--r-- 1 root root 83 sep 30 21:08 System.map-6.1.0-26-arm64
-rw-r--r-- 1 root root 32M sep 30 21:08 vmlinuz-6.1.0-26-arm64
```
I have 4 kernels installed, but listing them all isn't useful.
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 57.6G 0 disk
├─mmcblk1p1 179:1 0 16M 0 part
├─mmcblk1p2 179:2 0 228M 0 part /boot/efi vfat
└─mmcblk1p3 179:3 0 57.4G 0 part / ext4
```
```
Model: MMC A3A562 (sd/mmc)
Disk /dev/mmcblk1: 124GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 3670kB 3637kB loader1_part
2 8389kB 12.6MB 4194kB loader2_part
3 12.6MB 16.8MB 4194kB trust_part
4 16.8MB 134MB 117MB efi_part
5 134MB 124GB 124GB ext4 root_part legacy_boot
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS PARTFLAGS FSTYPE
mmcblk1 179:0 0 115.2G 0 disk
├─mmcblk1p1 179:1 0 3.5M 0 part
├─mmcblk1p2 179:2 0 4M 0 part
├─mmcblk1p3 179:3 0 4M 0 part
├─mmcblk1p4 179:4 0 112M 0 part
└─mmcblk1p5 179:5 0 115.1G 0 part / 0x4 ext4
```
FTR: I cannot think of any reason why the device type would matter.
And the Quartz64-A has those other partitions, but doesn't use them.
Just some other experiment with the 'official' Rockchip partition
layout.
And I just changed the 'legacy_boot' flag to 'boot' on the SDcard with
which I did my test, which then (apparently) also triggers the 'esp'
flag and it booted with that too.
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Hmm, I'm irritated now:
here you have a device|installation, which has the legacy_boot flag
installed, but it did not work|boot despite of this. And you changed that
legacy_boot flag into a boot flag, and that made the problem go away?
(so the device boots fine).

So, why should we then do some work to implement the setting of the
legacy_boot flag on arm64?

Post by Diederik de Haas

Post by Pascal Hambourg

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

It appears to not matter if it's 'boot' or 'legacy_boot', but it (ofc?)
needs to be set on the partition with the kernel as that needs to be
started for the rest of the boot process.
My guess is that on the tested system it tried to do something with TFTP
as it couldn't find a kernel as the '[legacy_]boot' flag was not set,
thus couldn't find a kernel thus tried another option (tftp).

Here you also imply, that both boot or legacy_boot flag do the job.


So, can I understand this, as there is no need to set the legacy_boot flag,
but just adjust the situations, where the 'boot' flag is being set?

Or do we explicitely want to support both the legacy_boot and the boot flag,
because we know that some systems are buggy and one needs the first
and other need the latter flag?



Sorry, or did I lost track somewhere?
(I was some days offline lately)


Holger

[Diederik de Haas]

Hi,

Post by Holger Wansing

Post by Diederik de Haas
```
Model: MMC NCard (sd/mmc)
Disk /dev/mmcblk1: 61.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 16.8MB 16.7MB primary
2 16.8MB 256MB 239MB fat16 primary boot, esp
3 256MB 61.9GB 61.6GB ext4 primary
```
```
Model: MMC A3A562 (sd/mmc)
Disk /dev/mmcblk1: 124GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 32.8kB 3670kB 3637kB loader1_part
2 8389kB 12.6MB 4194kB loader2_part
3 12.6MB 16.8MB 4194kB trust_part
4 16.8MB 134MB 117MB efi_part
5 134MB 124GB 124GB ext4 root_part legacy_boot
```
And I just changed the 'legacy_boot' flag to 'boot' on the SDcard with
which I did my test, which then (apparently) also triggers the 'esp'
flag and it booted with that too.
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

here you have a device|installation, which has the legacy_boot flag
installed, but it did not work|boot despite of this. And you changed that
legacy_boot flag into a boot flag, and that made the problem go away?
(so the device boots fine).

There are 3 devices under discussion:
1) `cs21`: Rock64 running Stable and apparently using EFI and it looks
like I installed that with d-i, but then manually made sure the first
16 MiB remained free
2) `quartz64a`: Quartz64 Model-A, running Testing/Sid and I manually
created that system (no d-i), including all its partitions
3) `debian-di-rock64`: This is *another* Rock64 device (I have 4 in
total) with which I performed the test with a very recent d-i.
On this system everything seemed to go well, but it still didn't boot
into it. So that's when I started to experiment with the
'legacy_boot' flag as I was pretty sure that was the problem ... and
it was. I then went comparing with system '1' and '2' and that's when
I noticed 'legacy_boot' vs 'boot'/'esp'.
But without some 'boot' flag set, the device would NOT boot.

Post by Holger Wansing
So, why should we then do some work to implement the setting of the
legacy_boot flag on arm64?

I do not know what the details are between 'boot' and 'legacy_boot'.
So I can't tell if setting 'boot' would be sufficient (in all cases).

Post by Holger Wansing

Post by Diederik de Haas

Post by Pascal Hambourg

Post by Diederik de Haas
So it looks like it just needs some indication (flag) of what the boot
device is, but it doesn't (seem to) matter which one.

Weird. Does it have to be on the boot/root partition or can it be on any
other partition ?

It appears to not matter if it's 'boot' or 'legacy_boot', but it (ofc?)
needs to be set on the partition with the kernel as that needs to be
started for the rest of the boot process.
My guess is that on the tested system it tried to do something with TFTP
as it couldn't find a kernel as the '[legacy_]boot' flag was not set,
thus couldn't find a kernel thus tried another option (tftp).

Here you also imply, that both boot or legacy_boot flag do the job.

That's what my (semi) random tests seemed to show.
But as said above, I don't know for sure.

Post by Holger Wansing
So, can I understand this, as there is no need to set the legacy_boot flag,
but just adjust the situations, where the 'boot' flag is being set?
Or do we explicitely want to support both the legacy_boot and the boot flag,
because we know that some systems are buggy and one needs the first
and other need the latter flag?

I DON'T KNOW.
I did several tests on one Rock64 that I have and compared that with
some other devices that I have in the hope that it would be useful.

But apparently it only pissed you off.

[Pascal Hambourg]

Post by Diederik de Haas

Post by Holger Wansing
here you have a device|installation, which has the legacy_boot flag
installed, but it did not work|boot despite of this. And you changed that
legacy_boot flag into a boot flag, and that made the problem go away?
(so the device boots fine).

IIUC, it booted successfully after setting the legacy_boot flag on the
/boot partition. And it still booted successfully after setting the
boot/esp flag and removing the legacy_boot flag.

Post by Diederik de Haas

Post by Holger Wansing
So, why should we then do some work to implement the setting of the
legacy_boot flag on arm64?

I do not know what the details are between 'boot' and 'legacy_boot'.
So I can't tell if setting 'boot' would be sufficient (in all cases).

On GPT, the 'boot' flag is the same as 'esp'.
Partman sets the boot/esp flag only with method=efi, which implies that
the partition is used as ESP (FAT, mounted on /boot/efi) and cannot be
used as the root or /boot partition.

Post by Diederik de Haas

Post by Holger Wansing
So, can I understand this, as there is no need to set the legacy_boot flag,
but just adjust the situations, where the 'boot' flag is being set?
Or do we explicitely want to support both the legacy_boot and the boot flag,

I think so, because both are needed in different use cases.
- EFI boot -> 'esp' flag on the EFI partition
- non-EFI boot -> 'legacy_boot' flag on the root/boot partition

[Holger Wansing]

Hi,

Post by Pascal Hambourg

Post by Holger Wansing
Or do we explicitely want to support both the legacy_boot and the boot flag,

I think so, because both are needed in different use cases.
- EFI boot -> 'esp' flag on the EFI partition
- non-EFI boot -> 'legacy_boot' flag on the root/boot partition

Ok, I will then file a new bugreport for this (as already suggested).

I there some change needed for the esp flag part on arm64?
(the legacy_boot flag part is already in your legacy_boot_specifier
branch.)


Holger

[Pascal Hambourg]

Post by Holger Wansing
I there some change needed for the esp flag part on arm64?

No, the 'efi' method already sets thz 'esp' flag in arm64-efi recipes.

[Vagrant Cascadian]

Post by Pascal Hambourg

Post by Holger Wansing
here you have a device|installation, which has the legacy_boot flag
installed, but it did not work|boot despite of this. And you changed that
legacy_boot flag into a boot flag, and that made the problem go away?
(so the device boots fine).

IIUC, it booted successfully after setting the legacy_boot flag on the
/boot partition. And it still booted successfully after setting the
boot/esp flag and removing the legacy_boot flag.

I think there are several things that u-boot treats as bootable ...

https://source.denx.de/u-boot/u-boot/-/blob/3fbc657669591ca893613f14d42e07069b7d56cd/doc/develop/distro.rst?plain=1#L38-62

Distros simply need to install the boot configuration files (see next
section) in an ext2/3/4 or FAT partition, mark the partition bootable (via
the MBR bootable flag, or GPT legacy_bios_bootable attribute), and U-Boot (or
any other bootloader) will find those boot files and execute them. This is
conceptually identical to creating a grub2 configuration file on a desktop
PC.

Note that in the absence of any partition that is explicitly marked bootable,
U-Boot falls back to searching the first valid partition of a disk for boot
configuration files. Other bootloaders are recommended to do the same, since
I believe that partition table bootable flags aren't so commonly used outside
the realm of x86 PCs.

So this is both relevent if using u-boot to using various boot methods,
such as extlinux.conf style configuration produced by u-boot-menu or
boot.scr produced by flash-kernel... but also it might be relevent if
you are using u-boot as your EFI implementation to load grub-efi.

I am not entirely sure the above documentation is exhaustive; I seem to
recall u-boot also treating the ESP/EFI partition as bootable, but that
may be because various other flags get set at the same time... though I
have used ESP/EFI partitions as /boot with u-boot before without any EFI
implementation.

A very cursory search of u-boot source code did not reveal any
additional ways u-boot might treat a partition as "bootable", though the
documentation I quoted from is kind of the old style "distro" boot
vs. the newer "bootstd" which I am less familiar with...


Mixed feelings about leaving the first 16MB unpartitioned vs. specifying
some partition for it that has some special flag set. Creating a
partition and marking it with MBR boot or legacy_bios_boot might trip up
u-boot if it does not contain the extlinux/boot.scr/efi stuff it is
expecting.


live well,
vagrant

[Pascal Hambourg]

Post by Vagrant Cascadian
https://source.denx.de/u-boot/u-boot/-/blob/3fbc657669591ca893613f14d42e07069b7d56cd/doc/develop/distro.rst?plain=1#L38-62
Distros simply need to install the boot configuration files (see next
section) in an ext2/3/4 or FAT partition, mark the partition bootable (via
the MBR bootable flag, or GPT legacy_bios_bootable attribute), and U-Boot (or
any other bootloader) will find those boot files and execute them. This is
conceptually identical to creating a grub2 configuration file on a desktop
PC.
Note that in the absence of any partition that is explicitly marked bootable,
U-Boot falls back to searching the first valid partition of a disk for boot
configuration files. Other bootloaders are recommended to do the same, since
I believe that partition table bootable flags aren't so commonly used outside
the realm of x86 PCs.

I understand from the above that if the first partition is the 16MiB
reserved partition then the root/boot partition should have the "legacy
BIOS bootable" (parted 'legacy_boot' flag) on GPT or the 'boot' flag on
DOS/MBR.

Post by Vagrant Cascadian
Mixed feelings about leaving the first 16MB unpartitioned vs. specifying
some partition for it that has some special flag set. Creating a
partition and marking it with MBR boot or legacy_bios_boot might trip up
u-boot if it does not contain the extlinux/boot.scr/efi stuff it is
expecting.

Note that we are discussing the pros and cons of marking the reserved
partition with the GPT "BIOS boot partition" type (parted 'bios_grub'
flag), not the DOS/MBR "boot" flag nor GPT "legacy BIOS bootable"
attribute (parted 'legacy_boot' flag). The "BIOS boot partition" type is
designed to contain a "legacy" (non-EFI) boot loader such as GRUB-BIOS
core image, which matches the use case pretty well. Do you have any
information that it may interact with ARM boot process in any way ?

Leaving the first 16MB unpartitioned is not possible currently with
automatic partitioning. A reserved partition must be created then
deleted by the user or a preseeded command. One option could be to
enforce that a new partition starts beyond the first 16MiB, but that
does not sound trivial.

[Holger Wansing]

Hi,

Post by Diederik de Haas

Post by Holger Wansing
Or do we explicitely want to support both the legacy_boot and the boot flag,
because we know that some systems are buggy and one needs the first
and other need the latter flag?

I DON'T KNOW.
I did several tests on one Rock64 that I have and compared that with
some other devices that I have in the hope that it would be useful.
But apparently it only pissed you off.

Not really. Sorry, if you understood it like that.
I was just unable to follow and understand the facts to some degree,
that's all.


Holger

[Pascal Hambourg]

Post by Pascal Hambourg

I already had a hunch why that may be and that's when I examined the
SDcard with `lsblk` and later with `fdisk`. Went into `fdisk`'s expert
menu and enabled the boot flag on partition 2 (`/boot`).
Plugged the SDcard into my Rock64 again and booted up and then it
succeeded \o/
IOW: It was so, so close from working ... but it needs the 'boot' flag.

Do you mean the "legacy BIOS bootable flag" (legacy_boot in parted) ?
Unfortunately partman does not manage it. IIRC the "$bootable{ }" flag
in recipes is translated into parted "boot" flag, but on GPT the "boot"
flag means the same as "esp" (this is a huge mess) and partman clears
them if the partition method is not "efi". It should not be hard to
write a patch to translate the "$bootable{ }" flag into the
"legacy_boot" flag instead of the "boot" flag on GPT.

Patch translating '$bootable{ }' in recipes into the 'legacy_boot' flag
on GPT:
<https://salsa.debian.org/pham/partman-auto/-/commit/f196cb9981fb0e27f43bb9a2b14a4550e53027f7>
("Add support for legacy_boot flag on GPT")

If this is merged, I should probably add support for 'legacy_boot' to
partman-base (for display) and partman-partitioning (for manual toggle) too.

Q: Is the 'legacy_boot' flag also needed on arm64/efi platforms ?

[Marcin Juszkiewicz]

For that first 16MB partition it would be nice to use GPT type EF02
(BIOS boot partition) so partitioning tools will see that it is
partition for bootloaders.

I created test 512MB disk and created some partitions by hand:

gdisk:
Number Start (sector) End (sector) Size Code Name
1 2048 34815 16.0 MiB EF02 BIOS boot partition
2 34816 165887 64.0 MiB EF00 EFI system partition
3 165888 690175 256.0 MiB 8300 Linux filesystem
4 690176 915455 110.0 MiB 8E00 Linux LVM

parted:
Number Start End Size File system Name Flags
1 1049kB 17.8MB 16.8MB BIOS boot partition bios_grub
2 17.8MB 84.9MB 67.1MB EFI system partition boot, esp
3 84.9MB 353MB 268MB Linux filesystem
4 353MB 469MB 115MB Linux LVM lvm

Parted is a tool from MBR era and it's support GPT is poor that's why
it has 'Flags' column with 'what parted thinks it is'.


This way part1/ef02 is not formatted, part2/ef00 is vfat (never
mind which variant) and rest of partitions are formatted to whatever
filesystem OS wants.

Separate /boot/ is handy if user wants to have / on LVM (crypted or
not) as this allows 'OS loader' (grub-efi in Debian) to load kernel,
initramfs (and dtb if needed) from /boot and start the OS.


And for our sanity we pretend that whatever firmware system is using
(edk2, u-boot, barebox, etc.) it knows how to load 'OS loader' EFI
binary from EF00 partition. For Debian it means grub2-efi.

If hardware does not handle EFI variables (so we can not store
BootOrder vars) then /efi/boot/bootaa64.efi should be written as this
is default name for OS loader in EFI/aarch64 world.

[Pascal Hambourg]

Hi Marcin,

Post by Marcin Juszkiewicz
For that first 16MB partition it would be nice to use GPT type EF02
(BIOS boot partition) so partitioning tools will see that it is
partition for bootloaders.

Partitioning tools do not really care about what a partition is for.
AFAIK the "BIOS boot" type is used only by GRUB for BIOS on x86.
Does U-Boot or any other arm64 boot loader use this type ?

I know this is very specific, but using the "BIOS boot" type for the
16MiB reserved partition would get in the way of a multi-boot x86+arm64
installation because GRUB for BIOS is written at the beginning (30-100
kB) of the BIOS boot partition. Is this a realistic use case ?

Post by Marcin Juszkiewicz
Separate /boot/ is handy if user wants to have / on LVM (crypted or
not) as this allows 'OS loader' (grub-efi in Debian) to load kernel,
initramfs (and dtb if needed) from /boot and start the OS.

GRUB can read LVM and does not require a separate /boot. Anyway
automatic partitioning built-in recipes create a separate /boot when
partitioning with LVM (encrypted or not).

Post by Marcin Juszkiewicz
If hardware does not handle EFI variables (so we can not store
BootOrder vars) then /efi/boot/bootaa64.efi should be written as this
is default name for OS loader in EFI/aarch64 world.

If EFI variables are read-only or not available, grub-installer prompts
the user to install GRUB in the removable media path and/or update EFI
boot variables.

[Pascal Hambourg]

Post by Marcin Juszkiewicz
For that first 16MB partition it would be nice to use GPT type
EF02 (BIOS boot partition) so partitioning tools will see that it
is partition for bootloaders.

(...)

AFAIK the "BIOS boot" type is used only by GRUB for BIOS on x86. Does
U-Boot or any other arm64 boot loader use this type ?

It is type which shows the purpose of partition. Probably none of OS
loaders (like grub etc) use it on arm64. In Fedora we have one big EFI
binary for Grub, Debian uses modular approach with modules kept in the
/boot/grub/ dir.

Debian also has signed monolithic GRUB images for UEFI secure boot.

I know this is very specific, but using the "BIOS boot" type for
the 16MiB reserved partition would get in the way of a multi-boot
x86+arm64 installation because GRUB for BIOS is written at the
beginning (30-100 kB) of the BIOS boot partition. Is this a
realistic use case ?

I would love to meet someone who uses storage that way. Using one disk
for 10+ years old PC (BIOS/CSM) and modern AArch64 system. It is
theoretically possible but in practice it is cheaper to buy some disk.

Except if you want all systems to share common data on the same portable
medium. I do not know if the notion of a portable installation makes
sense in the arm64 world. But I doubt that anyone would use automatic
partitioning to create such setup.

To be honest, I considered using the "BIOS boot" type when I added the
16MB reserved partition to arm64 recipes. It has the additional
advantage of allowing automatic partitioning in free space to reuse an
existing partition of the same type instead of uselessly creating a new
one, for multi-boot. But again I do not know if multi-boot is a common
use case or even possible on arm64.

The decision is up to d-i maintainers.

[Holger Wansing]

Hi,

Post by Pascal Hambourg

Post by Marcin Juszkiewicz
For that first 16MB partition it would be nice to use GPT type
EF02 (BIOS boot partition) so partitioning tools will see that it
is partition for bootloaders.

(...)

AFAIK the "BIOS boot" type is used only by GRUB for BIOS on x86. Does
U-Boot or any other arm64 boot loader use this type ?

It is type which shows the purpose of partition. Probably none of OS
loaders (like grub etc) use it on arm64. In Fedora we have one big EFI
binary for Grub, Debian uses modular approach with modules kept in the
/boot/grub/ dir.

Debian also has signed monolithic GRUB images for UEFI secure boot.

I know this is very specific, but using the "BIOS boot" type for
the 16MiB reserved partition would get in the way of a multi-boot
x86+arm64 installation because GRUB for BIOS is written at the
beginning (30-100 kB) of the BIOS boot partition. Is this a
realistic use case ?

I would love to meet someone who uses storage that way. Using one disk
for 10+ years old PC (BIOS/CSM) and modern AArch64 system. It is
theoretically possible but in practice it is cheaper to buy some disk.

Except if you want all systems to share common data on the same portable
medium. I do not know if the notion of a portable installation makes
sense in the arm64 world. But I doubt that anyone would use automatic
partitioning to create such setup.
To be honest, I considered using the "BIOS boot" type when I added the
16MB reserved partition to arm64 recipes. It has the additional
advantage of allowing automatic partitioning in free space to reuse an
existing partition of the same type instead of uselessly creating a new
one, for multi-boot. But again I do not know if multi-boot is a common
use case or even possible on arm64.
The decision is up to d-i maintainers.

I see the benefit of using the "BIOS boot" type for this reserved partition,
more easier allowing people to understand, what this partition is used for.

But on the other hand, since we don't know how dual-boot on this arch
is working (or will work at some time in the future), it may be required
to have separate partitions for boot loader for both of those dual-booted
OS'es. Therefore, I would be in favour of not creating a partition scheme,
which easily allows to mix them up by some semi-automatism (like
auto-partitioning, when installing a new Debian).

IOW /me voting for keeping this partition as it is now (FAT).


Holger

[Pascal Hambourg]

Post by Holger Wansing

Post by Pascal Hambourg
To be honest, I considered using the "BIOS boot" type when I added the
16MB reserved partition to arm64 recipes. It has the additional
advantage of allowing automatic partitioning in free space to reuse an
existing partition of the same type instead of uselessly creating a new
one, for multi-boot. But again I do not know if multi-boot is a common
use case or even possible on arm64.
The decision is up to d-i maintainers.

I see the benefit of using the "BIOS boot" type for this reserved partition,
more easier allowing people to understand, what this partition is used for.
But on the other hand, since we don't know how dual-boot on this arch
is working (or will work at some time in the future), it may be required
to have separate partitions for boot loader for both of those dual-booted
OS'es. Therefore, I would be in favour of not creating a partition scheme,
which easily allows to mix them up by some semi-automatism (like
auto-partitioning, when installing a new Debian).

The "BIOS boot" type identifier can be set with the "biosgrub" method
without the "$reusemethod{ }" specifier, so that an installation with
automatic partitioning in free space will create a reserved partition
regardless of whether one already exists or not. However IIUC, the
purpose of the reserved partition is to prevent any other partition from
using the first 16MiB of the storage device, because this is where the
boot loader is installed and the platform firmware expects it to be. So
another reserved partition in any other place does not make any sense.

Post by Holger Wansing
IOW /me voting for keeping this partition as it is now (FAT).

It is not FAT, it is unformatted with the default type identifier
("Linux filesystem").

[Holger Wansing]

Hi,

IOW: It was so, so close from working ... but it needs the 'boot' flag.

This should now be fixed in the daily images, some it's time for another
test :-)


Didi, could you take some time for this?


Thanks
Holger

[Diederik de Haas]

Hi,

Post by Holger Wansing

IOW: It was so, so close from working ... but it needs the 'boot' flag.

This should now be fixed in the daily images, some it's time for another
test :-)

I build an installer from daily-images dd 2024-11-08 and put that on
SDcard and booted my Rock64. At the end of the installation I rebooted
... (the suspense is killing me) ... and it came up \o/

So it's working :-) Thanks!

Cheers,
Diederik