Page MenuHomeVyOS Platform

Further shrink VyOS imagesize
Closed, ResolvedPublicFEATURE REQUEST

Description

Looking through the documentation at:

https://docs.vyos.io/en/latest/installation/install.html?highlight=vmlinuz#step-2-tftp

It turns out that the vmlinuz (currently: vmlinuz-6.1.53-amd64-vyos) and initrd.img (currently: initrd.img-6.1.53-amd64-vyos) in the /live path of the iso is the same as within the filesystems.squshfs file.

This made me wonder if it shouldnt be possible to do this during build (example for VyOS 1.5-rolling-202309170024)?

  1. Copy config-6.1.53-amd64-vyos and System.map-6.1.53-amd64-vyos from chroot and place them also in the /live path of the iso (or some other way during creation of the iso have them included in the /live path).
  1. Delete /boot directory from within filesystem.squashfs (for example through rootfs/excludes-file) except for the symlinks regarding /boot/vmlinuz and /boot/initrd.img (they are used by vyatta-grub-setup to create grub.cfg during install).
  1. Adjust the install scripts so they will copy /live/vmlinuz-6.1.53-amd64-vyos, /live/initrd.img-6.1.53-amd64-vyos, /live/System.map-6.1.53-amd64-vyos and /live/config-6.1.53-amd64-vyos to the persistence storage on the drive (/usr/lib/live/mount/persistence/boot/1.5-rolling-202309170024 ?).

The above would in this example shrink the filesystem.squashfs (and the iso who includes the filesystem.squashfs) by approx:

vyos@vyos:~$ ls -la /boot
total 383440
drwxr-xr-x 5 root root      4096 Sep 17 20:43 .
drwxr-xr-x 1 root root      4096 Sep 17 22:07 ..
-r--r--r-- 1 root root 352301056 Sep 17 03:48 1.5-rolling-202309170024.squashfs
-rw-r--r-- 1 root root    155147 Sep 14 08:07 config-6.1.53-amd64-vyos
drwxr-xr-x 5 root root      4096 Sep 17 22:08 grub
lrwxrwxrwx 1 root root        28 Sep 17 03:48 initrd.img -> initrd.img-6.1.53-amd64-vyos
-rw-r--r-- 1 root root  30434498 Sep 17 03:48 initrd.img-6.1.53-amd64-vyos
drwxr-xr-x 9 root root      4096 Sep 17 22:07 rw
-rw-r--r-- 1 root root   3439751 Sep 14 08:07 System.map-6.1.53-amd64-vyos
lrwxrwxrwx 1 root root        25 Sep 17 03:48 vmlinuz -> vmlinuz-6.1.53-amd64-vyos
-rw-r--r-- 1 root root   6282176 Sep 14 08:07 vmlinuz-6.1.53-amd64-vyos
drwxr-xr-x 3 root root      4096 Sep 17 22:07 work

30434498+6282176 = 36716674 bytes (approx 35MB).

Due to compression the actual saving for the iso file will be less than 35MB.

But still going from todays 390MB to give or take 355MB.

A saving by approx (up to) 9%.

Details

Version
-
Is it a breaking change?
Unspecified (possibly destroys the router)

Event Timeline

Testing point 2 above with VyOS 1.5-rolling-202309170024.

Adding this to rootfs/excludes:

# T5593: Linux kernel exists in /live-path of the ISO and is copied during install.
boot/!(initrd.img|vmlinuz)

Result (extracted the resulting filesystem.squashfs):

# ls -la ./squashfs-root/boot/
total 8
drwxr-xr-x  2 root root 4096 sep 17 03:48 .
drwxr-xr-x 18 root root 4096 sep 17 03:48 ..
lrwxrwxrwx  1 root root   28 sep 17 03:48 initrd.img -> initrd.img-6.1.53-amd64-vyos
lrwxrwxrwx  1 root root   25 sep 17 03:48 vmlinuz -> vmlinuz-6.1.53-amd64-vyos

Filesystem.squash size:

Before: 352301056 bytes (336MB)
After: 315002880 bytes (300.4MB)
Saved: 37298176 bytes (35.57MB)

Total save of 33703278 bytes (32.15MB) for the ISO when subtracting the above with size of config file (155147 bytes) and system-map file (3439751 bytes).

That is the ISO will be approx 8.24% smaller (shrink from 390MB down to approx 358MB).

It looks like point 3 can be taken care of:

In these two files:

https://github.com/vyos/vyatta-cfg-system/blob/current/scripts/install/install-image-new#L74
https://github.com/vyos/vyatta-cfg-system/blob/current/scripts/install/install-image-existing#L148

Alter how the files are being copied into persistence /boot during install/upgrade.

That is change source from:

/usr/lib/live/mount/rootfs/filesystem.squashfs/boot/*

aka

${CD_SQUASH_ROOT}/boot

to

/usr/lib/live/mount/medium/live/*

aka

${CD_ROOT}/live/

And instead of copying all files from that directory then restrict it to just these:

config-*
initrd.img-*
System.map-*
vmlinuz-*

Proposed change for https://github.com/vyos/vyatta-cfg-system/blob/current/scripts/install/install-image-new#L74

# Copy the squashfs image and boot files
echo "Copying release files..."

# These are the defaults if installing from a specified ISO image file.
# In such cases, the ISO image has already been mounted by caller.
squash_img=${CD_ROOT}/live/filesystem.squashfs
boot_dir=${CD_ROOT}/live
boot_files=$(find ${boot_dir} -maxdepth 1 -type f \( -name "config-*" -o -name "initrd.img-*" -o -name "System.map-*" -o -name "vmlinuz-*" \) 2>/dev/null)
if [ ! -f "${squash_img}" ] || [ -z "${boot_files}" ]; then
  # Maybe installing from a live CD boot?
  squash_img=/lib/live/mount/medium/live/filesystem.squashfs
  boot_dir=/lib/live/mount/medium/live
  boot_files=$(find ${boot_dir} -maxdepth 1 -type f \( -name "config-*" -o -name "initrd.img-*" -o -name "System.map-*" -o -name "vmlinuz-*" \) 2>/dev/null)
  if [ ! -f "${squash_img}" ] || [ -z "${boot_files}" ]; then
    echo "Cannot find the squashfs image or boot files. Exiting..."
    exit 1
  fi
fi

target_squash=${WRITE_ROOT}/boot/$image_name/$version.squashfs
echo "Copying squash image..."
cp -p ${squash_img} ${target_squash} >&/dev/null
echo "Copying kernel and initrd images..."
cp -dp ${boot_files} ${WRITE_ROOT}/boot/$image_name/ >&/dev/null

Proposed change for https://github.com/vyos/vyatta-cfg-system/blob/current/scripts/install/install-image-existing#L148

# Copy the squashfs image and boot files
echo "Copying new release files..."

squash_img=${CD_ROOT}/live/filesystem.squashfs
boot_dir=${CD_ROOT}/live
boot_files=$(find ${boot_dir} -maxdepth 1 -type f \( -name "config-*" -o -name "initrd.img-*" -o -name "System.map-*" -o -name "vmlinuz-*" \) 2>/dev/null)
if [ ! -f "${squash_img}" ] || [ -z "${boot_files}" ]; then
  echo "Cannot find the squashfs image or boot files. Exiting..."
  exit 1
fi

target_squash=${REL_ROOT}/$NEWVER.squashfs
echo "Copying squash image..."
cp -p ${squash_img} ${target_squash} >&/dev/null
echo "Copying kernel and initrd images..."
cp -dp ${boot_files} ${REL_ROOT}/ >&/dev/null

So what remains is how to solve point 1.

That is how to copy these files from the chroot so they end up at the /live directory of the ISO:

config-*
initrd.img-*
System.map-*
vmlinuz-*

Point 1 might be solved by using a hooks/live-script for the binary part which is the part after the chroot have been created.

For example naming it: data/live-build-config/hooks/live/99-copy-linux-kernel.binary.

The content will be a script to copy the following files from the chroot-directory (build/chroot/boot) into the live-directory (build/binary/live):

config-*
initrd.img
initrd.img-*
System.map-*
vmlinuz
vmlinuz-*

Proposed new file data/live-build-config/hooks/live/99-copy-linux-kernel.binary:

#!/bin/sh

#
# Copy Linux-kernel files from chroot/boot to binary/live.
#

# Set variables.
CHROOTDIR="../chroot/boot/"
LIVEDIR="./live/"

# Perform stuff.
echo "Copying kernel-files..."

cp -a ${CHROOTDIR}/config-* ${LIVEDIR}
cp -a ${CHROOTDIR}/initrd.img ${LIVEDIR}
cp -a ${CHROOTDIR}/initrd.img-* ${LIVEDIR}
cp -a ${CHROOTDIR}/System.map-* ${LIVEDIR}
cp -a ${CHROOTDIR}/vmlinuz ${LIVEDIR}
cp -a ${CHROOTDIR}/vmlinuz-* ${LIVEDIR}

The order of execution can be seen in:

https://salsa.debian.org/live-team/live-build/-/blob/master/scripts/build/build

https://salsa.debian.org/live-team/live-build/-/blob/master/scripts/build/chroot

https://salsa.debian.org/live-team/live-build/-/blob/master/scripts/build/binary

Reference:

https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-contents.en.html#hooks

@Apachez note that those legacy image install scripts will be removed following
https://vyos.dev/T4516
Work on completing that is active this week and should be finished soon. You may want to hold off on this investigation until then.

... of course, feel free to experiment; I have not yet considered the proposed idea.

If build and smoketests are successful a commit will arrive later today.

The three changes in combination will result in that the filesystem.squashfs itself only have symlinks in its /boot-directory while all the Linux kernel-files will exist in the /live-directory of the ISO (which the vyatta-cfg scripts will copy to the persistence /boot during install/upgrade).

A bonus is that the static files of initrd.img and vmlinuz will be replaced by symlinks so there (again if successful) should be a significant reducation in imagesize (aka ISO).

Build was successful and smoketests are currently in progress.

The ISO booted and installed (running smoketests through time sudo make test).

Within the ISO the symlinks for initrd.img and vmlinuz are successful.

filesystem.squashfs from 1.4-rolling-202308310021:

Filesystem size 374605951 bytes (365826.12 Kbytes / 357.25 Mbytes)

filesystem.squashfs from 1.5-rolling-202309170024:

Filesystem size 352297562 bytes (344040.59 Kbytes / 335.98 Mbytes)

filesystem.squashfs from 1.5-rolling-20230927_custom (todays build with the changes from T5593):

Filesystem size 335508994 bytes (327645.50 Kbytes / 319.97 Mbytes)
This comment was removed by Apachez.

PR created for part 1/2 (vyatta-cfg-system): https://github.com/vyos/vyatta-cfg-system/pull/209

PR created for part 2/2 (vyos-build): https://github.com/vyos/vyos-build/pull/427

Note! Both commits above must be merged at the same time for this to properly work.

Hopefully this can be resolved for VyOS 2.0 in the future...

Viacheslav triaged this task as Wishlist priority.Jan 20 2024, 1:25 PM
dmbaturin subscribed.

We've all done quite a bit of shrinking since the point when this task was created.

I suppose any further tasks should have more specific names. :)

I dont think this is resolved.

Once VyOS changes underlaying storage to LVM the feature request of this task can be implemented at the same time since changing to LVM will make a reinstall and wiping of the storage mandatory and by that the method of remove duplicated kernel binaries can be done for real.

syncer subscribed.

Closing this. Further shrinking is not priority now
we can comeback to this later if there will be proper business case