There are some packages that don't build any more in the docker build image, some of which I've already fixed, but got stuck at vyos-kernel. Looking at the repo, the repo is archived and there is a newer repo that replaces it, vyos-build-kernel. But simply replacing it isn't enough as the build process is different, and currently works only in CI as the Jenkinsfile has the up-to-date steps, which aren't present in vyos-build/scripts/build-packages.
Both build processes should work equally, so ideally the build process would be done by scripts which could be kept in the repo so there is a single place to make changes instead of 2, the CI and vyos-build.
I have some fixes for vyos-build already done which I'll submit as a PR but I'll leave the kernel package to someone else to fix.
Description
Details
- Difficulty level
- Unknown (require assessment)
- Version
- -
- Why the issue appeared?
- Will be filled on close
- Is it a breaking change?
- Unspecified (possibly destroys the router)
- Issue type
- Bug (incorrect behavior)
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved N/A | None | T2085 Building some packages with vyos-build no longer works for Equuleus/current | |||
| Resolved | jjakob | T2084 conntrack-tools package build error for current/equuleus |
Event Timeline
I'd like to know what the long term plan is here. How are "small time" developer supposed to be able to build the kernel related packages? I was in the middle of testing some change to wireguard, but now everything is on hold because I can't build the current kernel anymore without a significant effort. I'm open to help but I would need to know what the plan is for builds without Jenkins.
I'm not in the VyOS core team so I'm not able to make direct decisions on the resolution, but as I see it, there are several possible ways to approach this.
The core of the problem is that a move was made away from vyos-build, which used to be the way packages were build for the upstream repo, to per-package build chains in Jenkins (ci.vyos.net). This is not a problem for packages that have no custom build steps but just dpkg-buildpackage as the steps are the same in both cases. For packages that do have custom build steps (that either build modified upstream packages from git e.g. kernel, apply patches, use git submodules, etc.) this fragmented the places in which these custom steps were defined, out of the vyos-build build-packages script (where they were previously) into Jenkinsfiles. But the steps weren't kept in sync with vyos-build any more, so we ended up with the situation we're in today, where vyos-build either doesn't work at all or builds very outdated packages. This means the contributors have no way of easily building all packages or testing modifications and contributions to these packages themselves.
IMO there are several ways to fix this:
- move custom build steps out of Jenkinsfiles back into scripts
- For packages where we keep our own repos or forks, keep separate build scripts in a shell or python script that are compatible with both Jenkins and vyos-build (docker) repos. Call that script from the Jenkinsfile and build-packages scripts.
- for packages where we don't keep our own forks, either put the scripts into a dedicated folder in vyos-build, or make simple build-script-only repos (per each package or unified) that will be used by Jenkins and build-packages the same as above.
- move custom build steps back into build-packages and call it from Jenkins (not the best solution IMO, keep the custom scripts out of a packaging script)
- make a new build system for outside contributors that runs Jenkinsfiles directly (either jenkinsfile-runner or something similar). Not a good solution IMO as jenkinsfile-runner is Java and is larger and slower than the current python scripts, and build-packages would need to be largely rewritten
I'd prefer not keeping our own forks of whole source packages where possible but just do all required steps in the build script (apply patches, merge newer upstream sources into debian packages etc.). This cuts down on maintenance as we don't need to manually update all the forks for every upstream update. Update the build process when any custom patches get outdated or are merged into upstream and aren't needed any more, or we need to bump fixed tags or commit id's we build from.
Additionally it would be nice to automate installing build dependencies in the docker image. Something like a script ran from the Dockerfile that parses all build-deps of the source deb repos (from debian/control files) and installs them. This would cut out needing to update the Dockerfile each time a build-dep changes.
I once wrote a script as PoC to list build dependencies:
https://github.com/vyos/vyos-build/blob/current/scripts/list-build-dependencies
We probably should send this PoC script (vyos-build/scripts/build-packages.) to the graveyard as it causes more trouble then good I feel.
vyos-build-kernel now has dedicated build shell scripts