The folder `json_info_files` contains multiple JSON files which describe
created firmware images. The folder is not removed between builds as the
ImageBuilder does not use `image.mk`.
Not removing the JSON files result in a merged `profiles.json` file
containing entries for outdated or non-existing images.
This commit adds the `json_info_files/` cleanup step to the ImageBuilder
Makefile.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The patch 4a1a58a3 build, imagebuilder: Do not require libncurses-dev
was supposed to remove libncurses as a requirement for the ImageBuilder.
However as the IB=1 is only exported during building, not for checking
requirements, it did never actually work.
This commit export IB=1 to the requirement check.
Signed-off-by: Paul Spooren <mail@aparcar.org>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
For x86/64 (maybe more) target the SUPPORTED_DEVICES variable is empty
which causes the `&&` junction to fail, producing a non zero exit code.
Tested-by: Paul Spooren <mail@aparcar.org>
Fixed-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Adds a new variable DISABLED_SERVICES to ImageBuilder Makefile, which
defines a list of services (installed as /etc/init.d/*) to be disabled
during the build of a custom image (normally all are enabled).
It comes handy when a particular service should not be run under normal
circumstances, but should be ready in the image for situations when it
might be needed.
Signed-off-by: Richard Musil <risa2000x@gmail.com>
This is useful in for the attendedsyupsgrade server (asu) to
distinguish between snapshot version. Currently asu can't tell devices
requesting a snapshot build if the same build is already installed.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Instead of showing a slightly more readable target like
"ar71xx (Generic)" print the more generic format "ar71xx/genric"
Signed-off-by: Paul Spooren <mail@aparcar.org>
Having image metadata (and signature) appended is a condition for
semi-automated sysupgrade, hence IB needs to be able to tell which
images will end up with metadata.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
This really simplifies debugging, if a package is not found or a feed is
not reachable, a proper stderr is printed. Currently it would only say
`_call_manifest` failed.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Tested with 18.06.0-rc2/ar71xx/generic/tl-wdr4300-v1, image & list
This PR is based on the work of @fewckert[1] with slight improvements.
Add function `manifest` to show the manifest of the produced image,
before actually building it. The manifest contains an orderd list of
package name and version.
This is usefull to check package dependencies but also determine a
unique and reproducible image name before building the package. The
sysupgrade server[2] builds images on request with individual package
selection. To distignish between created images which contain differnt
packages, the EXTRA_IMAGE_NAME is set to a shortend hash of the
manifest's content. So far the image was renamed afterwards as the
manifests content was unknown, however this corrupts the signed
sha256sums. This patch allows a clean solution as to dtermine the
manifest in advance and set the EXTRA_IMAGE_NAME accordingly.
[1]: https://github.com/lede-project/source/pull/1591
[2]: https://github.com/aparcar/attendedsysupgrade-server
Signed-off-by: Paul Spooren <mail@aparcar.org>
In addition to removing redundant code, this fixes various issues in
IB-generated images that have been fixed in prepare_rootfs before,
including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts
from FILES.
We also reuse the opkg macro and remove --force-... flags that have been
removed from rootfs.mk as well.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
No longer rewrite opkg list output in package_list function, remove
the awk call in the pipe (which was intended for a single specific
use-case).
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
commit 19ac879954 (imagebuilder: add package_list function) introduced
a new function 'package_list' to the imagebuilder Makefile.
Unfortunately the package list was poluted by stdout noise of the
Makefile itself as well as opkg. Redirect those outputs to stderr to
make sure that the package_list returned doesn't contain progress
info output but really only packages.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The imagebuilder can now list all available packages by using make
package_list. This is usefull for scripts to retrieve a list of all
packages with versions (and size)
Signed-off-by: Paul Spooren <paul@spooren.de>
[daniel@makrotopia.org: fixed commit message]
Use silent make invocations for sub-makes like build_image or checksum to
avoid bloating the IB output with non-status info.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The name "Plat'Home OpenBlocks AX3" causes the imagebuilders "make info"
command to fail with:
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `echo; [...]'
Makefile:99: recipe for target '_call_info' failed
Properly escape single quotes to avoid breaking the echo commands.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Add a new "checksum" make target which generates an sha256sums file over the
image files produced in bin/targets/ and automatically call it during make
world after the package index generation.
The advantage of this new target is that it is guaranteed to run after the
images, the SDK and the ImageBuilder archives have been generated to ensure
that they all end up in the checksum file. Fixes FS#51.
Uses sed to postprocess the OpenSSL digest output into an sha256sum command
compatible format.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
When imagebuild sorts package lists it breaks opkg's ability to realize
that a providers for a Provides has already been installed, when the sort
results in the provider being later in the list of packages that a package
which depends on a Provides (and hence the provider is not yet installed
for opkg to realize the provider was available doesn't not handle the case
of a package that is to be installed satisfying a dependency, only one that
is already installed (or which it schedules to be installed, which in the
absence of an installed provider is whichever provider happens to be the
default)
Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>
For final output image names allow user to add an
extra string (which is sanitized). This is particularly
useful with ImageBuilder where you may generate multiple
images from the same base and for the same board,
with different package selections and additional files
(via FILES=).
Signed-off-by: Daniel Dickinson <openwrt@daniel.thecshore.com>
SVN-Revision: 48083
Force opkg to store the downloaded repository indizes into the cache
directory as well, this way the IB can be used in an offline setting
once all required files have been cached.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 46912
To improve reproducibility, prevent the inclusion of timestamps
in the gzip header.
Signed-off-by: Reiner Herrmann <reiner@reiner-h.de>
SVN-Revision: 46361
Change the IB packaging to only embed libc, kernel and kmod packages by default
and generate repositories.conf to refer to the remote package repositories.
Introduce a new config option CONFIG_IB_STANDALONE which restores the old
behaviour of building self contained IB archives.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 45772
This changeset implements a new menuconfig option to generate separate
repositories for each enabled package feed instead of one monolithic one.
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
SVN-Revision: 42002
V=99 and V=1 are now deprecated in favor of a new verbosity class system,
though the old flags are still supported.
You can set the V variable on the command line (or OPENWRT_VERBOSE in the
environment) to one or more of the following characters:
- s: stdout+stderr (equal to the old V=99)
- c: commands (for build systems that suppress commands by default, e.g. kbuild)
- w: warnings/errors only (equal to the old V=1)
SVN-Revision: 31484
The "Generating index for package" created by ipkg-make-index only
clutter stderr and serve to push previous warnings/errors off-screen.
Instead, send to stdout.
Note: This would send incorrect invocation messages to stdout as well.
If this is a concern, perhaps we could send them to FD 3, then send 3 to
2...
Signed-off-by: Kevin Locke <klocke@digitalenginesoftware.com>
SVN-Revision: 25678
Paul Spooren
Richard Musil
Daniel Golle
Matthias Schiffer
Paul Spooren
Felix Fietkau
Jo-Philipp Wich
Daniel Dickinson
Felix Fietkau
Jo-Philipp Wich
John Crispin
Gabor Juhos