Commit Graph

119 Commits (1634461bd208f4bd108ea5c3d3f1cf9eb56d4a7e)

Author SHA1 Message Date
Eneas U de Queiroz 11192484fa build: reduce number of files passed to ipk-remove
Instead of using xargs to pass a huge number of files to
script/ipkg-remove, which will usually pick only one, use a more
restrictive wildcard so that, currently, at the most 325 files are
examined, instead of up to over 2,300.  The 325-file package is python,
which is picking up python3* ipks.  It is about to be removed.
Runner-up is ddns-scripts with 7 files.

This makes a second run of make package/luci/compile go from
real 16.40s; user 17.42s; sys 2.73s
to
real 10.71s; user 9.51s; sys 1.27s

There is a caveat though: if one were to remove the ABI_VERSION of a
package that ends in a digit [0-9], then the old package ipk will not be
removed from the bin directory by make package/abc2/clean.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
4 years ago
Eneas U de Queiroz 78d1f3ac61 build: call ipkg-remove using xargs if #args>=512
The wildcard call to clean up luci package (luci*) can pick up over
2,300 files when the full tree is built. Running make package/luci/clean
or a second run of make package/luci/compile would fail with an
'Argument list too long' error.

To avoid that, a maybe_use_xargs function was created that runs the
command straight as usual if the number of arguments is < 512, or saves
the list in a temporary file and feeds it to xargs otherwise.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
4 years ago
Eneas U de Queiroz 40acc13ca7 build: package-ipkg: avoid calling wildcard twice
Instead of calling $(wildcard) to check if the removal list is empty,
then calling it again to actually remove the files, define a function so
that the arguments are expanded only once when it gets called.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
4 years ago
Florian Eckert 4d86e2ce06 buildsystem: add CPE information to ipkg packages and manifest files
Common Platform Enumeration (CPE) is a structured naming scheme for
information technology systems, software, and packages.

This information already exists in some makefiles. In order for the
information to be processed further, it should also be added to the
manifest file and the control file of ipkg packages.

Signed-off-by: Florian Eckert <fe@dev.tdt.de>
4 years ago
Xu Wang 44304c1d67 base-files: fix build for /sbin/pkg_check
Setting CONFIG_IPK_FILES_CHECKSUMS=y causes sha256 checksum files to be
included with the packages to check for corruption. This commit fixes two
issues:
- /sbin/pkg_check was being removed incorrectly if IPK_FILES_CHECKSUMS=y
- checksums were being saved in the wrong file

Signed-off-by: Xu Wang <xwang1498@gmx.com>
4 years ago
Jonas Gorski 5ede3fcdf7 build: warn when packages have no associated install section
Declaring a package without the appropriate install section is an easy
mistake to make, especially when renaming packages. Since this is also
easy to detect, warn about it when it happens.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
No-objections-at-all-by: Jo-Philipp Wich <jo@mein.io>
5 years ago
Eneas U de Queiroz a41f474d17 build: add support to && in DEPENDS
Adds support to && operand in DEPENDS.  Also, fixes generation of ||
dependencies by scripts/package-metadata.pl.

The precedence order from higher to lower is && then ||.  Use of
parentheses to change the order is not supported. As before, they are
silently ignored.  Use them for readability only.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Signed-off-by: Christian Lamparter <chunkeey@gmail.com> [DMARC removal]
5 years ago
Jo-Philipp Wich 790bce92ad build: formatting fixes for per-provide ABI_VERSION suffixes
- Filter out potential duplicates with the package name
   (e.g. when renaming libfoo1 w/ ABI_VERSION:=1 to libfoo)
 - Use the GetABISuffix macro to properly separate the suffix
   with a dash in case the basename ends with a number

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years ago
Jo-Philipp Wich 60558790a2 build: extend ABI_VERSION suffixing to provides
When a library package specifies additional provides, e.g. libncurses
which provides libncursesw, we should also append the abi version
suffix to each provide, since there may be more than one package
providing the virtual library.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years ago
Michal Hrusecky 74450124f6 build: Optionally provide file checksums in package metadata
This may be useful if you don't entirely trust your flash and want to be able
to check for corruptions.

Signed-off-by: Michal Hrusecky <Michal@Hrusecky.net>
5 years ago
Jo-Philipp Wich e6bcf1e4ac build: add ABI_VERSION to binary package names
Add the ABI_VERSION source makefile variable to the binary package basename
and resolve source dependencies on packages with ABI_VERSION set to such
expanded names.

If for example a package specifies DEPENDS:=libopenssl while the OpenSSL
Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data
dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file
will be called "libopenssl1.0.0_<version>_<arch>.ipk".

The next time a library such as OpenSSL is updated to an incompatible
version, the ABI_VERSION shall be changed accordingly to prevent opkg from
simply upgrading to an incompatible library without considering the
dependencies of already installed packages.

Also introduce another "SourceName" control field which is required by
the newly introduced "scritps/ipkg-remove" to determine the proper related
.ipk files to delete upon buildroot package clean operations.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years ago
Felix Fietkau 7c306ae640 build: fix compile error when a package includes itself in PROVIDES
Signed-off-by: Felix Fietkau <nbd@nbd.name>
6 years ago
Michael Heimpold 2cb75cd8b9 build: allow defining license information per binary package
At the moment, license information can only be specified on a
"per source package" level while other metadata fields (e.g. maintainer)
can be given for each binary package. Apply the same logic for license
fields as well. This can be used e.g. in cases where a library is
distributed under some license while related tools are distributed
under a different one.

Signed-off-by: Michael Heimpold <mhei@heimpold.de>
7 years ago
Yousong Zhou 73bc636aba build: ipkg: new field Alternatives
It's a list of specs of the following form seprated by commas to
describe alternatives provided by this package

    <prio>:<path>:<altpath>

<path> will be a symbolic link to <altpath> of the highest <prio>

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
7 years ago
Felix Fietkau 1f22957247 build: clean up redundant touching of the package install info file
Instead of using a separate .clean stamp file, remove the install info
file on compile, then append the install package list afterwards

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Felix Fietkau 3cd7f43017 build: fix package build after deleting staging dir
Ensure that the <package>.provides files are always created

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Florian Fainelli 25e2cf392a package-ipkg: Do not fail build without base-files
If the base-files package is not selected, we will fail executing the
very first postinst script:

make[3]: Leaving directory `/local/users/fainelli/openwrt/trunk'
cp -fpR
/local/users/fainelli/openwrt/trunk/build_dir/target-arm_xscale_musl-1.1.15_eabi/root-orion
/local/users/fainelli/openwrt/trunk/build_dir/target-arm_xscale_musl-1.1.15_eabi/root.orig-orion
./usr/lib/opkg/info/busybox.postinst: line 3:
/local/users/fainelli/openwrt/trunk/build_dir/target-arm_xscale_musl-1.1.15_eabi/root-orion/lib/functions.sh:
No such file or directory
./usr/lib/opkg/info/busybox.postinst: line 4: default_postinst: command
not found
postinst script ./usr/lib/opkg/info/busybox.postinst has failed with
exit code 127
make[2]: *** [package/install] Error 1

Check for the existence of lib/functions.sh, and if it does not exist,
just bail out gracefully.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
7 years ago
Felix Fietkau f784b3e195 build: fix regression in handling ABI_VERSION
The code was updated without changing $$@ into the proper target
filename

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Felix Fietkau 5aa03e16a7 build: disable CONFIG_AUTOREMOVE for packages/tools where QUILT is used
Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Felix Fietkau 87d9e3462d build: stage unstripped packages inside PKG_BUILD_DIR
This makes the build slightly more efficient by avoiding the need to
re-stage packages on every full build run.

It is also necessary for the upcoming CONFIG_AUTOREMOVE feature

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Felix Fietkau 8aa93dafc8 build: write ABI_VERSION file when staging the package
The make target does not rewrite the file if the contents didn't
change. This causes make to always consider the target old.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Felix Fietkau 9228d1c066 build: introduce extra targets that contain only proper dependencies
This can be used to check if targets like prepare or compile are up to date

Signed-off-by: Felix Fietkau <nbd@nbd.name>
7 years ago
Rafał Miłecki b522292405 base-files: add support for overlaying rootfs content
This adds support for install-overlay define. When used in package it
allows installing files to a special directory that gets copied to the
root when installing it.
It allows overwriting files provided by other packages.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
7 years ago
Felix Fietkau 64c386c566 build: remove stale .ipk files if package dir changes
If a package nonshared status is changed, a stale .ipk file might still
be present in the old package directory. Remove the .ipk file from all
package directories when building a new one (or explicitly running
clean)

Signed-off-by: Felix Fietkau <nbd@nbd.name>
8 years ago
Jo-Philipp Wich 317b3556a4 include: properly update .install stamp files
Right now the $(PKG_INSTALL_STAMP) files are only written if a package is
selected as <*> but never deleted or emptied if the corresponding package
is getting deselected.

For ordinary packages this usually is no problem as the package/install
recipe performs its own check for enabled packages when assembling the
list of install stamp files to consider, but this logic might fail under
certain circumstances for packages providing multiple build variants.

In case of a multi-variant package, the buildroot first checks if any
of the variants is enabled, then resolves all variants of the common
source package and finally processes the corresponding .install stamp
files of all variants, relying on the assumption that only the selected
.install stamp file exists.

When an initially selected variant is getting deselected or changed from
<*> to <m> and another variant is marked as <*> instead, the .install
stamp file of the deselected variant remains unchanged and a second
.install stamp file for the newly selected variant is getting created,
causing the package/install recipe to pick up two .install stamps with
conflicting variants, leading to opkg file clashes.

This issue happens for example if package "ip" is set to <m> and package
"ip-full" to <*> -  the install command will eventually fail with:

     * check_conflicts_for: The following packages conflict with ip:
     * check_conflicts_for: 	ip-full *
     * opkg_install_cmd: Cannot install package ip.

In order to fix the problem, always process the removal requests or the
.install stamp files, even for deselected packages but only write the
package base name into the stamp file if the corresponding package is
marked as builtin.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
8 years ago
Matthias Schiffer ec041920b7
include/package-ipkg.mk: use TARGET_PATH_PKG in Package/*/install steps
Package/*/install was using a different PATH than all other steps like
Build/Install, which was confusing and easily led to mistakes.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
8 years ago
Felix Fietkau 71d533eea9 package-ipkg: do not include feeds.mk at metadata dump time to speed up scanning
Signed-off-by: Felix Fietkau <nbd@nbd.name>
8 years ago
Matthias Schiffer 3ee6c17cd1
package-ipkg.mk: fix Provides for packages with multiple PROVIDES entries
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
8 years ago
Jo-Philipp Wich d9ad55a609 include: remove unused FeedPackageDir macro arguments
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
8 years ago
Jo-Philipp Wich 5170393f8c include: choose package output directory based on repository info
Use the new repository metadata field to choose the output directory of the
final package archives.

Non-sharable packages will be placed in the per-target package directory
while the rest will be placed in a per-repository sub directory within the
$OUTPUT_DIR/packages/$CPU_TYPE/ prefix.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
8 years ago
Felix Fietkau e597a4190e package-ipkg.mk: fix typo (patch from #19540)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45568
9 years ago
Felix Fietkau d600f9e957 package-ipkg.mk: fix typo
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45194
9 years ago
Felix Fietkau 765c73f309 package-ipkg.mk: add missing semicolon (#19381)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45193
9 years ago
Felix Fietkau 640318e620 build: ensure that preinst and postrm are executable (fixes #19372)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45187
9 years ago
Felix Fietkau 55c5d10ca9 tools: replace ipkg-utils with a reduced ipkg-build variant in scripts/
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45129
9 years ago
Felix Fietkau 61dd3c8013 build: make postinst/prerm scripts executable
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45128
9 years ago
John Crispin 4a1d29c5fc Fix possible fail to check for dependencies
Few characters in filenames (a plus sign, a dot) can be interpreted
specially by grep. This can lead to the omission of missing package
dependency. For example if we would have "some.file.so" then it
matches also "some2file.so". -F switch off special meaning
of any character and -x match against whole line.

Signed-off-by: Sławomir Demeszko <s.demeszko@wireless-instruments.com>

SVN-Revision: 44247
9 years ago
Nicolas Thill d8263c3025 include: don't use extended regex for deps checking
Closes: #18851

Signed-off-by: Nicolas Thill <nico@openwrt.org>

SVN-Revision: 44224
9 years ago
Jo-Philipp Wich c3787d4c0c include: use anchored patterns for dependency checking
Anchor search pattern when testing dependencies, otherwise the check may
succeed in cases where it shouldn't, e.g. when matching "udp_tunnel.ko"
against "ip6_udp_tunnel.ko".

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 44207
9 years ago
Jo-Philipp Wich 4de4827ddc include: unbreak conffiles, postinst & prerm exports
Changeset r43017 reworked the ipkg control metadata generation but broke
the export of conffiles, postinst and prerm defines.

Change the code back to rely on shvar and shexport, this is required to
properly output multiline contents.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 43041
10 years ago
Felix Fietkau 949ab1487f build: only print skipped packages that were selected
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43023
10 years ago
Felix Fietkau d8278dee4d kernel: use /bin/sh instead of bash to speed up processing of many kmod-* packages
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43022
10 years ago
Felix Fietkau d081edf7eb build: clean up and optimize ipkg control generator code
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43017
10 years ago
Felix Fietkau edc8e2022a build: optimize printing of disabled packages
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 43015
10 years ago
Felix Fietkau c6ebce701e build: propagate package conflicts to opkg metadata
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 42771
10 years ago
John Crispin 44249f71a8 postinst trigger: the new postinst trigger broke IB
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42571
10 years ago
John Crispin d6dbc22bbc ipkg: add a default postinst/prerm script
the postinst script enables/starts the init.d scripts upon package installation
and installs the users required by the package.

the prerm script stops and disables the init.d scripts.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42470
10 years ago
John Crispin 285012d897 include: add a meta data field for required system user:group
this is in preparation of having services run as !root with
ACL'ed access to ubus.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42469
10 years ago
Jo-Philipp Wich 9aeb0cff11 build: fix removal of old .ipk files with CONFIG_PER_FEED_REPO
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42090
10 years ago
Jo-Philipp Wich a720dd209f build: introduce per feed repository support
This changeset implements a new menuconfig option to generate separate
repositories for each enabled package feed instead of one monolithic one.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42002
10 years ago