Commit Graph

92 Commits (fbd4214bb0727268aa535af5b600d38939ec8dbc)

Author SHA1 Message Date
Felix Fietkau d5cd80a1e8 scripts/feeds: fix accepting "-" in feed type string
Fixes a syntax error in processing the type src-git-full

Signed-off-by: Felix Fietkau <nbd@nbd.name>
5 years ago
Bjørn Mork a21b70be31 scripts/feeds: fix 'src-include' directive
Commit 775b70f8d5 renamed parse_file() parameters without
updating the recursive call. This broke parsing of any feeds.conf
using 'src-include'.

 $ scripts/feeds update -a
 Can't use string ("defaults") as a HASH ref while "strict refs" in use at scripts/feeds line 63, <$fh> line 1.

Fixes: 775b70f8d5 ("scripts/feeds: allow adding parameters to feeds")
Signed-off-by: Bjørn Mork <bjorn@mork.no>
5 years ago
Jo-Philipp Wich 775b70f8d5 scripts/feeds: allow adding parameters to feeds
this allows adding "--" prefixed parameters inside feeds.conf between the
target and name. The first parameter is --force which has the same effect
as using -f when installing any of the packages. This allows creating
feeds that will override base packages by default.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years ago
Bjørn Mork 7a1b575ac4 scripts/feeds: add src-include method
The src-include method allows recursive inclusion of feeds.conf snippets.

This can for example be used for adding static local feeds to
feeds.conf.default without ever having to update the local feeds.conf:

 src-include defaults feeds.conf.default
 src-link custom /usr/local/src/lede/custom

Signed-off-by: Bjørn Mork <bjorn@mork.no>
5 years ago
Matthias Schiffer 16035a7dd3
include/feeds.mk: rework generation of opkg distfeeds.conf
Allow enabling/commenting/disabling each feed individually by using a
tristate config symbol.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 6bdd5d8459
scripts/feeds: add src-dummy method
The src-dummy method does not actually obtain any feed, but it can be used
to insert addtional entries into the opkg distfeeds.conf. This is useful to
make package feeds available to users without requiring the corresponding
source feeds to be available during build.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Felix Fietkau 29b2199eb0 build: fix target metadata scan dependencies
Move SCAN_DEPS to scan.mk to eliminate redundancy with scripts/feeds
Add image/*.mk to SCAN_DEPS for targets to pick up newly added devices

Signed-off-by: Felix Fietkau <nbd@nbd.name>
6 years ago
Felix Fietkau 41ab276fe4 scripts/feeds: add support for git feeds with submodules
Signed-off-by: Felix Fietkau <nbd@nbd.name>
6 years ago
Matthias Schiffer 06d51dbb9e
scripts/feeds: fix install of packages with different source/binary names
The logic for choice between source and binary packages was reversed.

Fixes: 52719c2b67 "metadata: scripts/feeds: distinguish between source
and binary packages, resolve virtual dependencies"
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 52719c2b67
metadata: scripts/feeds: distinguish between source and binary packages, resolve virtual dependencies
Properly resolve build depends to source packages and runtime depends to
binary packages. Dependencies on virtual packages are resolved to the first
provider now.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 47d6b05ad3
metadata: always resolve dependencies through provides list
Instead of adding virtual packages to the normal package list, keep a
separate list for provides, make each package provide itself, and resolve
all dependencies through this list. This allows to use PROVIDES to replace
existing packages.

Fixes FS#837.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 423ec18116
metadata: remove redundant fields from package hash
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 504cb913eb
metadata: change pkg->{src} field to hold a reference
We often want to access fields of a source packages through pkg->{src}.
Allow accessing them directly instead of resolving the source hash through
srcpackages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer b2ab9c9d33
metadata: move 'builddepends' from binary to source packages
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Matthias Schiffer 19577582e4
metadata: make srcpackage extensible
Turn the srcpackage values into hashes to allow storing more information
than just binary package names.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
6 years ago
Michal Sojka b1f102b03e scripts/feeds: Reuse TOPDIR if defined in environment
The feeds script sets value of TOPDIR in a way that is inconsistent
with how toplevel Makefile sets it. The inconsistency manifests when I
use a "build directory" with symlinks to LEDE source (see below).

When make is invoked in such a directory, make's TOPDIR variable is
set to that directory, whereas scripts/feeds sets TOPDIR to the top of
LEDE source, which results in creating feeds directory inside the LEDE
source instead of in the build directory.

This patch changes the script so that it reuses the TOPDIR value form
the environment if it exists. The result is that 'make
package/symlinks' correctly fetches feeds to the build directory
instead in the source.

I use the following commands to create the build directory:

    ln -s $SRC/config config
    ln -s $SRC/Config.in Config.in
    ln -s $SRC/feeds.conf.default feeds.conf.default
    ln -s $SRC/include include
    ln -s $SRC/Makefile Makefile
    mkdir package
    ln -s $SRC/package/base-files package/base-files
    ln -s $SRC/package/boot package/boot
    ln -s $SRC/package/devel package/devel
    ln -s $SRC/package/firmware package/firmware
    ln -s $SRC/package/kernel package/kernel
    ln -s $SRC/package/libs package/libs
    ln -s $SRC/package/Makefile package/Makefile
    ln -s $SRC/package/network package/network
    ln -s $SRC/package/system package/system
    ln -s $SRC/package/utils package/utils
    ln -s $SRC/rules.mk rules.mk
    ln -s $SRC/scripts scripts
    ln -s $SRC/target target
    ln -s $SRC/toolchain toolchain
    ln -s $SRC/tools tools

This allows me to easily test changes in LEDE on multiple targets.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
7 years ago
Jo-Philipp Wich 3bfcd21e4b feeds: add option to force feed update despite modified files
Implement a new flag "-f" for the feeds update command which causes the
script to fall back to a more agressive git update strategy in case there
are locally modified files in the feeds directory.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years ago
Rafał Miłecki 0a6439154a scripts/feeds: use git rev-parse for getting revision
It provides simpler output so we don't need extra head and cut commands.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
8 years ago
Rafał Miłecki 4f7947dab8 scripts/feeds: display "X" as revision of uninitialized feeds
So far we were displaying "local" which could be misinterpreted. It
wasn't possible e.g. to say if src-link feed was initialized or not.
Hopefully "X" makes (a bit) more sense.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
8 years ago
Rafał Miłecki a2944a0c09 scripts/feeds: use 10 chars for feed name column width
It's always hard to find a reasonable width that will make everyone
happy. This one at least makes "telephony" (one of default feeds) name
fit the column and hopefully isn't too big.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
8 years ago
Daniel Dickinson 34527688fc scripts/feeds: Prevent .config autocreation
When using scripts/feeds upgrade the .config needs to be
updated but the code to do so was also autocreating a
.config if one didn't exist.  This is counter-productive
when you have not yet used menuconfig (or other config targets)
because things like selecting or deselecting CONFIG_ALL
(to build all package by default) only works if the
package selection has not already been done via an
existing .config selection.

Signed-off-by: Daniel Dickinson <lede@daniel.thecshore.com>
8 years ago
Jo-Philipp Wich df18b3756f scripts: feeds: fix version detection for Make >= 4.2.1
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
8 years ago
Iblis Lin d9a0a8c78f scripts: avoid hard-coded paths in scripts
Signed-off-by: Iblis Lin <e196819@hotmail.com>
8 years ago
Jonas Gorski 0d54d44349 scripts/feeds: allow export of feeds with static revisions
Add a switch to scripts/feeds that allows listing the feeds with their
currently checked out revisions in feeds.conf compatible format.

This allows providing a feeds.conf for public builds to make replication
of the build easier.

Signed-off-by: Jonas Gorski <jogo@openwrt.org>

SVN-Revision: 46706
9 years ago
Jo-Philipp Wich c5fbcf59e0 scripts/feeds: return error status from feed update
This patch is a modified version of the patch being discussed at here:
https://patchwork.ozlabs.org/patch/471303/
from Martin Strbacka <martin.strbacka@nic.cz>

My version modifies scripts/feeds so that an error with one of the feeds just
raises the error flag, but script continues and tries to update the other
feeds. After all feeds have been updated, the script returns 1 if at least
one feed failed, and 0 on success with all feeds. The user can then utilise
the status in his build script, if he wants.

signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 46374
9 years ago
Steven Barth 37c015fe25 scripts/feeds: print the source of packages installed
Can be helpful in diagnosing unexpected package overlaps between
feeds/core.

Signed-off-by: Karl Palsson <karlp@remake.is>

SVN-Revision: 46255
9 years ago
Steven Barth b6d0061fec scripts/feeds: observe -p flag for preferential feeds
lookup_target was trampling the $feed variable, resulting in the -p flag
no longer preferentially installing from the named feed.

Make sure to use a local variable for this instead.

Signed-off-by: Karl Palsson <karlp@remake.is>

SVN-Revision: 46120
9 years ago
Felix Fietkau 7de4c92804 scripts/feeds: fix typo in copied code
do_install_target copied code from do_install_package, but didn't
replace all text.

Signed-off-by: Karl Palsson <karlp@remake.is>

SVN-Revision: 45784
9 years ago
John Crispin 803ebd2725 scripts/feeds: Delete package/feeds folder in the clean procedure to prevent dangling links.
Hello,

if you run:
./scripts/feeds clean
It removes ./feeds folder but not ./package/feeds/ which is full of dangling links then. This patch fixes it.

Best Regards,
Martin Strbačka

Signed-off-by: Martin Strbacka <martin.strbacka@nic.cz>

SVN-Revision: 45738
9 years ago
Felix Fietkau 08436887fb scripts/feeds: add support for optionally using a full clone of git repositories
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45668
9 years ago
Felix Fietkau 4b1165d5c7 scripts/feeds: add support for uninstalling targets
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44782
9 years ago
Felix Fietkau 2484f3a86a scripts/feeds: add support for installing targets
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44781
9 years ago
Felix Fietkau b7ebb19b02 scripts/feeds: get rid of redundant feed cache copy
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44779
9 years ago
Felix Fietkau 0efcb02c9d scripts/feeds: remove useless install_method abstraction
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44778
9 years ago
Felix Fietkau b6ac0d799c scripts/feeds: add support for searching for targets
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44777
9 years ago
Felix Fietkau f8b95e05f1 scripts/feeds: scan for target metadata
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44775
9 years ago
Felix Fietkau 1592f3f607 scripts/feeds: remove a few trailing tabs
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 44773
9 years ago
John Crispin 2e2c04777b build: allow openwrt.git packages to be replaced by feeds
Currently, replacing a package available in openwrt.git requires
modifications in openwrt.git, or requires duplicating the package in a
feed but with a different name, which causes all kind of problems
related to dependencies (all packages selecting it would have to be
modified accordingly to select the new package).

With this change, if a package with the same name is present both in
feeds/ and package/ folders, the one in feeds/ can override the one
in package/, both in the menuconfig and during the build, by passing the
"-f" option to "./scripts/feeds install"

This mechanism is particularly useful for vendor tree, or in general for
application which needs to replace one particular package which exists
within openwrt.git by a custom/newer version.

Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>

SVN-Revision: 44334
9 years ago
John Crispin b6f268d775 build: revert r44076
the patch causes issues when installing feeds

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 44078
9 years ago
John Crispin de3bf7e4c2 build: allow openwrt.git packages to be replaced by feeds
Currently, replacing a package available in openwrt.git requires
modifications in openwrt.git, or requires duplicating the package in a
feed but with a different name, which causes all kind of problems
related to dependencies (all packages selecting it would have to be
modified accordingly to select the new package).

With this change, if a package with the same name is present both in
feeds/ and package/ folders, the one in feeds/ can override the one
in package/, both in the menuconfig and during the build, by passing the
"-f" option to "./scripts/feeds install"

This mechanism is particularly useful for vendor tree, or in general for
application which needs to replace one particular package which exists
within openwrt.git by a custom/newer version.

Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>

SVN-Revision: 44076
9 years ago
John Crispin 590b855c26 scripts/feeds: handle missing/broken feeds better
pts/feeds update -a" can fail rather silently for feeds using git, as
the script does not pause when updating a feed fails. Instead it prints the
error message and calmly continues to the next feed. It is very easy to
overlook update errors with the feeds updated first, as their text scrolls
rapidly away from the screen.

This behaviour has not been a big problem with svn feeds, as svn update stops
with a conflict message and interactively forces the user to resolve or
postpone the conflict. In any case the svn error is noticed by the user.

Majority of the feeds use now git, so this silent failure can affect users
doing private builds in an increasing amount.

Below is an example of update failing and script continuing:

perus@v1404:/Openwrt/barrier$ ./scripts/feeds update -a
Updating feed 'packages' from
'https://github.com/openwrt/packages.git;for-14.07' ...
remote: Counting objects: 17, done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 17 (delta 10), reused 8 (delta 1)
Unpacking objects: 100% (17/17), done.
 From https://github.com/openwrt/packages
62031da..dc26009  for-14.07  -> origin/for-14.07
Updating 62031da..dc26009
error: Your local changes to the following files would be overwritten by merge:
utils/collectd/Makefile
Please, commit your changes or stash them before you can merge.
Aborting
failed.
Updating feed 'luci' from 'http://git.openwrt.org/project/luci.git;luci-0.12' ...
Already up-to-date.
Create index file './feeds/luci.index'
Updating feed 'routing' from
'https://github.com/openwrt-routing/packages.git;for-14.07' ...
...

The script prints "failed.", but does not break the updating process. The
"update_feed" function returns an error code 1, but that value is not checked
in the "update" function, which continues to the next feed.
Return 1 as error:
​https://dev.openwrt.org/browser/trunk/scripts/feeds#L547
Call to update_feed without any error monitoring:
​https://dev.openwrt.org/browser/trunk/scripts/feeds#L585

The included patch makes the feeds script to stop updating after failing to
update a feed.

The script continues to the refresh_config step despite a possible failure in
updating, so the stopping action just prevents the other feeds from updating
and makes the error more clearly visible.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 42891
10 years ago
Jo-Philipp Wich a720dd209f build: introduce per feed repository support
This changeset implements a new menuconfig option to generate separate
repositories for each enabled package feed instead of one monolithic one.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42002
10 years ago
Felix Fietkau dcb5d00875 scripts: unset GREP_OPTIONS in env and feeds (fixes #16924)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 41407
10 years ago
Luka Perkov 49722e7dcb feeds: enable cloning of specific git commit
Following syntax should be used to enable this feature:

src-git custom git://openwrt/custom-feed^SHA_COMMIT_ID

Signed-off-by: Luka Perkov <luka@openwrt.org>

SVN-Revision: 38172
11 years ago
Florian Fainelli a1267290ed scripts/feeds: ensure that --ff is enabled
Users may have merge.ff set to false, which will force merge commits to
be generated.  The intent here is to have a fast-forward merge when
possible, so let's make sure fast-forwards are enabled.

Signed-off-by: John Szakmeister <john@szakmeister.net>
Signed-off-by: Florian Fainelli <florian@openwrt.org>

SVN-Revision: 37884
11 years ago
Jo-Philipp Wich 103b69e250 feeds: fix check for core packages that are not in the package/ toplevel directory
SVN-Revision: 36763
11 years ago
Felix Fietkau 3df534502c scripts/feeds: redirect stderr of the which call to /dev/null to avoid potentially confusing error messages on some systems
SVN-Revision: 32499
12 years ago
Felix Fietkau 85f1a2c67a add further countermeasures against the git core.autocrlf option (fixes #9075)
SVN-Revision: 31650
12 years ago
Felix Fietkau 34ffb526a2 build: rework verbosity level selection
V=99 and V=1 are now deprecated in favor of a new verbosity class system,
though the old flags are still supported.
You can set the V variable on the command line (or OPENWRT_VERBOSE in the
environment) to one or more of the following characters:

- s: stdout+stderr (equal to the old V=99)
- c: commands (for build systems that suppress commands by default, e.g. kbuild)
- w: warnings/errors only (equal to the old V=1)

SVN-Revision: 31484
12 years ago
Mirko Vogt 1f489e4084 re-enable shallow git clones ('--depth 1')
SVN-Revision: 31469
12 years ago