sudoxreboot/fdroiddata
1
0
Fork 0
mirror of https://github.com/sudoxnym/fdroiddata.git synced 2026-07-29 02:07:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
128646 commits 3 branches 0 tags 307 MiB
Commit graph

315 commits

Author SHA1 Message Date
Hans-Christoph Steiner
f7bf5a747a
gitlab-ci: improved detection of actual builds for `fdroid build 
Apps and Builds that are disabled should obviously not be run.  But if a
merge request disables a single build and adds a build, then `fdroid build`
should run on the new build.

https://gitlab.com/fdroid/fdroiddata/-/merge_requests/7821#note_458798083
 2020-12-02 16:25:29 +01:00
Hans-Christoph Steiner
2346b1565b
gitlab-ci: check all .yml files with yamllint 
fdroidserver!721
 2020-11-17 19:08:24 +01:00
Hans-Christoph Steiner
d37179a4a7
gitlab-ci: git URL without redirect to avoid warning 2020-10-21 12:19:15 +02:00
Marcus Hoffmann
fcef821b6b checkupdates-trigger: use a working parallels version 
GNU parallels in Debian is 4 years out of date, missing about 40
releases and not updated in sid either.
The version in Debian claims to support the -X option for
splitting arguments in $jobs buckets but generates completely bogus jobs,
where the first job contains something like 80% of all arguments and the
other jobs contain much less.
It also generates 11 jobs when we asked for 10... :-/.

Also simplify the setup further by using parallel's {/.} replacement
pattern, which strips leading path's and extensions instead of bash
pattern substitution magic.
 2020-09-15 14:26:51 +02:00
Marcus Hoffmann
38022eead0 checkupdates: always run 10 jobs, split arguments between them 
Previously we limited the number of arguments for each job, but this
meant we outgrow our current concurrency value.

The `-X` argument makes parallel split all arguments between the 10 jobs
 2020-09-14 20:34:34 +02:00
Hans-Christoph Steiner
11a7290a48 gitlab-ci: fix fdroid build for new metadata files 
!6845
 2020-06-25 21:22:21 +02:00
Hans-Christoph Steiner
07e70de82e
gitlab-ci: make 'fdroid build' job fetch srclibs 
This uses @uniqx's awesome new 'plugin' feature to create a 'fetchsrclibs`
command.  The 'fdroid build' gitlab-ci job uses --on-server, which does not
set up the srclibs.  This plugin does the missing setup.
 2020-06-25 20:36:09 +02:00
Marcus Hoffmann
c3808b511f CI: checkudpates: remove debug output of upstream URL 2020-06-25 15:24:36 +02:00
Marcus Hoffmann
543979b083 checkupdates: use ssh deploy key 
closes fdroid/fdroiddata#2073
 2020-06-25 15:06:55 +02:00
Marcus Hoffmann
83ee207ea8 checkupdates: retry git rebase/push once 
This currently fails once every few days, so let's hope that in retrying
once we can make this almost never happen.
 2020-06-24 20:50:01 +02:00
Hans-Christoph Steiner
8d99dfd3c6
gitlab-ci: only run fdroid build on changed build entries 2020-06-23 08:58:57 +02:00
Hans-Christoph Steiner
626bcdb8d1
gitlab-ci: fix creation of 'fdroid build' job 
This should be run on all branches in forks and all merge requests, as
long as they change a metadata file.  Unfortunately gitlab-ci's "changes"
logic is not as sophisticated as the logic written in the job for detecting
which files changed.  gitlab-ci's logic only detects changes that happened
in the current push event, this job's logic compares to master.

Using gitlab-ci's "changes" logic means that if someone changed a metadata
file in a merge request, "fdroid build" would be triggered.  If they then
updated the merge request with a change to another file, then the next run
would not include a "fdroid build" job, since no metadata file was changed
in that push.
 2020-06-23 08:58:54 +02:00
Hans-Christoph Steiner
979d42858e
gitlab-ci: improve issuebot trigger for merge requests 
This should hopefully provide reliable triggering for issuebot on merge
requests.  Before, detecting the merge request number was happening on the
receiving side, and it had weird issues.
 2020-06-18 18:59:45 +02:00
Hans-Christoph Steiner
18b80e5b39 gitlab-ci: ditch 'merge_requests' rule, it has strange behavior 
It makes its own pipeline, which hides other jobs like lint which should be
displayed together on the merge request.

https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html
 2020-06-18 12:58:16 +02:00
Marcus Hoffmann
b6e01ce401 CI: remove except variables for lint and pages 
Those shouldn't be needed in the current setup.
lint is already exempted in pipelines triggered via CI_JOB_TOKEN

pages: is the only triggered job currently (via an explicit
TRIGGER_TOKEN, checkupdates gets triggered via CI_JOB_TOKEN)
 2020-06-18 12:42:59 +02:00
Marcus Hoffmann
d99508f014 CI: fix typo piplines -> pipelines 2020-06-18 11:25:23 +02:00
Hans-Christoph Steiner
54fd723992 gitlab-ci: log commit used when using git projects from tarballs 2020-06-16 19:57:30 +02:00
Marcus Hoffmann
b1e410972f CI: rename checkuodates image 2020-06-10 12:20:48 +02:00
Marcus Hoffmann
8c455c75ba CI: checkupdates runner seems to cope with up to 10 parallel jobs 
Also set 90s start delay instead of 60.
 2020-06-06 00:01:08 +02:00
Marcus Hoffmann
67af4ef3df CI: remove now obsolete package installs 2020-06-05 22:46:42 +02:00
Marcus Hoffmann
acc0d0722c CI; better checkupdates job scheduling 
* run 250 app batches, let's see if our runner can handle the resulting
  14 jobs in parallel
* stagger the startup of the jobs by 60 seconds
* use a specialized image so we save setup time per job
 2020-06-05 22:39:45 +02:00
Marcus Hoffmann
b4f3db616f run checkupdates on gcc farm runners 2020-06-05 18:40:16 +02:00
Marcus Hoffmann
bd8a4db539 use gnu parallel to trigger parallel checkupdates pipelines 
Batch checkupdate invocations into calls with 100 apps. This should
allow more than enough headroom for gitlab CI runner limits.
Currently it'll spawn 34 child pipelines, these each complete in about
15-20 min.
 2020-06-05 03:21:39 +02:00
Marcus Hoffmann
ffaee55b36 CI: set git email such that the gitlab avatar is detected correctly 2020-06-04 20:32:51 +02:00
Marcus Hoffmann
29d3040ce9 CI: fix apt-get 2020-06-04 20:05:54 +02:00
Marcus Hoffmann
137eac6c0e CI: fix schedules abort line 2020-06-04 19:57:24 +02:00
Marcus Hoffmann
90eec1a4bd CI: trigger only the job we want in schedules 2020-06-04 19:50:15 +02:00
Marcus Hoffmann
b27f3cde83 CI: need allow-dirty because we have fdroidserver under $PWD 2020-06-04 19:41:58 +02:00
Marcus Hoffmann
ecc214bc75 CI: typo 2020-06-04 19:35:55 +02:00
Marcus Hoffmann
a378e81183 gitlab CI checkupdates run 2020-06-04 19:30:56 +02:00
Hans-Christoph Steiner
905ce30d37 gitlab-ci: build MRs with provisioned ci-images-client image 
This purges the old attempt at doing something similar.
 2020-06-02 17:17:49 +02:00
Hans-Christoph Steiner
6d9777c134 gitlab-ci: check git repo availability for metadata and srclibs MRs 
This also ports check-git-repo-availability.py to Debian/stretch for the
buildserver image.
 2020-06-02 17:12:24 +02:00
Hans-Christoph Steiner
e7c78b5ad1 gitlab-ci: set up for srclibs lint 2020-06-02 15:33:31 +02:00
Hans-Christoph Steiner
894e1a8be0
gitlab-ci: only look for .yml in fdroid-buildserver job 2020-05-26 11:23:55 +02:00
Hans-Christoph Steiner
ce0969d18b
gitlab-ci: make merge requests trigger issuebot 2020-05-21 11:55:04 +02:00
Hans-Christoph Steiner
5fb8e612e6
gitlab-ci: add issuebot pages job to run on merge requests 
This won't run automatically yet since the trigger has not been setup. But
it will be there for manual testing until the trigger stabilizes.  There
can be only one "pages" job, so git_stats was moved to fdroid.gitlab.io.
 2020-05-20 15:29:32 +02:00
Hans-Christoph Steiner
db5f0f0081
gitlab-ci: stop looking for metadata/*.txt, they are gone and obsolete! 2020-05-20 15:17:48 +02:00
Hans-Christoph Steiner
9ccf886978
gitlab-ci: check active git repos to ensure they're available 
This then edits the metadata file for apps with failing git repos, then
includes those edited files in the gitlab job artifacts.

fdroid/fdroiddata!5262
 2019-08-26 11:24:17 +02:00
Hans-Christoph Steiner
a972bb6c93
gitlab-ci: do not run fdroid-buildserver automatically 2019-08-15 14:34:37 +02:00
Hans-Christoph Steiner
d170d3e5fb
gitlab-ci: do not run fdroid-buildserver jobs on my fork 2019-08-15 14:32:51 +02:00
Hans-Christoph Steiner
24ce07db01
gitlab-ci: use androguard and python3-ruamel.yaml from backports 
This should fix the lint issue in:
https://gitlab.com/fdroid/fdroiddata/merge_requests/5250
 2019-08-15 14:32:49 +02:00
Michael Pöhn
178b2625c5 skip git checkout if NoSourceSince is set 2019-07-03 21:08:53 +02:00
Hans-Christoph Steiner
9e1177255c
gitlab-ci: ensure localized metadata has proper format 2019-05-20 14:26:39 +02:00
Hans-Christoph Steiner
180a6c93af
gitlab-ci: check that localized metadata is properly named 2019-05-10 13:37:18 +02:00
Marcus Hoffmann
f03619945d
CI: don't git clone git-svn repos 2019-02-21 14:08:36 +01:00
Hans-Christoph Steiner
a89b121a3b gitlab-ci: temp disable buildserver ci 2019-01-03 23:18:11 +01:00
Hans-Christoph Steiner
c613b43520 gitlab-ci: allow @beuc's runner 2018-12-17 16:31:24 +01:00
Hans-Christoph Steiner
c87a57a430 gitlab-ci: do not rely on static paths for the gitlab runner 
Before, this assumed that the gitlab-runner was running as the 'fdroid'
user.  Now, it shouldn't matter which user it is running as.
 2018-12-08 09:48:13 +01:00
Marcus Hoffmann
0ea601afc7 CI: enable for Bubu instead of bubu 2018-11-26 21:50:07 +01:00
Hans-Christoph Steiner
d1a38247d5 fix typo: fdroidata --> fdroiddata 2018-11-23 15:14:06 +01:00
Hans-Christoph Steiner
0594d15ed6 gitlab-ci: only run fdroid-buildserver for those with runners 2018-11-23 15:11:30 +01:00
Hans-Christoph Steiner
f2c4bff0a3 gitlab-ci: cache APKs and git repos between runs 2018-11-15 18:06:32 +01:00
Hans-Christoph Steiner
f129254d82 gitlab-ci: use official build flags for all builds 2018-11-15 15:07:50 +01:00
Hans-Christoph Steiner
2b975a3274 gitlab-ci: run LibScout on any APKs that were produced 2018-11-15 15:07:26 +01:00
Hans-Christoph Steiner
2753ecef80 gitlab-ci: build recent commits on the main repo 2018-11-14 16:56:58 +01:00
Hans-Christoph Steiner
a31aa3d100 gitlab-ci: run fdroid build on new buildserver runner 2018-11-14 13:37:51 +01:00
Hans-Christoph Steiner
c6e6a04960 gitlab-ci: only run gitstats when scheduled 2018-09-21 10:56:34 +02:00
Hans-Christoph Steiner
c138e75a8d gitlab-ci: GitStats page in GitLab Pages 2018-09-21 10:13:11 +02:00
Hans-Christoph Steiner
c24ce3007b gitlab-ci: use xargs when running exiftool 
This should handle long argument lists better.
 2018-08-29 22:49:57 +02:00
Hans-Christoph Steiner
8dade6ece0 gitlab-ci: check if Application ID will cause keyalias collision 
fdroid/fdroidserver#553
 2018-08-29 22:49:51 +02:00
Hans-Christoph Steiner
cddf3589f7 gitlab-ci: make sure EXIF is stripped from all included images 
EXIF state can be used as an exploit vector and for malicioius code
delivery.  Neither fdroidclient nor fdroid-website use the EXIF data, so
just strip it entirely.  'exiftool' is available in most distros.
 2018-03-20 20:16:53 +01:00
Marcus Hoffmann
8852fde452
CI: don't try to fetch repos of deleted apps. 
--diff-filter=d excludes (because of the lowercase letter) deleted
files.
 2018-03-12 13:34:23 +01:00
Hans-Christoph Steiner
253a1b144c gitlab-ci: only check commits included in the current merge request 2018-02-28 22:12:04 +01:00
Hans-Christoph Steiner
38d0e8cd39 gitlat-ci: fix detection of appids changed in this merge request 
It was detecting changed description files in subfolders, i.e.:
metadata/org.fdroid.fdroid.ota/en-US/description.txt
metadata/org.fdroid.fdroid.ota/en-US/summary.txt
 2018-02-23 22:37:34 +01:00
Hans-Christoph Steiner
23a62fd5fa gitlab-ci: download fdroidserver as tarball rather than git clone 
This should hopefully be quicker and lower impact.
 2018-02-23 22:32:55 +01:00
Hans-Christoph Steiner
d6e81e47bb gitlab-ci: error if apps use insecure plain HTTP gradle repositories 
It is very easy to mess up and include plain HTTP URLs for gradle
repositories, which can lead to gradle downloading code from HTTP and
immediately executing it.  The fix is almost always changing "http:" to
"https:".

https://max.computer/blog/how-to-take-over-the-computer-of-any-java-or-clojure-or-scala-developer
 2018-01-26 13:41:41 +01:00
Hans-Christoph Steiner
fed1acd8f6 gitlab-ci: if running on a fork, only check the files in the fork 
To avoid confusing newbies, the CI should only run on the files that were
changed in the fork.  Sometimes lint/rewrite errors slip into the git repo,
and the CI for all merge requests will fail until those are fixed.  This
changes that to make the merge request CI only fail if the files in the
merge request are the ones with problems.

The CI runs on the main repo should will check everything.
 2018-01-26 13:41:41 +01:00
Hans-Christoph Steiner
58070a0982 update gitlab-ci base image to Debian/stretch 
https://forum.f-droid.org/t/f-droid-ci-images-reorg
 2017-07-03 13:07:10 +02:00
Hans-Christoph Steiner
5115d51d8c gitlab-ci: use server image from https://gitlab.com/fdroid/ci-images 
This image provides everything needed to run all aspects of fdroidserver,
and is only like 100 megs bigger than the base image that this was using
before.
 2017-04-04 21:56:20 +02:00
Hans-Christoph Steiner
92b09c7ef7 gitlab-ci: show diff when there are lint failures 
Oftentimes, the lint failures are just simple whitespace issues.  Many
people submitting metadata do not really know much about the fdroid tools,
so they have to look up or ask what rewritemeta is.  This just shows a diff
after running rewritemeta so it should be clearer what the issue is.
 2017-01-26 14:28:11 +01:00
Daniel Martí
ffab5a5ad6 CI: bump image 2016-12-23 12:45:31 +01:00
Daniel Martí
d35c748ee7 CI: bump image 2016-10-23 17:14:10 +01:00
Daniel Martí
271bd368b9 CI: bump docker version 2016-09-10 12:09:23 +02:00
Daniel Martí
6400ac98c3 CI: bump docker image 2016-08-21 11:00:50 +02:00
Daniel Martí
f8e0dd58cf CI: bump image 2016-07-31 18:48:11 +02:00
Daniel Martí
a875c40e56 CI: bump docker image 2016-07-06 11:37:26 +01:00
Daniel Martí
fb2a5dfd7d CI: bump docker image 
To be in sync with the other repos.
 2016-06-23 11:44:33 +01:00
Daniel Martí
145a9d2ab2 CI: bump image 2016-06-20 11:18:48 +01:00
Daniel Martí
396ab81a42 CI: bump docker image 2016-06-13 19:33:59 +01:00
Daniel Martí
28d04376c0 CI: Bump docker image 2016-05-12 23:11:17 +01:00
Daniel Martí
dd83c50a92 CI: bump image, re-add --depth=1 to clone 2016-04-29 19:17:23 +01:00
Daniel Martí
14da2ce04e CI: Temporarily remove --depth=1 
See https://gitlab.com/gitlab-com/support-forum/issues/652.
 2016-04-18 15:22:42 +01:00
Daniel Martí
9e52a153a2 CI: Bump image 2016-04-05 10:52:37 +01:00
Daniel Martí
8f8a427db7 CI: Use the new base image 
Lighter, since it doesn't include all the stuff required to build and
test the client that we don't need here.
 2016-03-29 13:36:43 +01:00
Daniel Martí
b55f2a7628 CI: Bump docker image 
Now python3 comes installed.
 2016-03-17 14:17:15 +00:00
Daniel Martí
1df2d03ed3 CI: Install py3 deps until image is updated 2016-03-13 13:27:00 +00:00
Daniel Martí
255781f489 CI: No longer need to install deps 2016-02-29 22:57:21 +00:00
Daniel Martí
44824d6946 CI: Specify our docker image 2016-02-29 14:57:49 +00:00
Daniel Martí
05b7c67542 CI: Simplify script 2016-02-26 16:37:50 +00:00
Daniel Martí
485c6b4aa2 CI: explicitly install git 
The base system might not have it.
 2016-01-07 15:06:29 +01:00
Daniel Martí
27ec7f82f8 CI: run lint -f 2015-12-08 17:58:12 +01:00
Daniel Martí
5741599f1c CI: clone --depth=1 is enough 2015-11-14 23:49:53 +01:00
Daniel Martí
eb45f97576 CI: fix path typo 2015-09-11 18:41:15 -07:00
Daniel Martí
cbf2de9caa CI: check formatting via rewritemeta 2015-09-11 18:37:56 -07:00
Daniel Martí
6f26316caa Set up fdroidserver at the start of PATH 2015-09-02 11:51:14 -07:00
Daniel Martí
761f923fb7 CI: add python-yaml dependency 2015-09-01 11:45:02 -07:00
Daniel Martí
895a9a4a68 Gitlab CI: server no longer attached to magic and requests 2015-08-31 17:06:33 -07:00
Daniel Martí
26eca05d4d Gitlab CI: echo what we're doing 2015-08-27 15:50:20 -07:00
Daniel Martí
06ce8f95b7 Gitlab CI: make apt-get quiet 2015-08-27 15:40:18 -07:00
Daniel Martí
7f65876811 Join readmeta and lint into the same task/job 
This way, the preparing commands are only run once. Also, if readmeta fails,
lint will fail with the same message. So having the two messages thrown in
parallel is of little use.
 2015-07-30 14:08:28 -07:00
Daniel Martí
f7eae17fae Also run lint as part of the CI job 2015-07-30 12:41:50 -07:00
Daniel Martí
c870b7311d Set up fdroiddata with empty config.py 2015-07-30 12:36:17 -07:00
Daniel Martí
53cdcedcbc Run fdroid readmeta inside fdroiddata 2015-07-30 12:34:40 -07:00
Daniel Martí
2cd2c669fe Add deps necessary for fdroid readmeta 2015-07-30 12:33:07 -07:00
Daniel Martí
3061e4856e Remove existing fdroidserver clone 
It may be outdated or wrong
 2015-07-30 12:28:58 -07:00
Daniel Martí
75576e013a Give up on setup.py 2015-07-30 12:20:01 -07:00
Daniel Martí
f2370fed5b https clone address requires .git 2015-07-30 11:57:48 -07:00
Daniel Martí
5cb76d0f89 Run setup inside dir, clone instead of wget 2015-07-30 11:55:39 -07:00
Daniel Martí
643e5c9d90 Apparently no sudo required 2015-07-30 11:52:26 -07:00
Daniel Martí
6fc3f02b9f Make sure python-setuptools is installed 2015-07-30 11:47:29 -07:00
Daniel Martí
c7437a8f9f Use setup.py directly 2015-07-29 17:10:49 -07:00
Daniel Martí
2d00570a48 Apparently custom images aren't allowed on ci.gitlab.com 
Output from ci.gitlab.com:

ERROR: No images are allowed
Please check runner's configuration: http://doc.gitlab.com/ci/builds_configuration/docker.html#overwrite-image-and-services
 2015-07-29 16:57:53 -07:00
Daniel Martí
bc2328899a Use python 2.7 docker image 2015-07-29 16:52:09 -07:00
Daniel Martí
592b9fba97 Fix CI job 2015-07-29 16:41:37 -07:00
Daniel Martí
3bc655d288 First attempt at Gitlab CI yml file 2015-07-29 16:16:43 -07:00