heptapod issueshttps://foss.heptapod.net/groups/heptapod/-/issues2023-09-05T12:37:26Zhttps://foss.heptapod.net/heptapod/heptapod/-/issues/724Webhooks sent ' branch/default' as branch name2023-09-05T12:37:26ZCédric Kriercedric.krier@b2ck.comWebhooks sent ' branch/default' as branch nameTrying to use the gitlab webhook for weblate: https://docs.weblate.org/en/latest/admin/continuous.html#automatically-receiving-changes-from-gitlab
It does not work because the name of the branch sent is `branch/default` instead of `defa...Trying to use the gitlab webhook for weblate: https://docs.weblate.org/en/latest/admin/continuous.html#automatically-receiving-changes-from-gitlab
It does not work because the name of the branch sent is `branch/default` instead of `default`. So weblate does not find a matching branch then it ignore the webhook (instead of pulling the repository).
It will be great if Heptapod could sent the mercurial name of the branch.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/205host hg-confman2023-02-06T11:27:34ZAurélien Campéashost hg-confman## Description of the projects to be hosted
- Project Name: hg-confman
- Project URL: https://hg.sr.ht/~auc/confman
- Project Maintainers: Aurélien Campéas, and now people at orus.io
- Project License: GPL v2+
Please explain here how t...## Description of the projects to be hosted
- Project Name: hg-confman
- Project URL: https://hg.sr.ht/~auc/confman
- Project Maintainers: Aurélien Campéas, and now people at orus.io
- Project License: GPL v2+
Please explain here how they meet our current criteria.
### Main criteria
- [X] Free / Open Source
- [X] main development repositories
- [X] you are an official maintainer of the project
### Additional information
Knowing this can help in case there are constraints on resources and/or
administrators availability at the time of the request. It can also help us
suggest the right solution for the needs of the project.
- is the project officially released? In downstream distributions or
packaging systems? Yes and Yes
- estimation of the user base if possible, or number of downloads,
reverse dependencies… : Small user base
- number of maintainers and regular contributors: Very small
- where is the project currently hosted? Are there issue trackers,
review systems, CI/CD to hook to or to replace somehow? -> hosted on sourcehut, currently no infra but one CI will be wanted
### Hosting type
What would you like to have?
- [ ] a top-level group
- [X] a project in an existing group (Mercurial)
- [ ] just a project in your personal namespace
## Post approval maintainer task list
See [README](README.md#maintainer-task-list-for-the-import-to-happen)
for explanations.
- [ ] Reviewing [Heptapod workflow](https://heptapod.net/pages/faq.html#workflow)
## Post import maintainer task list
- [ ] Add the Octobus and Clever cloud acknowledgement at the selected place.
- [ ] Communicate about the migration to help others to aware of the options
(for example on mastodon, twitter, bitbucket thread, mailing list, …)Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod-runner/-/issues/44Heptapod PAAS Runner: OpenStack support2023-01-17T13:17:15ZGeorges RacinetHeptapod PAAS Runner: OpenStack supportInstead of having 8 permanent runners at OSUOSL for a grand total of 96 vCPUs and 192GB of RAM with the [concurrency problems induced in them](foss.heptapod.net#202), and the related very low concurrency wasting resources, we would be be...Instead of having 8 permanent runners at OSUOSL for a grand total of 96 vCPUs and 192GB of RAM with the [concurrency problems induced in them](foss.heptapod.net#202), and the related very low concurrency wasting resources, we would be better off with on the fly provisioning.
Of course, that could be of use with any OpenStack provider.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/203Clever Runner: problems Cellar downloads ( large job caches, clone-bundles)2023-02-04T14:18:23ZGeorges RacinetClever Runner: problems Cellar downloads ( large job caches, clone-bundles)The `clever-cloud` runner (see #196) has a shared (S3) cache. See [GitLab docs](https://docs.gitlab.com/ce/ci/caching/) about that. This should end up in some user documentation about foss.h.n.
Projects with lots of parallel jobs will p...The `clever-cloud` runner (see #196) has a shared (S3) cache. See [GitLab docs](https://docs.gitlab.com/ce/ci/caching/) about that. This should end up in some user documentation about foss.h.n.
Projects with lots of parallel jobs will probably need to make sure that their jobs don't race to update the cache. Upstream GitLab general advice is to have a separate stage for cache update (with no concurrency) and the parallel jobs only pulling the cache in a later stage.
Also we currently experience problems with very large caches on the S3 backend, such as in line 66 of [this job](https://foss.heptapod.net/heptapod/heptapod/-/jobs/1608078#L66). This is currently under investigation by network administrators. The same problem also occurred at least once with the clone-bundle for Trytonhttps://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/202Slow jobs at OSUOSL2023-01-26T14:54:44ZGeorges RacinetSlow jobs at OSUOSLAccording to @dan, there are frequent job timeouts, such as [this one](https://foss.heptapod.net/mercurial/hg-git/-/jobs/1608226)According to @dan, there are frequent job timeouts, such as [this one](https://foss.heptapod.net/mercurial/hg-git/-/jobs/1608226)https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/43Heptapod PAAS Runner: logging format options2023-01-15T13:52:16ZGeorges RacinetHeptapod PAAS Runner: logging format optionsThe current format includes process id and date, but that is redundant when logs are handled by `systemd-journald`, making the log harder to watch for humans.
```
Jan 15 12:35:29 heptapod-foss heptapod-paas-runner[447399]: 2023-01-15 12...The current format includes process id and date, but that is redundant when logs are handled by `systemd-journald`, making the log harder to watch for humans.
```
Jan 15 12:35:29 heptapod-foss heptapod-paas-runner[447399]: 2023-01-15 12:35:29,184 [447399] heptapod_runner.paas_dispatcher INFO Launching PAAS Ressource for job 1570198 for runner 'ee4cgV2g'
```https://foss.heptapod.net/heptapod/heptapod/-/issues/7232 pipelines for 1 merge request2023-01-12T10:15:50ZCédric Kriercedric.krier@b2ck.com2 pipelines for 1 merge requestWithout `workflow`, Heptapod triggers 2 pipelines, 1 that does not seem to detect the changed files and another with the merge request content as modified files.
Here is an example: https://foss.heptapod.net/tryton/tryton/-/merge_reques...Without `workflow`, Heptapod triggers 2 pipelines, 1 that does not seem to detect the changed files and another with the merge request content as modified files.
Here is an example: https://foss.heptapod.net/tryton/tryton/-/merge_requests/152/pipelines
I cancelled the first one because it runs all the jobs which is resource consuming.
I tried to remove the workflow as it does not trigger a pipeline when:
* push directly to the protected branches
* when publish changeset from command line
The problem is that it is not clear how to use the CI variables in the Heptapod context because the documentation is based on Git design with fork.
But basically we could expect from a usage without `workflow`, Heptapod will trigger only 1 pipeline for a merge request and only another pipeline when the merge request is published.
It seems that `workflow` like this:
```
workflow:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH =~ /^branch\/.*/
```
does not work when pushing directly or make changeset public from command line.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/201Registry garbage collection eating multi-arch images2023-01-09T19:51:15ZGeorges RacinetRegistry garbage collection eating multi-arch imagesSeen with `http://registry.heptapod.net/heptapod/heptapod-runner/runner:0.6.0rc5` which is currently in use by the Clever Runner for the group Heptapod and about to be used for the whole instance.
This image was first pushed to registry...Seen with `http://registry.heptapod.net/heptapod/heptapod-runner/runner:0.6.0rc5` which is currently in use by the Clever Runner for the group Heptapod and about to be used for the whole instance.
This image was first pushed to registry.heptapod.net, and only then copied over to Docker Hub.
Today, I had to copy it back to from Docker Hub to registry.heptapod.net. Before that, it could not be pulled and the registry listing looked like this one in the [listing page](https://foss.heptapod.net/heptapod/heptapod-runner/container_registry/58):
![broken_registry_image](/uploads/16212b11960e3f88438877af173b48b5/broken_registry_image.png)
Trying to pull it would give rise to the following error:
```
failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to copy: httpReadSeeker: failed open: content at https://registry.heptapod.net/v2/heptapod/heptapod-runner/runner/manifests/sha256:9854f07eccd99e39803de397ce83a7320f0d4cdce032a3b7826a5b945c3d5f27 not found: not found
```https://foss.heptapod.net/heptapod/website/-/issues/24quick-start: How to purge old/obsolete topics?2023-01-08T15:04:04ZHartmut Goebelquick-start: How to purge old/obsolete topics?When a merge-request based on one of my topics gets merged or closed, the topic still exists in my local repository.
Same for other person's topics which get pulled into my local repo.
How can I get rid of old topics? Is there some “pru...When a merge-request based on one of my topics gets merged or closed, the topic still exists in my local repository.
Same for other person's topics which get pulled into my local repo.
How can I get rid of old topics? Is there some “prune/purge topics not existing on the server” functionality?https://foss.heptapod.net/heptapod/heptapod/-/issues/721Topic-branches of closed merge-requests are marked as “merged”2023-01-08T15:00:52ZHartmut GoebelTopic-branches of closed merge-requests are marked as “merged”When a merge-request gets closed, the respective topic-branch will be marked as “merged”.
Example:
- merge request which has been closed: https://foss.heptapod.net/tryton/tryton/-/merge_requests/63
![Info of this merge-request](/upl...When a merge-request gets closed, the respective topic-branch will be marked as “merged”.
Example:
- merge request which has been closed: https://foss.heptapod.net/tryton/tryton/-/merge_requests/63
![Info of this merge-request](/uploads/5d8f4f5c3539247247c174ac2463cde4/grafik.png)
- corresponding topic-branch: https://foss.heptapod.net/tryton/tryton/-/branches?state=all&sort=updated_desc&search=doc
![topic-branch in branch list](/uploads/41e1700be2f7749d7ac4848f9fd6a9b1/grafik.png)
Expected results:
Topic-branches of closed merge-requests are _not_ marked as “merged”, but as eg. “closed”.https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/41Mercurial: fail the job if pull failed2023-01-05T12:37:54ZGeorges RacinetMercurial: fail the job if pull failedThe job reported in foss.heptapod.net#199 has `abort: error: Try again`, but it carried on, which cannot be the good behaviour in any case.
Would that be worthy of a Runner system failure banner ?
Also, in that case, an automatic retry...The job reported in foss.heptapod.net#199 has `abort: error: Try again`, but it carried on, which cannot be the good behaviour in any case.
Would that be worthy of a Runner system failure banner ?
Also, in that case, an automatic retry would be appreciable. Not sure what gave this error, but the retry could be a nice addition to the future `chainsaw` extension.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/199CA certificate problem on runner2023-01-09T16:23:01ZCédric Kriercedric.krier@b2ck.comCA certificate problem on runnerThere seems to be a problem with the CA of some runners, see https://foss.heptapod.net/tryton/tryton/-/jobs/1465101There seems to be a problem with the CA of some runners, see https://foss.heptapod.net/tryton/tryton/-/jobs/1465101https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/38PAAS dispatching: end user logs2023-09-19T14:49:04ZGeorges RacinetPAAS dispatching: end user logsCurrently, when there is a provisioning problem (typically because there aren't enough resources left), we have ordinary server-side logs,
but nothing is accesible to the end user. This would be very interesting if we implement a retry l...Currently, when there is a provisioning problem (typically because there aren't enough resources left), we have ordinary server-side logs,
but nothing is accesible to the end user. This would be very interesting if we implement a retry logic, flavor capping (interesting with the
weighted quota system, to reduce the margin that is enforced to prevent overflowing).
The obvious thing to do would be to append to the job logs, as visible in the interface (job trace, in GitLab terminology). There is an API endpoint for this, unsurprisingly. Here's a successful example (pdb session within heptapod-tests):
```
(Pdb) job_url = test_project.heptapod.api_url + '/jobs/715'
(Pdb) trace_url = job_url + '/trace'
(Pdb) range = '0-4'
(Pdb) resp = requests.patch(trace_url, params=dict(token=job['token']), headers={'content-range': range}, data='foo\n')
(Pdb) resp.status_code
202
```
Unfortunately, if we put anything to the job trace before handing the job over to the PAAS Resource, it won't work: the trick is that the range has to be tailored exactly so that the first value is the offset of the append (hence exactly the previous length of the job trace), and the second value has to be the resulting full trace size. Hence if we write anything before starting `heptapod-runner` in the PAAS resource, it will break the job traces. Example:
```
(Pdb) range = '0-8'
(Pdb) resp = requests.patch(trace_url, params=dict(token=job['token']), headers={'content-range': range}, data='aaaanew\n')
(Pdb) resp.text
'{"error":"416 Range Not Satisfiable"}'
```
A solution could be to introduce the suitable offset in the job payload for `gitlab-runner exec-fetched-job` and have the counter start at the proper value in [newJobTrace](https://foss.heptapod.net/heptapod/heptapod-runner/-/blob/26e94cabeb3d232289763bd7644935d79df8437e/network/trace.go#L302). Perhaps we can use `sentTrace`, perhaps we need a separate offset. In all cases, this will impact all Runner operation, hence it must be treated with care and proper testing.
Perhaps also a more recent version of GitLab Runner provides the kind of facility we need.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/195Name resolution failure in some runner2023-01-04T18:08:16ZCédric Kriercedric.krier@b2ck.comName resolution failure in some runnerIt seems that some runner as DNS issue, we often have failure like https://foss.heptapod.net/tryton/tryton/-/jobs/1125484It seems that some runner as DNS issue, we often have failure like https://foss.heptapod.net/tryton/tryton/-/jobs/1125484https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/194SnowRunner Resource Tools project2022-12-25T20:26:56ZMightyEagle82SnowRunner Resource Tools project## Description of the projects to be hosted
- Project Name: SnowRunner Resource Tools
- Project URL: snowrunner-resource-tools
- Project Maintainers: MightyEagle82
- Project License: Creative Commons Attribution-NoDerivatives 4.0 Intern...## Description of the projects to be hosted
- Project Name: SnowRunner Resource Tools
- Project URL: snowrunner-resource-tools
- Project Maintainers: MightyEagle82
- Project License: Creative Commons Attribution-NoDerivatives 4.0 International
Please explain here how they meet our current criteria.
### Main criteria
- [x] Free / Open Source
- [x] main development repositories
- [x] you are an official maintainer of the project
- [x] agreement to acknowledge where appropriate the help provided by Octobus
and Clever Cloud by hosting on foss.heptapod.net
### Additional information
Knowing this can help in case there are constraints on resources and/or
administrators availability at the time of the request. It can also help us
suggest the right solution for the needs of the project.
- is the project officially released? In downstream distributions or
packaging systems? No
- estimation of the user base if possible, or number of downloads,
reverse dependencies… 1 user, no dependencies
- number of maintainers and regular contributors = Just me
- where is the project currently hosted? Are there issue trackers,
review systems, CI/CD to hook to or to replace somehow? Nowhere
### Hosting type
What would you like to have?
- [ ] a top-level group
- [ ] a project in an existing group
- [x] just a project in your personal namespace
## Post approval maintainer task list
See [README](README.md#maintainer-task-list-for-the-import-to-happen)
for explanations.
- [x] Reviewing [Heptapod workflow](https://heptapod.net/pages/faq.html#workflow)
## Post import maintainer task list
- [ ] Add the Octobus and Clever cloud acknowledgement at the selected place.
- [ ] Communicate about the migration to help others to aware of the options
(for example on mastodon, twitter, bitbucket thread, mailing list, …)https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/193Some runner does not recongnize the registry certificate2022-12-26T12:43:31ZCédric Kriercedric.krier@b2ck.comSome runner does not recongnize the registry certificateWe use CI to build and publish on the registry docker images. But I found that depending on the runner the push fails like this one https://foss.heptapod.net/tryton/tryton-docker-ci/-/jobs/1085646
Relaunch the job if it runs on another ...We use CI to build and publish on the registry docker images. But I found that depending on the runner the push fails like this one https://foss.heptapod.net/tryton/tryton-docker-ci/-/jobs/1085646
Relaunch the job if it runs on another runner succeed.https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/37Clever Runner: cache instance type query2022-12-25T15:01:16ZGeorges RacinetClever Runner: cache instance type queryCurrently, a `GET /products/instances` is issued each time there is a job to provision, to find about instance types and their versions.
It would be better to cache the results on the client side (hence in `heptapod-paas-runner`), but t...Currently, a `GET /products/instances` is issued each time there is a job to provision, to find about instance types and their versions.
It would be better to cache the results on the client side (hence in `heptapod-paas-runner`), but that would require a way to
invalidate. Typically, a change would be about the instance version and we need to use the version in that case.
For proper invalidation, we would hence need a way to detect that a provision failure is due to an outdated version, and only then invalidate the instance types and retry.https://foss.heptapod.net/heptapod/heptapod-import-roundup/-/issues/8Make confidential Issue types configurable2022-12-16T09:47:53ZGeorges RacinetMake confidential Issue types configurableThe Roundup types that imply confidentiality of the resulting GitLab issue are currently hardcoded for our first use-case.The Roundup types that imply confidentiality of the resulting GitLab issue are currently hardcoded for our first use-case.https://foss.heptapod.net/heptapod/heptapod-import-roundup/-/issues/7Make open Issue statuses configurable2022-12-16T09:44:18ZGeorges RacinetMake open Issue statuses configurableThe numeric id of the status values that imply that the resulting GitLab issue is open are currently hardcoded for our first use-case and should be configurable.The numeric id of the status values that imply that the resulting GitLab issue is open are currently hardcoded for our first use-case and should be configurable.https://foss.heptapod.net/heptapod/heptapod-api-import/-/issues/6Fallback for unmappable users2022-12-16T09:26:03ZGeorges RacinetFallback for unmappable usersIf user is unknown and cannot be created, we'd need to have a fallback system like what the GitLab standard import (adding original event information within Markdown).If user is unknown and cannot be created, we'd need to have a fallback system like what the GitLab standard import (adding original event information within Markdown).