Repository navigation
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
Moved from beta4 to beta5. All bitbucket and github repos when step gitextractor starts - fails now.
attached stack trace -- stack trace: | github.com/apache/incubator-devlake/server/services.runPipeline | /app/server/services/pipeline_runner.go:79 | [...repeated from below...] Wraps: (2) Error running pipeline 261. Wraps: (3) attached stack trace -- stack trace: | github.com/apache/incubator-devlake/server/services.RunTasksStandalone | /app/server/services/task.go:217 | github.com/apache/incubator-devlake/server/services.(*pipelineRunner).runPipelineStandalone.func1 | /app/server/services/pipeline_runner.go:42 | github.com/apache/incubator-devlake/core/runner.runPipelineTasks | /app/core/runner/run_pipeline.go:90 | github.com/apache/incubator-devlake/core/runner.RunPipeline | /app/core/runner/run_pipeline.go:54 | github.com/apache/incubator-devlake/server/services.(*pipelineRunner).runPipelineStandalone | /app/server/services/pipeline_runner.go:38 | github.com/apache/incubator-devlake/server/services.runPipeline | /app/server/services/pipeline_runner.go:76 | github.com/apache/incubator-devlake/server/services.RunPipelineInQueue.func1 | /app/server/services/pipeline.go:347 | runtime.goexit | /usr/local/go/src/runtime/asm_amd64.s:1598 Wraps: (4) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | | /app/server/services/task.go:189 | Wraps: (2) Error running task 26893. | Wraps: (3) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/core/runner.RunPluginSubTasks | | /app/core/runner/run_task.go:329 | | github.com/apache/incubator-devlake/core/runner.RunPluginTask | | /app/core/runner/run_task.go:163 | | github.com/apache/incubator-devlake/core/runner.RunTask | | /app/core/runner/run_task.go:137 | | github.com/apache/incubator-devlake/server/services.runTaskStandalone | | /app/server/services/task_runner.go:113 | | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | | /app/server/services/task.go:187 | Wraps: (4) subtask Clone Git Repo ended unexpectedly | Wraps: (5) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/plugins/gitextractor/parser.(*GitcliCloner).execGitCloneCommand | | /app/plugins/gitextractor/parser/clone_gitcli.go:144 | | [...repeated from below...] | Wraps: (6) failed to fetch all branches from the remote server | Wraps: (7) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/plugins/gitextractor/parser.(*GitcliCloner).execCommand | | /app/plugins/gitextractor/parser/clone_gitcli.go:238 | | github.com/apache/incubator-devlake/plugins/gitextractor/parser.(*GitcliCloner).execGitCommandIn | | /app/plugins/gitextractor/parser/clone_gitcli.go:227 | | github.com/apache/incubator-devlake/plugins/gitextractor/parser.(*GitcliCloner).execGitCloneCommand | | /app/plugins/gitextractor/parser/clone_gitcli.go:143 | | github.com/apache/incubator-devlake/plugins/gitextractor/parser.(*GitcliCloner).CloneRepo | | /app/plugins/gitextractor/parser/clone_gitcli.go:92 | | github.com/apache/incubator-devlake/plugins/gitextractor/tasks.CloneGitRepo | | /app/plugins/gitextractor/tasks/repo_cloner.go:57 | | github.com/apache/incubator-devlake/core/runner.runSubtask | | /app/core/runner/run_task.go:405 | | github.com/apache/incubator-devlake/core/runner.RunPluginSubTasks | | /app/core/runner/run_task.go:327 | | github.com/apache/incubator-devlake/core/runner.RunPluginTask | | /app/core/runner/run_task.go:163 | | github.com/apache/incubator-devlake/core/runner.RunTask | | /app/core/runner/run_task.go:137 | | github.com/apache/incubator-devlake/server/services.runTaskStandalone | | /app/server/services/task_runner.go:113 | | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | | /app/server/services/task.go:187 | | runtime.goexit | | /usr/local/go/src/runtime/asm_amd64.s:1598 | Wraps: (8) git cmd [git fetch --depth=1 origin -c http.sslVerify=false] in /tmp/gitextractor2334407235 failed: error: unknown switch c' | | usage: git fetch [] [ [...]] | | or: git fetch [] | | or: git fetch --multiple [] [( | )...] | | or: git fetch --all [] | | | | -v, --verbose be more verbose | | -q, --quiet be more quiet | | --all fetch from all remotes | | --set-upstream set upstream for git pull/fetch | | -a, --append append to .git/FETCH_HEAD instead of overwriting | | --upload-pack path to upload pack on remote end | | -f, --force force overwrite of local reference | | -m, --multiple fetch from multiple remotes | | -t, --tags fetch all tags and associated objects | | -n do not fetch all tags (--no-tags) | | -j, --jobs number of submodules fetched in parallel | | -p, --prune prune remote-tracking branches no longer on remote | | -P, --prune-tags prune local tags no longer on remote and clobber changed tags | | --recurse-submodules[=] | | control recursive fetching of submodules | | --dry-run dry run | | --write-fetch-head write fetched references to the FETCH_HEAD file | | -k, --keep keep downloaded pack | | -u, --update-head-ok allow updating of HEAD ref | | --progress force progress reporting | | --depth deepen history of shallow clone | | --shallow-since | | deepen history of shallow repository based on time | | --shallow-exclude | | deepen history of shallow clone, excluding rev | | --deepen deepen history of shallow clone | | --unshallow convert to a complete repository | | --update-shallow accept refs that update .git/shallow | | --refmap specify fetch refmap | | -o, --server-option | | option to transmit | | -4, --ipv4 use IPv4 addresses only | | -6, --ipv6 use IPv6 addresses only | | --negotiation-tip | | report that we have only objects reachable from this object | | --filter object filtering | | --auto-maintenance run 'maintenance --auto' after fetching | | --auto-gc run 'maintenance --auto' after fetching | | --show-forced-updates | | check for forced-updates on all updated branches | | --write-commit-graph write the commit-graph after fetching | | --stdin accept refspecs from stdin | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *withstack.withStack (8) *errutil.leafError Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.leafError`
What do you expect to happen
Extract of git info should be successful.
How to reproduce
Move from beta4 to beta5
Anything else
No response
Version
v1.0.1-beta5@ebddfe3
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
dosubot commentedon Aug 6, 2024
Hey @Capone1983! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this issue together!
The issue you're encountering with the
gitextractorstep failing after moving from beta4 to beta5 is likely due to a permission problem with your tokens. The error message "error: unknown switchc" suggests that there might be an issue with the Git command being executed.To resolve this issue, you should:
Refer to the corresponding plugin's Config UI manuals for the supported authentication methods and required token permissions. For example, you can find the necessary information for configuring GitHub tokens here.
Additionally, you can refer to the Configuration and Blueprint Troubleshooting guide for more details on common error codes and their solutions. Specifically, the guide mentions that a 403 error is usually caused by the permission of your tokens, and it provides steps to ensure you have the correct permissions [1][2][3][4][5].
Also, ensure that you have
libgit2version 1.3.0 installed, as it is required to run and debug thegitextractorplugin on your local machine [6][7][8][9].To continue talking to Dosu, mention @dosu.
Capone1983 commentedon Aug 6, 2024
We should not have to change anything when moving to different versions.
Reverted back to beta4 and its working fine again.
Startrekzky commentedon Aug 6, 2024
@Capone1983 Got it. We'll look at it asap.
denodaeus commentedon Aug 6, 2024
Also encountering this issue on beta5:
Mtime="2024-08-05 14:10:04" level=error msg=" [pipeline service] [pipeline #12] run tasks failed\n\tcaused by: Error running task 81.\n\tWraps: (2) subtask Clone Git Repo ended unexpectedly\n\tWraps: (3) failed to fetch all branches from the remote server\n\tWraps: (4) git cmd [git fetch --depth=1 origin -c http.sslVerify=false] in /tmp/gitextractor3731976518 failed: error: unknown switchc'\n\t | usage: git fetch [] [ [...]]\n\t | or: git fetch [] \n\t | or: git fetch --multiple [] [( | )...]\n\t | or: git fetch --all []\n\t |\n\t | -v, --verbose be more verbose\n\t | -q, --quiet be more quiet\n\t | --all fetch from all remotes\n\t | --set-upstream set upstream for git pull/fetch\n\t | -a, --append append to .git/FETCH_HEAD instead of overwriting\n\t | --upload-pack path to upload pack on remote end\n\t | -f, --force force overwrite of local reference\n\t | -m, --multiple fetch from multiple remotes\n\t | -t, --tags fetch all tags and associated objects\n\t | -n do not fetch all tags (--no-tags)\n\t | -j, --jobs number of submodules fetched in parallel\n\t | -p, --prune prune remote-tracking branches no longer on remote\n\t | -P, --prune-tags prune local tags no longer on remote and clobber changed tags\n\t | --recurse-submodules[=]\n\t | control recursive fetching of submodules\n\t | --dry-run dry run\n\t | --write-fetch-head write fetched references to the FETCH_HEAD file\n\t | -k, --keep keep downloaded pack\n\t | -u, --update-head-ok allow updating of HEAD ref\n\t | --progress force progress reporting\n\t | --depth dee deepen history of shallow repository based on time\n\t | --shallow-exclude \n\t | deepen history of shallow clone, excluding rev\n\t | --deepen deepen history of shallow clone\n\t | --unshallow convert to a complete repository\n\t | --update-shallow accept refs that update .git/shallow\n\t | --refmap specify fetch refmap\n\t | -o, --server-option \n\t | option to transmit\n\t | -4, --ipv4 use IPv4 addresses only\n\t | -6, --ipv6 use IPv6 addresses only\n\t | --negotiation-tip \n\t | report that we have only objects reachable from this object\n\t | --filter object filtering\n\t | --auto-maintenance run 'maintenance --auto' after fetching\n\t | --auto-gc run 'maintenance --auto' after fetching\n\t | --show-forced-updates\n\t | check for forced-updates on all updated branches\n\t | --write-commit-graph write the commit-graph after fetching\n\t | --stdin accept refspecs from stdin\n\tError types: (1) *hintdetail.withDetail (2) *hintdetail.withDetail (3) *hintdetail.withDetail (4) *errors.errorString\n\t"time="2024-08-05 14:10:04" level=info msg=" [pipeline service] [pipeline #12] return error"`
I noticed that due to some self signed certs issues with my on-prem Bitbucket Server instance, the -c is being passed in with an argument after with sslVerify=false.
git -c expects a configuration string. I also launched the worker container to make sure the -c argument was available it it's version of git.d4x1 commentedon Aug 7, 2024
@denodaeus Can you run
git fetch --depth=1 origin -c http.sslVerify=falsesucessfully in your local repo?I think this cli should be
git -c http.sslVerify=false clone_or_fetch.Startrekzky commentedon Aug 7, 2024
@dosu Could you stop labeling GitHub issues with tags starting with
pr-type/?dosubot commentedon Aug 7, 2024
The
pr-type/labels are used to categorize pull requests based on their nature, such aspr-type/bug-fixorpr-type/feature-development[1].However, there is no information available about any automated workflows or bots that add these labels, nor are there specific guidelines or documentation on labeling GitHub issues in the repository [2][1].
If you need to stop labeling issues with
pr-type/tags, you may need to manually adjust your labeling practices or update any existing automation scripts that apply these labels.To continue talking to Dosu, mention @dosu.
devstein commentedon Aug 7, 2024
Hey @Startrekzky apologies for the confusion here. It looks like Dosu's issue and PR label settings weren't set up correctly. I just fixed it, so Dosu won't manage
pr-typetags or labels moving or other irrelevant issues labels moving forwarrd.If you have any other issues or question, feel free to ping me in our community slack
denodaeus commentedon Aug 7, 2024
Any idea when you'll have a beta6 with this change? beta4 has a bunch of broken dashboards with bitbucket server, and beta5 won't run for me because of the above
klesh commentedon Aug 22, 2024
Hi, All. PR #7941 should fix the ssl verify problem.