You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug][Module Name] Bug title Gitextractor plugin takes long time. #6138
I had searched in the issues and found no similar issues.
What happened
The gitextractor plugin in the pipeline takes 13+ hours to complete. and the scheduled pipeline will fail because the gitextractor task and GitHub repo task will fail, If I run it manually then it will be successful by taking 13+ hours.
it says "the process terminated unexpectedly" on both tasks.
What do you expect to happen
Expected the scheduled pipeline should be completed without errors and not take so much time
@pramodh-b
If you don't need some data, such as commit_files, you can skip it, which will save you a lot of time. In subsequent versions, we will continue to optimize the gitextractor module to reduce its time consumption as much as possible.
how to skip commit_files collector:
you should set SKIP_COMMIT_FILES=true in .env file or enviroment variable.
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
Activity
d4x1 commentedon Sep 22, 2023
@pramodh-b
How big and how many repos are you trying to clone?
pramodh-b commentedon Sep 22, 2023
Hi @d4x1 It's only 1 private repo and its size is 1.34GB
pramodh-b commentedon Sep 22, 2023
with the v0.19.0-beta1 version, when I run manually also the pipeline is getting failed.
abeizn commentedon Sep 28, 2023
@pramodh-b
If you don't need some data, such as commit_files, you can skip it, which will save you a lot of time. In subsequent versions, we will continue to optimize the gitextractor module to reduce its time consumption as much as possible.
how to skip commit_files collector:
you should set SKIP_COMMIT_FILES=true in .env file or enviroment variable.
github-actions commentedon Nov 28, 2023
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
Startrekzky commentedon Jan 19, 2024
This will be tracked and solved in #6853.