Conversation
monsieuricon

K. Ryabitsev-Prime 🍁

We now publish tar'd up shallow git clones you can use with your CI. Please carefully read the safety caveats and use the script we provide to make sure you're not getting something malicious:

https://people.kernel.org/monsieuricon/using-shallow-git-tarballs-for-ci
2
22
21
pbarker@social.afront.org

Paul Barker

Reply to @monsieuricon

@monsieuricon this is great! Thanks for the write up as well. This is something we might want to look at for Yocto (we could probably integrate the shallow tarball fetch & validation into our bitbake-setup command as an option).

0
0
1
Logical_Error@fosstodon.org

LogicalErzor

Reply to @monsieuricon

@monsieuricon would it be better to follow this method instead?

eg.
- user issues git clone depth 1
- if it doesn’t exist in cache server, git goes thru normal path, otherwise serves the cache from https://www.kernel.org/pub/linux/kernel/shallow-tar/
- server can cleanup cache based on its own preferences (older than a month, space requirements, etc.)

1
0
0
monsieuricon

K. Ryabitsev-Prime 🍁

Reply to @Logical_Error@fosstodon.org
@Logical_Error we can't reliably prebuild a "git clone --depth N" pack that git can use, for several reasons. For one, we can't know what the "N" value is going to be (some users will use --depth 5/10/50 just depending on how they are feeling that day); for two, on quickly moving repos like torvalds/linux.git, this would require a pack rebuild after each repo update.
1
0
0
Logical_Error@fosstodon.org

LogicalErzor

Reply to @monsieuricon
Edited yesterday

@monsieuricon https://www.kernel.org/pub/linux/kernel/shallow-tar/ is when N == 1 tho right? i would assume it would make sense to only cache when N == 1 for `git clone` as well

also why prebuild instead of lazyloading (eg. cache only when a user requests)?

im just mainly curious why the kernel decided this path. i feel like transparently rerouting `git` to use the cache would be more effective than asking to use a separate mirror

1
0
0
monsieuricon

K. Ryabitsev-Prime 🍁

Reply to @Logical_Error@fosstodon.org
@Logical_Error I'm not disagreeing, but... how would you lazy-cache an arbitrary pack? blobcatthink
1
0
0
Logical_Error@fosstodon.org

LogicalErzor

Reply to @monsieuricon

@monsieuricon oh i have no idea… i was assuming that git had some sort of mechanism like that as it would help services like github/gitlab/codeberg etc

1
0
0
monsieuricon

K. Ryabitsev-Prime 🍁

Reply to @Logical_Error@fosstodon.org
@Logical_Error Right... nothing in the standard git tooling, I'm afraid. It's possible that github and others are using some kind of clever hacking to do this, but, to my knowledge, there is no way to make git cache out packs generated on the fly.
0
0
1
About social.kernel.org

Terms of service

  1. Please do not use this service in violation of the Linux Kernel Code of Conduct. Doing so will result in your account suspension with the referral of the matter to the CoC committee.

  2. "Repeating"/"boosting" someone else's status on this platform will be treated as endorsement and will fall under rule #1.

  3. You are encouraged to use this platform to promote your work on the Linux Kernel, but there is no restriction on permitted topics (with the exception of anything covered by #1 above).

  4. There is no requirement to post in English, but it should be considered the primary language of communication on this platform.

Privacy notice

The admins of this service have access to all posted statuses. They aren't looking, but if it's something they shouldn't know about, then you should not post it on this platform.

Please see the Linux Foundation Privacy Policy, which applies to this platform as well.

Getting your own account

If you would like an account on this instance, please check that the following applies to you:

  • You are listed in MAINTAINERS or CREDITS

  • OR: You have a kernel.org account or email address

  • OR: You have a long and established history of involvement with the Linux Kernel

If the above is true and you agree with the Terms of Service and Privacy Notice listed above, please use these instructions to request an account: