Split GitHub repository

Get help with compiling or installing the game, and discuss announcements of new official releases.

Moderator: Forum Moderators

Post Reply
techtonik
Posts: 30
Joined: February 28th, 2008, 7:57 pm

Split GitHub repository

Post by techtonik »

What upsets me is than the time needed to download Wesnoth source to see it is > 2 hours on 2 Mbit channel. `git clone` says 10%, but it already downloaded 300Mb. It would be nice to split repositories into `source` and `assets` to make it easier for people to use the full source code history with only the latest version of assets.

Maybe a subversion mirror will do the thing.
User avatar
Iris
Site Administrator
Posts: 6797
Joined: November 14th, 2006, 5:54 pm
Location: Chile
Contact:

Re: Split GitHub repository

Post by Iris »

You can actually use Subversion with GitHub if you are so inclined:
Alternatively, you can come on IRC and ask me for a link to a tar archive of a Git clone that you can download using wget -c or any other HTTP client with support for resuming interrupted downloads.
Author of the unofficial UtBS sequels Invasion from the Unknown and After the Storm.
techtonik
Posts: 30
Joined: February 28th, 2008, 7:57 pm

Re: Split GitHub repository

Post by techtonik »

Subversion support is cool. Never thought about it. Thanks.

It is still not very good to have binaries stored in Git. I don't know how big the repository is, but I won't be surprised if it is beyond some GB. I can understand the media in the same repository when the project can be executed from checkout right away, otherwise the media dependencies can be fetched during the build process.
User avatar
Iris
Site Administrator
Posts: 6797
Joined: November 14th, 2006, 5:54 pm
Location: Chile
Contact:

Re: Split GitHub repository

Post by Iris »

We had a long discussion about this and other issues before making a decision last year, and we all were aware of the approximate target size — especially those of us who had been using git-svn for years. The current organization is not likely to change any time soon.
Author of the unofficial UtBS sequels Invasion from the Unknown and After the Storm.
techtonik
Posts: 30
Joined: February 28th, 2008, 7:57 pm

Re: Split GitHub repository

Post by techtonik »

Do you plan on updating http://wiki.wesnoth.org/WesnothRepository with the info about repository size and workarounds? I find it a major showstopper for small contributions. I tried to clone it in the past several times and run out of free time.

I have little experience with Git, but a lot in Mercurial. Recently Facebook released their speed up for working with remote repositories - https://code.facebook.com/posts/2186788 ... -facebook/ - which is basically fetch on-demand. Does Git has something similar what is worth mentioning on this repository page along with git-svn method?

Current size of clone without checkout is ~1.8Gb, and about ~2.5Gb with checkout.
AI
Developer
Posts: 2396
Joined: January 31st, 2008, 8:38 pm

Re: Split GitHub repository

Post by AI »

Git does not have a resumeable cloning system (which is an open feature request) as of a few months ago, but as shadowm said, we host a tarball of a clone for this exact purpose. You have to get on IRC and ask us for the link though, as downloads of this size can fill up bandwidth quotas rather quickly.
Post Reply