Pull request #185
closedchecking out git repo with the dir format: don't symlink .git, and use a gitdir instead
Description
Hi,
Since https://github.com/jrl-umi3218/jrl-cmakemodules/pull/162 we need to have a working `git describe` command available when we build our packages.
In our current CI process, we use `make checkout MASTER_REPOSITORY="dir ${CI_PROJECT_DIR}"` in gitlab-CI with robotpkg, but this break git (it doesn't consider robotpkg/$category/$packages/work.$hostname/$distname/.git as a valid git directory, and use robotpkg/.git instead)
https://github.com/nim65s/robotpkg/commit/c2dad40d97fad411bac2668127f78f04115d61b2 fixes the issue by avoiding symlink of .git/ files, and instructs git to use the already available $extract_dir/.git instead
Here are tests:
- without this commit https://gepgitlab.laas.fr/gsaurel/pinocchio/-/jobs/8350
- with this commit https://gepgitlab.laas.fr/gsaurel/pinocchio/-/jobs/8349
This has the advantage of speeding up a bit the checkout, as the .git/ folder contains a lot of files, and a few limitations:
- git submodules are available, but without their .git configuration file
- if $extract_dir/.git is already a file containing the gitrepo instruction instead of a regular .git directory, it will not work
- if someone use a .git file or directory for something else than git, it will not work
For the records, I also tried to set the GIT_DIR environment variable to $CI_PROJECT_DIR/.git, and it worked for the `git describe`, but the build failed: https://gepgitlab.laas.fr/gsaurel/pinocchio/-/jobs/8346
In addition, I think that we should update the cmake submodule to avoid failing that much if `git describe` is not working, but on the other hand it yould be nice to have it when we generate the documentation, as it is also used by doxygen. Right now, http://projects.laas.fr/gepetto/doc/stack-of-tasks/pinocchio/master/doxygen-html/ displays a big `UNKNOWN` instead of a pretty version tag, which would be much more useful.
Updated by Anthony Mallet over 5 years ago
On Tuesday 30 Oct 2018, at 20:17, Guilhem Saurel wrote:
In our current CI process, we use `make checkout
MASTER_REPOSITORY="dir ${CI_PROJECT_DIR}"` in gitlab-CI with
robotpkg, but this break git (it doesn't consider
robotpkg/$category/$packages/work.$hostname/$distname/.git as a
valid git directory, and use robotpkg/.git instead)
Confirmed: surprisingly, git does not traverse symlinks in the .git
directory. While this makes sense for regular files of the repository,
this sounds like a limitation for the git-dir itself ...
stat("/some/where/.git", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
lstat("/some/where/.git/HEAD", {st_mode=S_IFLNK|0777, st_size=..., ...}) = 0
readlink("/some/where/.git/HEAD", "/else/where/.git/HEAD"..., 255) = ...
write(2, "fatal: not a git repository: '/s"...
https://github.com/nim65s/robotpkg/commit/c2dad40d97fad411bac2668127f78f04115d61b2
fixes the issue by avoiding symlink of .git/ files, and instructs
git to use the already available $extract_dir/.git instead
This does probably the trick, however it sounds like a specific and fragile
hack. The 'dir' method of checkout is not supposed to know about git
(or it should know about all the version control systems, and maybe
other weird things ...)
There is the 'git' method already, you could just use it as a
replacement. With CHECKOUT_VCS_OPTS="--depth 1" and the local
MASTER_REPOSITORY it should be more or less equivalement in terms of
disk space and I/O, as git does hardlinks in this situation.
Here are tests:
- without this commit
https://gepgitlab.laas.fr/gsaurel/pinocchio/-/jobs/8350
- with this commit
https://gepgitlab.laas.fr/gsaurel/pinocchio/-/jobs/8349
"error: 'UNKNOWN' was not declared in this scope" sounds like an error
specific to this software. Relying on git is probably a bad idea. For
instance, this will not work either with regular tarballs of the
project.
Relying on git to get the version is probably bad as well: the version
is typically something set in the build system, and should not change
until the next release (while git describe will change it for every
commit). Also, git describe output is not sortable, so you cannot do
version comparison, etc. BTW, this is why the 'checkout version' for a
robotpkg package is computed with the date instead of a
git/cvs/svn/whatever tag.
What you could do instead could be to add a "patchlevel" stuff to the
regular (static) VERSION.MAJOR.MINOR, computed by git describe if you
like, but defaulting to 0 if git does not work for any reason.
My 2c :)
Updated by Guilhem Saurel over 5 years ago
- Status changed from New to Closed
Using
make checkout MASTER_REPOSITORY="git ${CI_PROJECT_DIR}/.git"
works for me, so we can close this.