[ocaml-platform] Is there a Travis check expert in the room?

Discussion:

Gabriel Scherer

2015-11-20 12:17:23 UTC

Hi platform,

I would like to experiment with two new optional travis checks on the
ocaml/ocaml repository, to verify that pull requests come with some
tests and a Changes entry. I created an issue to track this need at:
http://caml.inria.fr/mantis/view.php?id=7052

This is not higher priority but if there was a Travis check expert
around here, a pull request to do this would be much appreciated.

(It may turn out that there are too many cases where these tests fail
for a good reason and we later decide to disable them. I do suspect
they would have a positive net effect, though.)

Thanks!

Thomas Gazagnaire

2015-11-20 12:44:25 UTC

Permalink

Hi Gabriel,

basically if you just have a bash script doing these checks (controlled by an environment variable), it's pretty easy to add them either as additional runs in Travis. See:

https://github.com/ocaml/opam-repository/blob/master/.travis.yml#L5-L25

I'm happy to help integrating these once you have the basic shell scripts running.

Thomas

Post by Gabriel Scherer
Hi platform,
I would like to experiment with two new optional travis checks on the
ocaml/ocaml repository, to verify that pull requests come with some
http://caml.inria.fr/mantis/view.php?id=7052
This is not higher priority but if there was a Travis check expert
around here, a pull request to do this would be much appreciated.
(It may turn out that there are too many cases where these tests fail
for a good reason and we later decide to disable them. I do suspect
they would have a positive net effect, though.)
Thanks!
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

Gabriel Scherer

2015-11-20 13:12:43 UTC

Permalink

I suspect my needs are significantly more complex than this example has.

- I want to run several checks (on the same os whatever) implemented
in several different bash scripts, one mandatory (the existing
testsuite/ and stuff) and some optional (the new tests). The results
should also be reported separately to the user.
- I need to inspect the diff of the pull request being tested to check
that Changes is modified (for one test) and at least a file in
testsuite/ (for the other test), the scripts cannot be just programmed
on the post-PR state of the index

I'm sure it is possible to read lots of documentation somewhere and
learn how to do all this by myself, but I was suspecting that there
may exist someone that already knows a bit of that and thus be much
faster at doing it (or someone willing to learn and do it) on this
list. I'm trying to *not do it myself*.

On Fri, Nov 20, 2015 at 1:44 PM, Thomas Gazagnaire

Post by Thomas Gazagnaire
Hi Gabriel,
https://github.com/ocaml/opam-repository/blob/master/.travis.yml#L5-L25
I'm happy to help integrating these once you have the basic shell scripts running.
Thomas

Thomas Gazagnaire

2015-11-20 13:48:32 UTC

Permalink

Post by Gabriel Scherer
I suspect my needs are significantly more complex than this example has.

not really...

To get the diff file of your PR:
https://github.com/ocaml/opam-repository/blob/master/.travis-ci.sh#L12

An example of a check that you can run on the PR file:
cat pullreq.diff | sed -E -n -e 's,\+\+\+ b/packages/[^/]*/([^/]*)/.*,\1,p' | sort -u > tobuild.txt
(https://github.com/ocaml/opam-repository/blob/master/.travis-ci.sh#L41)

And use the matrix thing to decide which test is optional or not.

Post by Gabriel Scherer
- I want to run several checks (on the same os whatever) implemented
in several different bash scripts, one mandatory (the existing
testsuite/ and stuff) and some optional (the new tests). The results
should also be reported separately to the user.
- I need to inspect the diff of the pull request being tested to check
that Changes is modified (for one test) and at least a file in
testsuite/ (for the other test), the scripts cannot be just programmed
on the post-PR state of the index
I'm sure it is possible to read lots of documentation somewhere and
learn how to do all this by myself, but I was suspecting that there
may exist someone that already knows a bit of that and thus be much
faster at doing it (or someone willing to learn and do it) on this
list. I'm trying to *not do it myself*.

The hard part is not the integration with the CI. If you manage to write simple bash scripts which parse the output of `git log` (or whatever diff file that you provide), plugging that into the CI is very easy and I can help doing it.

Thomas

Post by Gabriel Scherer
On Fri, Nov 20, 2015 at 1:44 PM, Thomas Gazagnaire

Anil Madhavapeddy

2015-11-20 14:50:58 UTC

Permalink

Yes, just small discrete shell scripts or OCaml programs that run over Git checkouts are the most useful.

The integration can then be assembled to be Travis-friendly (e.g. with code folding for sections) but also repurposed into other CIs in the future.

I do need to look at the clang scan-build patch as well, as that pushed an HTML report to a GitHub pages repo and was quite a convenient CI status. Would you like something similar for your PR checks?

Anil

Post by Thomas Gazagnaire

Post by Gabriel Scherer
I suspect my needs are significantly more complex than this example has.

not really...
https://github.com/ocaml/opam-repository/blob/master/.travis-ci.sh#L12
cat pullreq.diff | sed -E -n -e 's,\+\+\+ b/packages/[^/]*/([^/]*)/.*,\1,p' | sort -u > tobuild.txt
(https://github.com/ocaml/opam-repository/blob/master/.travis-ci.sh#L41)
And use the matrix thing to decide which test is optional or not.

Post by Gabriel Scherer
On Fri, Nov 20, 2015 at 1:44 PM, Thomas Gazagnaire

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

Gabriel Scherer

2015-11-21 09:01:03 UTC

Permalink

Thanks to both for your advice. My two scripts would be as follows:

- check that Changes has been modified by the proposed change

git diff $TRAVIS_COMMIT_RANGE --name-only --exit-code Changes > /dev/null \
&& echo fail || echo pass

- check that testsuite/ has been modified by the proposed change

git diff $TRAVIS_COMMIT_RANGE --name-only --exit-code testsuite >
/dev/null \
&& echo fail || echo pass

Post by Anil Madhavapeddy
Yes, just small discrete shell scripts or OCaml programs that run over Git checkouts are the most useful.
The integration can then be assembled to be Travis-friendly (e.g. with code folding for sections) but also repurposed into other CIs in the future.
I do need to look at the clang scan-build patch as well, as that pushed an HTML report to a GitHub pages repo and was quite a convenient CI status. Would you like something similar for your PR checks?
Anil

Post by Thomas Gazagnaire

Post by Gabriel Scherer
I suspect my needs are significantly more complex than this example has.

Post by Gabriel Scherer
On Fri, Nov 20, 2015 at 1:44 PM, Thomas Gazagnaire

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform