[ocaml-platform] Is it taking too long for OCaml software to become 4.03-compatible? Would release process changes help?

[Gabriel Scherer]

Dear platform,

TL;DR: I propose that we change the OCaml release process to remain in
"beta" stage while there remain OPAM packages that are broken by the new
releases and their version bounds have not been updated.

I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.

A large chunk of my personal hacking time following the release was spent
on updating Batteries, with a 4.03-compatible release happening on May 12,
so two weeks after the release -- a time I would still consider too long
for our downstream users waiting to try the 4.03 goodness but held back by
a Batteries dependency. I recently helped make atdgen compatible with 4.03
(it's a dependency of IOCaml for example), and just today found out that
Coq releases (or its CoqIDE part) did not work on 4.03 (with no upper-bound
in opam, it would just miscompile).

I find it frustrating because that means that there is a large time window
where we advertise (externally, to beginners, etc.) that the best OCaml
versions ever is 4.03.0, while running opam from this switch to install any
major software whatsoever feels like playing a rigged russian roulette. I
feel bad whenever I'm informed that my software (or anyone else's, in fact)
does not work by end-users that would rather have spent their day doing
something else than trying broken software.

Do you agree that this is a problem? What could we do to fix that?

When I started writing this email I intended to send it to Damien Doligez,
as the release manager of the compiler distribution, with the following
suggestion: maybe we could have a *much longer* beta-testing period for
OCaml versions that would be used not (only) for finding bugs in the new
version, but also to let important pieces of the ecosystem update to the
new version. Say, at least a month. We could have a fixed set of software
that we monitor, or in fact all of what's in OPAM, and delay the release
until they get updated/fixed or their OPAM version bounds get fixed. (Aha,
a Platform indeed!)

Does that sound like a reasonable idea to you? Should we monitor all OPAM,
or only a subset?

This suggestion is rooted in the idea that when we announce the release to
our end-users, stuff should just work -- or refuse to install itself. Maybe
this is an old idea of the SVN times and we're now in the era of
continuously-released, always-broken stuff?

Finally, there has been all this work on the "OPAM weather services",
continuous build of opam packages etc., plus the work that Damien is doing
around release time. It doesn't seem to have worked very well for 4.03. Who
is monitoring this stuff? Do we need to fix them to make them work better,
or to fix our process to use them better? To be honest, I'm never looking
at these services -- I just don't think of using them. Can I get an email
whenever they detect that my stuff is broken? Say I'm a new package
manager, are there guidelines somewhere on what the best practices are and
what I should do?

[Simon Cruanes]

Post by Gabriel Scherer
I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.
...
Do you agree that this is a problem? What could we do to fix that?

I agree, I wanted to update my packages after the release and had to
wait for oasis to work on 4.03; some other dependencies of projects of
mine also took some time. Even now, I just stumbled upon Coq 8.5 not
compiling, so I'd say the problem is real.

Post by Gabriel Scherer
When I started writing this email I intended to send it to Damien Doligez,
as the release manager of the compiler distribution, with the following
suggestion: maybe we could have a *much longer* beta-testing period for
OCaml versions that would be used not (only) for finding bugs in the new
version, but also to let important pieces of the ecosystem update to the
new version. Say, at least a month. We could have a fixed set of software
that we monitor, or in fact all of what's in OPAM, and delay the release
until they get updated/fixed or their OPAM version bounds get fixed. (Aha,
a Platform indeed!)

As usual, the issue is: what set of packages is monitored, and who
chooses?

Post by Gabriel Scherer
Does that sound like a reasonable idea to you? Should we monitor all OPAM,
or only a subset?

I think OWS should be used to see which packages are broken on the new
release, indeed. I would propose the following workflow:

- the beta-testing period, feature-frozen, starts (for some time)
- shortly, maintainers whose package doesn't compile (we know thanks to
OWS) are emailed to be informed of the problem
- if, at the end of the beta-testing period (just before the release) a
package that used to build on the previous release, but does not build
anymore, is modified by opam-repo maintainers so it gets an upper
bound on the OCaml version.

This means that, if no action is taken, a
package that does not build on 4.04 would have "ocaml-version < 4.04".
New releases of this package, that hopefully fix the build on 4.04,
would lift this barrier. It would also prevent opam from falling back
to old, incompatible versions of a package in case of conflict with
some other library.

New packages are checked for build on most versions of OCaml; I think
old packages should also be checked for compatibility with newer
versions of OCaml so that the appropriate bound check is added. I don't
add those bounds myself on my own packages (too tedious to do it on 15
old releases), so it should probably be automated.

Cheers!

[Sylvain Le Gall]

Post by Simon Cruanes

Post by Gabriel Scherer
I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.
...
Do you agree that this is a problem? What could we do to fix that?

I agree, I wanted to update my packages after the release and had to
wait for oasis to work on 4.03; some other dependencies of projects of
mine also took some time. Even now, I just stumbled upon Coq 8.5 not
compiling, so I'd say the problem is real.

Just to be fair to the OASIS release cycle:
- OCaml 4.03 was released 2016-04-26
- OASIS 0.4.6 for OCaml 4.03 was released 2016-04-29

This is only 3 days and this was mainly due to a transitive dependency on
Batteries for the tests. Considering OSS standards, this is not a long time
for a fix.

Post by Simon Cruanes

Post by Gabriel Scherer
When I started writing this email I intended to send it to Damien

Doligez,

Post by Gabriel Scherer
as the release manager of the compiler distribution, with the following
suggestion: maybe we could have a *much longer* beta-testing period for
OCaml versions that would be used not (only) for finding bugs in the new
version, but also to let important pieces of the ecosystem update to the
new version. Say, at least a month. We could have a fixed set of software
that we monitor, or in fact all of what's in OPAM, and delay the release
until they get updated/fixed or their OPAM version bounds get fixed.

(Aha,

Post by Gabriel Scherer
a Platform indeed!)

As usual, the issue is: what set of packages is monitored, and who
chooses?

Post by Gabriel Scherer
Does that sound like a reasonable idea to you? Should we monitor all

OPAM,

Post by Gabriel Scherer
or only a subset?

I think OWS should be used to see which packages are broken on the new
- the beta-testing period, feature-frozen, starts (for some time)
- shortly, maintainers whose package doesn't compile (we know thanks to
OWS) are emailed to be informed of the problem
- if, at the end of the beta-testing period (just before the release) a
package that used to build on the previous release, but does not build
anymore, is modified by opam-repo maintainers so it gets an upper
bound on the OCaml version.
This means that, if no action is taken, a
package that does not build on 4.04 would have "ocaml-version < 4.04".
New releases of this package, that hopefully fix the build on 4.04,
would lift this barrier. It would also prevent opam from falling back
to old, incompatible versions of a package in case of conflict with
some other library.
New packages are checked for build on most versions of OCaml; I think
old packages should also be checked for compatibility with newer
versions of OCaml so that the appropriate bound check is added. I don't
add those bounds myself on my own packages (too tedious to do it on 15
old releases), so it should probably be automated.
Cheers!
--
Simon Cruanes
http://weusepgp.info/
key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3 7D8D 4AC0 1D08 49AA 62B6
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Daniel Bünzli]

This is only 3 days and this was mainly due to a transitive dependency on Batteries for the tests. Considering OSS standards, this is not a long time for a fix.

This is good, however this should have happened during the betas, not after the release. If key parts of the infrastructure are not installable during the beta, programmers (myself included) very quickly shy away from trying to test the new release or even live on it for a go.

Best,

Daniel

[Fabrice Le Fessant]

Post by Sylvain Le Gall
This is only 3 days and this was mainly due to a transitive dependency

on Batteries for the tests. Considering OSS standards, this is not a long
time for a fix.
This is good, however this should have happened during the betas, not
after the release. If key parts of the infrastructure are not installable
during the beta, programmers (myself included) very quickly shy away from
trying to test the new release or even live on it for a go.

You can ask this kind of diligence from sub-contractors, not from
open-source contributors, especially when a package is maintained only by
one developer on his/her sparetime. Or if you think these components are
key to the infrastructure, why not move their development to
github.com/ocaml/ with a larger team of maintainers, as it was done for
OPAM ?

--Fabrice

[Daniel Bünzli]

You can ask this kind of diligence from sub-contractors, not from open-source contributors, especially when a package is maintained only by one developer on his/her sparetime.

I'm not asking anything. I'm only commenting on the fact that Sylvain said that the time response was not long. It wasn't indeed but release time is not the right time point to consider. The right time point to consider is the rc betas.

Or if you think these components are key to the infrastructure, why not move their development to github.com/ocaml/ (http://github.com/ocaml/) with a larger team of maintainers, as it was done for OPAM ?

I personally don't care as I'm not interested in oasis. But many other programmer seem to use it so it's a pre-requesite for them to be able to work in a given release and if this doesn't happen during the betas then we get less testing. Regarding oasis' development model I'll let users of oasis decide on what is best them.

Best,

Daniel

[Yotam Barnoy]

I've also been bitten by this issue with 4.03. Specifically, Merlin
took about 2 weeks post-release to be ready, making it much harder to
program in 4.03, and I had a dependency chain that needed OASIS to
build.

I don't think we can avoid having a specialized pre-release period for
getting the packages up and running. I noticed many of these breakages
before 4.03 was released, but package authors aren't responsive to a
pre-release breakage.

It's important to prioritize though -- those packages that have the
most reverse-dependencies and the ones that are most popular, should
be the highest priorities to fix. OPAM has this info, and just as the
https://opam.ocaml.org/packages/index-popularity.html page shows the
most popular packages, I think it should pull their build-state for
the latest OCAML release version and highlight them if they're broken.
Speaking of which, it would be nice if the opam webpage also had a
'sort by reverse dependencies' list.

One schedule that perhaps could work would be to have a
pre-platform-release month. Within this month, package authors would
have the first 2 weeks to fix their packages. After these 2 weeks,
there would be a rallying cry to fix packages sorted by priority
(highest reverse-dependencies & highest popularity), and the entire
community would be mobilized to help in the effort. Once a
"sufficient" number of the highest priority packages are fixed, the
full release can take place, but the work can be ongoing.

-Yotam

On Thu, Jun 30, 2016 at 9:17 AM, Daniel Bünzli

Post by Daniel Bünzli

You can ask this kind of diligence from sub-contractors, not from open-source contributors, especially when a package is maintained only by one developer on his/her sparetime.

I'm not asking anything. I'm only commenting on the fact that Sylvain said that the time response was not long. It wasn't indeed but release time is not the right time point to consider. The right time point to consider is the rc betas.

Or if you think these components are key to the infrastructure, why not move their development to github.com/ocaml/ (http://github.com/ocaml/) with a larger team of maintainers, as it was done for OPAM ?

I personally don't care as I'm not interested in oasis. But many other programmer seem to use it so it's a pre-requesite for them to be able to work in a given release and if this doesn't happen during the betas then we get less testing. Regarding oasis' development model I'll let users of oasis decide on what is best them.
Best,
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Gabriel Scherer]

Let's clarify that I am not trying to blame maintainers of packages for the
issues that I highlighted. I believe that we are all doing our best given
the current processes and expectations, and I would like to fix the process
to improve the results. (In particular, I think that the beta period for
4.03 was insufficient for maintainers to update their packages.)

Yotam: I don't think a clear separation of responsibility (package authors
vs. "the entire community") is possible. The free software that powers the
OCaml ecosystem should be maintained collectively at all times. It is
particularly true in the context of adapting to a new OCaml version that
changed some things; it is often the case that the package authors are not
the most qualified to understand the required changes -- they know their
codebase well, but don't necessarily follow OCaml evolutions closely.
Early-adopters and enthusiasts, such as you and I, have a large role to
play in helping the ecosystem evolve gracefully by sending patches to fix
the bugs.

Fabrice: again, it's not about who maintains the software (and I don't
think that the ocaml/ umbrella automatically guarantees more maintenance).
External contributors can easily fix software to work on trunk or new OCaml
versions -- I've done it for atdgen and Coqide and it was not much work.

Fabrice: Indeed, I think that opam-builder is a great tool to put at the
forefront of such an effort. The current version

http://opam.ocamlpro.com/builder/html/report-last.html

monitors the state of the OPAM repository for released OCaml versions.
Everything that is orange (except maybe for the "lint" column) is a bug
that we must collectively fix.

I will try to experiment more with opam-builder (understand its error
explanations better and try to fix a part of it to see what the experiment
is) before suggesting any specific process.

Post by Yotam Barnoy
I've also been bitten by this issue with 4.03. Specifically, Merlin
took about 2 weeks post-release to be ready, making it much harder to
program in 4.03, and I had a dependency chain that needed OASIS to
build.
I don't think we can avoid having a specialized pre-release period for
getting the packages up and running. I noticed many of these breakages
before 4.03 was released, but package authors aren't responsive to a
pre-release breakage.
It's important to prioritize though -- those packages that have the
most reverse-dependencies and the ones that are most popular, should
be the highest priorities to fix. OPAM has this info, and just as the
https://opam.ocaml.org/packages/index-popularity.html page shows the
most popular packages, I think it should pull their build-state for
the latest OCAML release version and highlight them if they're broken.
Speaking of which, it would be nice if the opam webpage also had a
'sort by reverse dependencies' list.
One schedule that perhaps could work would be to have a
pre-platform-release month. Within this month, package authors would
have the first 2 weeks to fix their packages. After these 2 weeks,
there would be a rallying cry to fix packages sorted by priority
(highest reverse-dependencies & highest popularity), and the entire
community would be mobilized to help in the effort. Once a
"sufficient" number of the highest priority packages are fixed, the
full release can take place, but the work can be ongoing.
-Yotam
On Thu, Jun 30, 2016 at 9:17 AM, Daniel BÃŒnzli

Post by Daniel Bünzli

Post by Fabrice Le Fessant
You can ask this kind of diligence from sub-contractors, not from

open-source contributors, especially when a package is maintained only by
one developer on his/her sparetime.

Post by Daniel Bünzli
I'm not asking anything. I'm only commenting on the fact that Sylvain

said that the time response was not long. It wasn't indeed but release time
is not the right time point to consider. The right time point to consider
is the rc betas.

Post by Daniel Bünzli

Post by Fabrice Le Fessant
Or if you think these components are key to the infrastructure, why not

move their development to github.com/ocaml/ (http://github.com/ocaml/)
with a larger team of maintainers, as it was done for OPAM ?

Post by Daniel Bünzli
I personally don't care as I'm not interested in oasis. But many other

programmer seem to use it so it's a pre-requesite for them to be able to
work in a given release and if this doesn't happen during the betas then we
get less testing. Regarding oasis' development model I'll let users of
oasis decide on what is best them.

Post by Daniel Bünzli
Best,
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Yotam Barnoy]

Just to clarify, I wasn't really separating between package authors
and the community, as much as delineating stages when specific aspects
are targeted. I know very well how hard it is to maintain one's
projects after one has released them and the 'passion' has died down.
Finding time and energy to do this promptly is often hard. My
suggestion is a process that tries to make the most of limited
resources: after the OCaml release is made, which in itself is a very
hectic and demanding process, the community turns to fixing those
packages which are most critical, and people who would normally be
spending some of their time on other things (such as compiler
features) could instead try to help the package authors of the most
critical broken packages to update their code. A rallying cry as part
of a formal process towards the places where most attention is needed
could be better than just letting things work themselves out on their
own. People who do not even use a specific package (and therefore
don't care about it directly) would be told about contributions they
could make to the ocaml platform as a whole, and end up updating that
package.

That is all.

On Thu, Jun 30, 2016 at 10:47 AM, Gabriel Scherer

Post by Gabriel Scherer
Let's clarify that I am not trying to blame maintainers of packages for the
issues that I highlighted. I believe that we are all doing our best given
the current processes and expectations, and I would like to fix the process
to improve the results. (In particular, I think that the beta period for
4.03 was insufficient for maintainers to update their packages.)
Yotam: I don't think a clear separation of responsibility (package authors
vs. "the entire community") is possible. The free software that powers the
OCaml ecosystem should be maintained collectively at all times. It is
particularly true in the context of adapting to a new OCaml version that
changed some things; it is often the case that the package authors are not
the most qualified to understand the required changes -- they know their
codebase well, but don't necessarily follow OCaml evolutions closely.
Early-adopters and enthusiasts, such as you and I, have a large role to play
in helping the ecosystem evolve gracefully by sending patches to fix the
bugs.
Fabrice: again, it's not about who maintains the software (and I don't think
that the ocaml/ umbrella automatically guarantees more maintenance).
External contributors can easily fix software to work on trunk or new OCaml
versions -- I've done it for atdgen and Coqide and it was not much work.
Fabrice: Indeed, I think that opam-builder is a great tool to put at the
forefront of such an effort. The current version
http://opam.ocamlpro.com/builder/html/report-last.html
monitors the state of the OPAM repository for released OCaml versions.
Everything that is orange (except maybe for the "lint" column) is a bug that
we must collectively fix.
I will try to experiment more with opam-builder (understand its error
explanations better and try to fix a part of it to see what the experiment
is) before suggesting any specific process.

Post by Yotam Barnoy
I've also been bitten by this issue with 4.03. Specifically, Merlin
took about 2 weeks post-release to be ready, making it much harder to
program in 4.03, and I had a dependency chain that needed OASIS to
build.
I don't think we can avoid having a specialized pre-release period for
getting the packages up and running. I noticed many of these breakages
before 4.03 was released, but package authors aren't responsive to a
pre-release breakage.
It's important to prioritize though -- those packages that have the
most reverse-dependencies and the ones that are most popular, should
be the highest priorities to fix. OPAM has this info, and just as the
https://opam.ocaml.org/packages/index-popularity.html page shows the
most popular packages, I think it should pull their build-state for
the latest OCAML release version and highlight them if they're broken.
Speaking of which, it would be nice if the opam webpage also had a
'sort by reverse dependencies' list.
One schedule that perhaps could work would be to have a
pre-platform-release month. Within this month, package authors would
have the first 2 weeks to fix their packages. After these 2 weeks,
there would be a rallying cry to fix packages sorted by priority
(highest reverse-dependencies & highest popularity), and the entire
community would be mobilized to help in the effort. Once a
"sufficient" number of the highest priority packages are fixed, the
full release can take place, but the work can be ongoing.
-Yotam
On Thu, Jun 30, 2016 at 9:17 AM, Daniel Bünzli

Post by Daniel Bünzli

Post by Fabrice Le Fessant
You can ask this kind of diligence from sub-contractors, not from
open-source contributors, especially when a package is maintained only by
one developer on his/her sparetime.

I'm not asking anything. I'm only commenting on the fact that Sylvain
said that the time response was not long. It wasn't indeed but release time
is not the right time point to consider. The right time point to consider is
the rc betas.

Post by Fabrice Le Fessant
Or if you think these components are key to the infrastructure, why not
move their development to github.com/ocaml/ (http://github.com/ocaml/) with
a larger team of maintainers, as it was done for OPAM ?

I personally don't care as I'm not interested in oasis. But many other
programmer seem to use it so it's a pre-requesite for them to be able to
work in a given release and if this doesn't happen during the betas then we
get less testing. Regarding oasis' development model I'll let users of oasis
decide on what is best them.
Best,
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Fabrice Le Fessant]

The reason why OPAM-builder cannot monitor trunk is by design: to achieve
almost real time on a 4-core computer for 5 versions of OCaml, opam-builder
makes an extensive use of caching binary installations of dependencies.
This cannot be done on a moving target like trunk, where you would have to
recompile all packages (all versions, sometimes with different depends
because of depopts) at every commit in the trunk, after clearing the cache.
That's the reason why I only do it for official beta releases, and
opam-builder takes two or three days to compile the repo.

Post by Gabriel Scherer
Let's clarify that I am not trying to blame maintainers of packages for
the issues that I highlighted. I believe that we are all doing our best
given the current processes and expectations, and I would like to fix the
process to improve the results. (In particular, I think that the beta
period for 4.03 was insufficient for maintainers to update their packages.)
Yotam: I don't think a clear separation of responsibility (package authors
vs. "the entire community") is possible. The free software that powers the
OCaml ecosystem should be maintained collectively at all times. It is
particularly true in the context of adapting to a new OCaml version that
changed some things; it is often the case that the package authors are not
the most qualified to understand the required changes -- they know their
codebase well, but don't necessarily follow OCaml evolutions closely.
Early-adopters and enthusiasts, such as you and I, have a large role to
play in helping the ecosystem evolve gracefully by sending patches to fix
the bugs.
Fabrice: again, it's not about who maintains the software (and I don't
think that the ocaml/ umbrella automatically guarantees more maintenance).
External contributors can easily fix software to work on trunk or new OCaml
versions -- I've done it for atdgen and Coqide and it was not much work.
Fabrice: Indeed, I think that opam-builder is a great tool to put at the
forefront of such an effort. The current version
http://opam.ocamlpro.com/builder/html/report-last.html
monitors the state of the OPAM repository for released OCaml versions.
Everything that is orange (except maybe for the "lint" column) is a bug
that we must collectively fix.
I will try to experiment more with opam-builder (understand its error
explanations better and try to fix a part of it to see what the experiment
is) before suggesting any specific process.

Post by Yotam Barnoy
I've also been bitten by this issue with 4.03. Specifically, Merlin
took about 2 weeks post-release to be ready, making it much harder to
program in 4.03, and I had a dependency chain that needed OASIS to
build.
I don't think we can avoid having a specialized pre-release period for
getting the packages up and running. I noticed many of these breakages
before 4.03 was released, but package authors aren't responsive to a
pre-release breakage.
It's important to prioritize though -- those packages that have the
most reverse-dependencies and the ones that are most popular, should
be the highest priorities to fix. OPAM has this info, and just as the
https://opam.ocaml.org/packages/index-popularity.html page shows the
most popular packages, I think it should pull their build-state for
the latest OCAML release version and highlight them if they're broken.
Speaking of which, it would be nice if the opam webpage also had a
'sort by reverse dependencies' list.
One schedule that perhaps could work would be to have a
pre-platform-release month. Within this month, package authors would
have the first 2 weeks to fix their packages. After these 2 weeks,
there would be a rallying cry to fix packages sorted by priority
(highest reverse-dependencies & highest popularity), and the entire
community would be mobilized to help in the effort. Once a
"sufficient" number of the highest priority packages are fixed, the
full release can take place, but the work can be ongoing.
-Yotam
On Thu, Jun 30, 2016 at 9:17 AM, Daniel BÃŒnzli

Post by Daniel Bünzli

Post by Fabrice Le Fessant
You can ask this kind of diligence from sub-contractors, not from

open-source contributors, especially when a package is maintained only by
one developer on his/her sparetime.

Post by Daniel Bünzli
I'm not asking anything. I'm only commenting on the fact that Sylvain

said that the time response was not long. It wasn't indeed but release time
is not the right time point to consider. The right time point to consider
is the rc betas.

Post by Daniel Bünzli

Post by Fabrice Le Fessant
Or if you think these components are key to the infrastructure, why

not move their development to github.com/ocaml/ (http://github.com/ocaml/)
with a larger team of maintainers, as it was done for OPAM ?

Post by Daniel Bünzli
I personally don't care as I'm not interested in oasis. But many other

programmer seem to use it so it's a pre-requesite for them to be able to
work in a given release and if this doesn't happen during the betas then we
get less testing. Regarding oasis' development model I'll let users of
oasis decide on what is best them.

Post by Daniel Bünzli
Best,
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Gabriel Scherer]

Fabrice: indeed.

I thought of proposing a process for using opam-builder against trunk (eg.
swipe the state every two weeks and update the compiler used to the HEAD of
trunk at this moment only; the incremental capabilities are still useful to
monitor the effect of proposed fixes in opam packages), but I think that
for trunk fixing there are more delicate questions that arise (than at the
beta stage). For example, deciding when the trunk state is "good" for
testing, and deciding what to do with breakage report, as reporting them to
maintainers "too soon" is not always the right choice -- we may want to
revert the change in trunk instead, or improve its compatibility status.

I have a clear, simple proposal for the end of the release process: remain
in beta stage as long as there are OPAM packages that are considered
installable but fail to build with the beta. (Provided we have tools to
automate the addition of ocaml version bounds, the release manager would
have the avaibility to declare it done and simply mark all remaining
packages un-installable whenever they want to get the release out.)

As a first step, I would get all installable OPAM packages to build on
4.03.0, and I will try to push fixes and bounds, and motivate people to
participate to that effect in the following weeks, using the opam-builder
reports.

I'm interested in working to get more software to build on trunk, but I
don't have a clear proposal or process in mind yet, so I won't make any
specific proposal for now.

P.S.: I'm a bit worried by the "uses a fork of OPAM" part of opam-builder,
isn't there a risk of increased divergence over time and eventual
unusability? Could we integrate the salient features in opam proper?

On Thu, Jun 30, 2016 at 11:09 AM, Fabrice Le Fessant <

Post by Fabrice Le Fessant
The reason why OPAM-builder cannot monitor trunk is by design: to achieve
almost real time on a 4-core computer for 5 versions of OCaml, opam-builder
makes an extensive use of caching binary installations of dependencies.
This cannot be done on a moving target like trunk, where you would have to
recompile all packages (all versions, sometimes with different depends
because of depopts) at every commit in the trunk, after clearing the cache.
That's the reason why I only do it for official beta releases, and
opam-builder takes two or three days to compile the repo.

Post by Gabriel Scherer
Let's clarify that I am not trying to blame maintainers of packages for
the issues that I highlighted. I believe that we are all doing our best
given the current processes and expectations, and I would like to fix the
process to improve the results. (In particular, I think that the beta
period for 4.03 was insufficient for maintainers to update their packages.)
Yotam: I don't think a clear separation of responsibility (package
authors vs. "the entire community") is possible. The free software that
powers the OCaml ecosystem should be maintained collectively at all times.
It is particularly true in the context of adapting to a new OCaml version
that changed some things; it is often the case that the package authors are
not the most qualified to understand the required changes -- they know
their codebase well, but don't necessarily follow OCaml evolutions closely.
Early-adopters and enthusiasts, such as you and I, have a large role to
play in helping the ecosystem evolve gracefully by sending patches to fix
the bugs.
Fabrice: again, it's not about who maintains the software (and I don't
think that the ocaml/ umbrella automatically guarantees more maintenance).
External contributors can easily fix software to work on trunk or new OCaml
versions -- I've done it for atdgen and Coqide and it was not much work.
Fabrice: Indeed, I think that opam-builder is a great tool to put at the
forefront of such an effort. The current version
http://opam.ocamlpro.com/builder/html/report-last.html
monitors the state of the OPAM repository for released OCaml versions.
Everything that is orange (except maybe for the "lint" column) is a bug
that we must collectively fix.
I will try to experiment more with opam-builder (understand its error
explanations better and try to fix a part of it to see what the experiment
is) before suggesting any specific process.

Post by Yotam Barnoy
I've also been bitten by this issue with 4.03. Specifically, Merlin
took about 2 weeks post-release to be ready, making it much harder to
program in 4.03, and I had a dependency chain that needed OASIS to
build.
I don't think we can avoid having a specialized pre-release period for
getting the packages up and running. I noticed many of these breakages
before 4.03 was released, but package authors aren't responsive to a
pre-release breakage.
It's important to prioritize though -- those packages that have the
most reverse-dependencies and the ones that are most popular, should
be the highest priorities to fix. OPAM has this info, and just as the
https://opam.ocaml.org/packages/index-popularity.html page shows the
most popular packages, I think it should pull their build-state for
the latest OCAML release version and highlight them if they're broken.
Speaking of which, it would be nice if the opam webpage also had a
'sort by reverse dependencies' list.
One schedule that perhaps could work would be to have a
pre-platform-release month. Within this month, package authors would
have the first 2 weeks to fix their packages. After these 2 weeks,
there would be a rallying cry to fix packages sorted by priority
(highest reverse-dependencies & highest popularity), and the entire
community would be mobilized to help in the effort. Once a
"sufficient" number of the highest priority packages are fixed, the
full release can take place, but the work can be ongoing.
-Yotam
On Thu, Jun 30, 2016 at 9:17 AM, Daniel BÃŒnzli

Post by Daniel Bünzli

Post by Fabrice Le Fessant
You can ask this kind of diligence from sub-contractors, not from

open-source contributors, especially when a package is maintained only by
one developer on his/her sparetime.

Post by Daniel Bünzli
I'm not asking anything. I'm only commenting on the fact that Sylvain

said that the time response was not long. It wasn't indeed but release time
is not the right time point to consider. The right time point to consider
is the rc betas.

Post by Daniel Bünzli

Post by Fabrice Le Fessant
Or if you think these components are key to the infrastructure, why

not move their development to github.com/ocaml/ (
http://github.com/ocaml/) with a larger team of maintainers, as it was
done for OPAM ?

Post by Daniel Bünzli
I personally don't care as I'm not interested in oasis. But many other

programmer seem to use it so it's a pre-requesite for them to be able to
work in a given release and if this doesn't happen during the betas then we
get less testing. Regarding oasis' development model I'll let users of
oasis decide on what is best them.

Post by Daniel Bünzli
Best,
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Xavier Leroy]

Dear platform fellows,

Many good points have been made already, so let me just add a few
remarks.

- I have the impression that OCaml 4.03 introduces more breaking
changes than 4.02 did. That could explain why OPAM packages are
taking longer to be upgraded to 4.03. I don't know what it means
for the future: should the OCaml core team be more prudent with
breaking changes? should the packaging community get ready to
handle even more breakage?

- As an upstream author of a few libraries, it's not obvious to me
what the best / recommended way is to communicate with the packagers
of "my" software, e.g. being informed of problem reports with the
corresponding packages, or informing them of new versions of "my"
soft.

- OPAM-wide continuous integration is great, of course. Concerning

Post by Gabriel Scherer
On Thu, Jun 30, 2016 at 11:09 AM, Fabrice Le Fessant
The reason why OPAM-builder cannot monitor trunk is by design: to
achieve almost real time on a 4-core computer for 5 versions of OCaml,
opam-builder makes an extensive use of caching binary installations of
dependencies. This cannot be done on a moving target like trunk, where
you would have to recompile all packages (all versions, sometimes with
different depends because of depopts) at every commit in the trunk,
after clearing the cache. That's the reason why I only do it for
official beta releases, and opam-builder takes two or three days to
compile the repo.

What would it take to get, say, one full build every day? I think we
could find some funds for that, e.g. from the Caml consortium. After
all, a cool 12 cores / 24 threads blade server costs only 2.5 kEur or
so.

- Xavier Leroy

[Fabrice Le Fessant]

Post by Xavier Leroy
What would it take to get, say, one full build every day? I think we
could find some funds for that, e.g. from the Caml consortium. After
all, a cool 12 cores / 24 threads blade server costs only 2.5 kEur or
so.

I forgot to say that, actually, the multi-core arch is only used for
parallel compilation of multiple OCaml versions (i.e. each switch is
compiled in parallel with other switches), but within a switch, parallel
build of packages is disabled, because we cache modifications to the
installation (even with the new version of OPAM, that builds in parallel
and install sequentially, we don't have the guarrantee that an incorrect
package will not modify the installation during its build, i.e. for example
`coq` that creates a symlink in $bindir during its build...).

I might have over-estimated the time it takes to rebuild completely a
switch, I think it is lower with 4.03.0, because many packages are not
available for it, and probably the same for trunk. It might actually
already be under one day.

--Fabrice

[Alain Frisch]

Post by Fabrice Le Fessant
The reason why OPAM-builder cannot monitor trunk is by design: to
achieve almost real time on a 4-core computer for 5 versions of OCaml,
opam-builder makes an extensive use of caching binary installations of
dependencies. This cannot be done on a moving target like trunk, where
you would have to recompile all packages (all versions, sometimes with
different depends because of depopts) at every commit in the trunk,
after clearing the cache. That's the reason why I only do it for
official beta releases, and opam-builder takes two or three days to
compile the repo.

Testing only the latest version of each package would already be very
useful. As a matter of fact, it could even detect earlier some problems
by restricting the universe only to latest version of all packages (see
below).

Moreover, it could be interesting to define a subset of "strategic"
packages that we should really be up-to-date on OCaml release dates.
Wasn't it part of the "platform" project to define such a curated list
of packages?

Do you have a description of OPAM-builder current strategy? I found out
yesterday that a simple "opam install ppx_type_conv" would fail on a
clean 4.03 switch, because the latest version of that package compatible
with 4.03 depends on ppx_deriving with no upper bound constraint, and
OPAM chooses to install the latest version of ppx_deriving which turns
out to be incompatible with that version of ppx_type_conv. But
OPAM-builder did not report any error for that version of ppx_type_conv:
it picked an older version of ppx_deriving (which works fine). I'm
wondering if this is because of a different solver configuration, or a
deliberate (or not) consequence of OPAM-builder strategy.


Alain

[Fabrice Le Fessant]

Post by Alain Frisch
Testing only the latest version of each package would already be very
useful. As a matter of fact, it could even detect earlier some problems
by restricting the universe only to latest version of all packages (see
below).

Actually, one of my plans when creating opam-builder was to output a list
of versions that would be compatible for a small set of packages (the
platform), and create an opam-repository with only them, i.e. it would be
the equivalent of Debian stable while the standard repository would be
Debian unstable. Clearly, we currently under-use the information that is
available in opam-builder.

Post by Alain Frisch
Moreover, it could be interesting to define a subset of "strategic"
packages that we should really be up-to-date on OCaml release dates.
Wasn't it part of the "platform" project to define such a curated list
of packages?

About that part of the discussion, I am really against any additional
constraint that would delay a release of OCaml. We waited for years before
deciding to have a regular release schedule (every 6 months), and now,
everybody tries to delay releases for various reasons. So, yes to more
monitoring of the compatibility of platform packages with trunk and
beta-releases, but no to delaying the release because of platform packages
being late to meet compatibility.

Post by Alain Frisch
Do you have a description of OPAM-builder current strategy?

Everything is on Github:
Opam-Builder: http://github.com/OCamlPro/opam-builder
OPAM fork: https://github.com/lefessan/opam/tree/2016-03-02-opam-builder

For the strategy, it is the same as OPAM's standard strategy, i.e. it tries
to match your query while minimizing the number of installed packages.
That's the reason why it prefers `ppx_deriving.3.3` to `ppx_deriving.4.0`
(that have an additional requirement to `result`). As a side note, as there
are more and more meta packages (i.e. whose build/installation cost is
null), we should clearly change that strategy to (1) add an installation
cost in every package (`1` by default, `0` for meta-packages) and (2) try
to minimize installation cost instead of number of packages.

--Fabrice

[Alain Frisch]

Post by Fabrice Le Fessant
About that part of the discussion, I am really against any additional
constraint that would delay a release of OCaml. We waited for years
before deciding to have a regular release schedule (every 6 months), and
now, everybody tries to delay releases for various reasons. So, yes to
more monitoring of the compatibility of platform packages with trunk and
beta-releases, but no to delaying the release because of platform
packages being late to meet compatibility.

I fully agree. The responsibility of the core team is to ensure the
quality of the core distribution and to provide some clear and reliable
information about the release cycle. Each package maintainer is then
responsible for ensuring their package work correctly, in a timely
manner. Then, if some groups want to create more stable subsets (your
idea of a stable opam repository, or the "platform"), they would be
responsible for managing the life-cycle and the stability of their subsets.

Post by Fabrice Le Fessant
Do you have a description of OPAM-builder current strategy?
For the strategy, it is the same as OPAM's standard strategy, i.e. it
tries to match your query while minimizing the number of installed
packages. That's the reason why it prefers `ppx_deriving.3.3` to
`ppx_deriving.4.0` (that have an additional requirement to `result`).

Weird. I tried again from an empty 4.03 switch and I get:

$ opam install ppx_type_conv

The following actions will be performed:
∗ install ocamlbuild 0.9.2 [required by ppx_type_conv]
∗ install ocamlfind 1.6.2 [required by ppx_type_conv]
∗ install result 1.2 [required by ppx_deriving]
∗ install ppx_tools 5.0+4.03.0 [required by ppx_type_conv]
∗ install cppo 1.3.2 [required by ppx_deriving]
∗ install ppx_core 113.33.01+4.03 [required by ppx_type_conv]
∗ install ppx_deriving 4.0 [required by ppx_type_conv]
∗ install ppx_optcomp 113.33.00+4.03 [required by ppx_driver]
∗ install ppx_driver 113.33.01+4.03 [required by ppx_type_conv]
∗ install ppx_type_conv 113.33.01+4.03

(which leads to a failure)

If I install ppx_deriving.3.3 manually, I get ocamlbuild, ocamlfind,
cppo, ppx_tools, ppx_deriving, and then "opam install ppx_type_conv"
suggests to install: ppx_core, ppx_optcomp, ppx_driver, ppx_type_conv.
And this succeeds. So the solver did not suggest in my case the path
with fewer installed packages. And I don't think that I ever tweaked
the OPAM strategy.



Alain

[Hendrik Boom]

Post by Fabrice Le Fessant

Post by Alain Frisch
Testing only the latest version of each package would already be very
useful. As a matter of fact, it could even detect earlier some problems
by restricting the universe only to latest version of all packages (see
below).

Actually, one of my plans when creating opam-builder was to output a list
of versions that would be compatible for a small set of packages (the
platform), and create an opam-repository with only them, i.e. it would be
the equivalent of Debian stable while the standard repository would be
Debian unstable. Clearly, we currently under-use the information that is
available in opam-builder.

I'm told that that is how Debian thought they could organise things.
But they discovered that they also needed a testing repository. I'm
not sure why, but I think it had to do with unstabe being, well, too
unstable. Packages mgrated from unstable to testing after a week with
no serious bugs appearing aginst them, but nly after their dependencies
had also migrated.

Post by Fabrice Le Fessant

Post by Alain Frisch
Moreover, it could be interesting to define a subset of "strategic"
packages that we should really be up-to-date on OCaml release dates.
Wasn't it part of the "platform" project to define such a curated list
of packages?

About that part of the discussion, I am really against any additional
constraint that would delay a release of OCaml. We waited for years before
deciding to have a regular release schedule (every 6 months), and now,
everybody tries to delay releases for various reasons. So, yes to more
monitoring of the compatibility of platform packages with trunk and
beta-releases, but no to delaying the release because of platform packages
being late to meet compatibility.

Post by Alain Frisch
Do you have a description of OPAM-builder current strategy?

Opam-Builder: http://github.com/OCamlPro/opam-builder
OPAM fork: https://github.com/lefessan/opam/tree/2016-03-02-opam-builder
For the strategy, it is the same as OPAM's standard strategy, i.e. it tries
to match your query while minimizing the number of installed packages.
That's the reason why it prefers `ppx_deriving.3.3` to `ppx_deriving.4.0`
(that have an additional requirement to `result`). As a side note, as there
are more and more meta packages (i.e. whose build/installation cost is
null), we should clearly change that strategy to (1) add an installation
cost in every package (`1` by default, `0` for meta-packages) and (2) try
to minimize installation cost instead of number of packages.
--Fabrice
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Thomas Gazagnaire]

The main issue with 4.03 is that killing camlp4 took a bit longer than expected. Things starts to stabilise now that all the ppx infrastructure is in place, much more used than before, and than maintainers did the ppx bump (that at least the case for most of the mirage breakage on 4.03).

The jump from 4.01 to 4.02 was much easier in comparison. I think that’s instability is a one-off. It could make sense to define a set of packages and check that they compile fine before tagging 4.04 though.

Finally, there has been all this work on the "OPAM weather services", continuous build of opam packages etc., plus the work that Damien is doing around release time. It doesn't seem to have worked very well for 4.03. Who is monitoring this stuff? Do we need to fix them to make them work better, or to fix our process to use them better? To be honest, I'm never looking at these services -- I just don't think of using them. Can I get an email whenever they detect that my stuff is broken? Say I'm a new package manager, are there guidelines somewhere on what the best practices are and what I should do?

OPAM weather service will not help you very much here as it doesn’t try to build packages.

Best,
Thomas

[Martin Jambon]

First, thank you Gabriel, Damien, and others who've been working hard on
improving package quality with each new release of OCaml.

I have the chance to be most of the time a passive user of OCaml, i.e.
we develop a commercial application, and we typically upgrade OCaml only
when it becomes necessary. Also it's normally not a problem for us to
wait for 6 months to a year to upgrade. We have really no complaints
there. I feel however that it's a missed opportunity to contribute back
to the community. We generally don't mind bugs in features that we start
using, but we're really afraid of regression. So it would be good for us
to confidently upgrade OCaml as soon as we know it's unlikely to break
our application, but not delay the upgrade out of fear.

I'm also involved in maintaining a small number of public tools because
I started them. Bugs due to incompatibilities with newer versions of
OCaml have been reported and sometimes fixed by others, and while this
is great, I'd rather (1) not have these bugs in the first place and (2)
fix the bugs before they start affecting users. I must say I really lack
the motivation for such maintenance, compared to developing a new
feature that I need.

What could be useful is a few user-visible stats on what's installable
and what's usable in opam, for a given OCaml version. It's the same idea
as labeling an OCaml/opam distribution as beta, but finer-grained. Each
package known to have version-specific problems would be labeled as
such. A package could be in one of a few states such as: OK, Usable but
has some version-specific bugs, Uninstallable. For the opam user
wondering whether they should upgrade OCaml, it would be useful to get
the percentage of packages in each state for a given OCaml version, as
well as the list of packages in each state and links to the relevant bug
reports. Hopefully this would help users be more adventurous and make
informed decisions on whether to upgrade.

Post by Gabriel Scherer
Dear platform,
TL;DR: I propose that we change the OCaml release process to remain in
"beta" stage while there remain OPAM packages that are broken by the new
releases and their version bounds have not been updated.
I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.
A large chunk of my personal hacking time following the release was
spent on updating Batteries, with a 4.03-compatible release happening on
May 12, so two weeks after the release -- a time I would still consider
too long for our downstream users waiting to try the 4.03 goodness but
held back by a Batteries dependency. I recently helped make atdgen
compatible with 4.03 (it's a dependency of IOCaml for example), and just
today found out that Coq releases (or its CoqIDE part) did not work on
4.03 (with no upper-bound in opam, it would just miscompile).
I find it frustrating because that means that there is a large time
window where we advertise (externally, to beginners, etc.) that the best
OCaml versions ever is 4.03.0, while running opam from this switch to
install any major software whatsoever feels like playing a rigged
russian roulette. I feel bad whenever I'm informed that my software (or
anyone else's, in fact) does not work by end-users that would rather
have spent their day doing something else than trying broken software.
Do you agree that this is a problem? What could we do to fix that?
When I started writing this email I intended to send it to Damien
Doligez, as the release manager of the compiler distribution, with the
following suggestion: maybe we could have a *much longer* beta-testing
period for OCaml versions that would be used not (only) for finding bugs
in the new version, but also to let important pieces of the ecosystem
update to the new version. Say, at least a month. We could have a fixed
set of software that we monitor, or in fact all of what's in OPAM, and
delay the release until they get updated/fixed or their OPAM version
bounds get fixed. (Aha, a Platform indeed!)
Does that sound like a reasonable idea to you? Should we monitor all
OPAM, or only a subset?
This suggestion is rooted in the idea that when we announce the release
to our end-users, stuff should just work -- or refuse to install itself.
Maybe this is an old idea of the SVN times and we're now in the era of
continuously-released, always-broken stuff?
Finally, there has been all this work on the "OPAM weather services",
continuous build of opam packages etc., plus the work that Damien is
doing around release time. It doesn't seem to have worked very well for
4.03. Who is monitoring this stuff? Do we need to fix them to make them
work better, or to fix our process to use them better? To be honest, I'm
never looking at these services -- I just don't think of using them. Can
I get an email whenever they detect that my stuff is broken? Say I'm a
new package manager, are there guidelines somewhere on what the best
practices are and what I should do?
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Gabriel Scherer]

TL;DR:
1. We need to have a "test:" field in OPAM metadata and get packages to use
it.
2. We should do a weekly build of OPAM using the ocaml *trunk* version, to
help find and fix (with the cooperation of compiler developers upstream)
issues before the actual release.
3. If such a weekly build is effective, the policy of "stay in beta as long
as allegedly-installable OPAM packages fail to build on the new release"
would be much easier to accept. We could target *all* opam-repository
packages instead of any subset, and it could take less than a month -- say
two weeks.
4. It would be nice to have an automated tool to add a given version bound
on a set of opam files. (And that is easy to implement.)

Post by Simon Cruanes
As usual, the issue is: what set of packages is monitored, and who
chooses?

In this case there is a canonical answer: all of them (on the
public/central OPAM repository).

Note that OWS only tests installability from metadata, which is a useful
metric, but doesn't help when the metadata is wrong and build actually
fails. Damien has a script to compile all of OPAM that could give
buildability result.

Which gets us to the next point: we really should use a "test:" field in
opam metadata and use that. In the atdgen case for example, the software
would compile fine, but it would segfault at usage -- already in the
testsuite. I think that introducing such a "test:" field should be among
our priorities to improve the quality of the OPAM repository.

I agree with the workflow you propose, as long as the beta-testing period
is long enough -- I suggested "one month" and I think it's a good proposal.
Because we could do this with the one-week-long beta period we had in 4.03,
and it would amount to just adding an upper bound on all software, and that
would better for users but only marginally so -- they don't get a crippled
opam-repository, they get a crippled release with no installable software.

In fact I think we should go further and try to make all of OPAM work with
trunk OCaml versions. We could have a weekly build of all-opam from trunk
and collectively fix the issues. This would help ocaml-trunk development
(being able to compile software to test trunk changes is one pain point
right now), and this would help have working software by the release time
-- maybe we wouldn't even need such a long beta period if we did this
effectively.

I don't add upper bounds myself on my own packages (too tedious to do it on

Post by Simon Cruanes
15 old releases), so it should probably be automated.

I agree; this should be very easy to do using existing opam scripting
capabilities. Any volunteers?

Post by Simon Cruanes
The main issue with 4.03 is that killing camlp4 took a bit longer than
expected. [...] The jump from 4.01 to 4.02 was much easier in comparison. I
think that’s instability is a one-off.

I don't buy it. I think the rate of OCaml change is accelerating rather
than slowing down. Think of what will happen if we merge some form of
multicore-ocaml in the future; all FFI code will break loose (well maybe
not all, but like Obj.magic in 4.03, enough will).

Post by Simon Cruanes
Bugs due to incompatibilities with newer versions of OCaml have been
reported and sometimes fixed by others, and while this is great, I'd rather
(1) not have these bugs in the first place and (2) fix the bugs before they
start affecting users. I must say I really lack the motivation for such
maintenance, compared to developing a new feature that I need.

I agree. I think that staring testing packages using trunk rather than
waiting for the release could help with that. First, if the breakage is
under the eyes of the upstream OCaml devs, it may be easier for them to
understand the cause and find the fix -- in my experience helping people
un-break software after a new OCaml version, having a deep knowledge of
said software was rarely necessary, and understanding the compiler was more
useful. Plus it may also encourage people to revert or tame changes that
break user software, reducing the number of changes required of package
maintainers.

Post by Simon Cruanes
First, thank you Gabriel, Damien, and others who've been working hard on
improving package quality with each new release of OCaml.
I have the chance to be most of the time a passive user of OCaml, i.e. we
develop a commercial application, and we typically upgrade OCaml only when
it becomes necessary. Also it's normally not a problem for us to wait for 6
months to a year to upgrade. We have really no complaints there. I feel
however that it's a missed opportunity to contribute back to the community.
We generally don't mind bugs in features that we start using, but we're
really afraid of regression. So it would be good for us to confidently
upgrade OCaml as soon as we know it's unlikely to break our application,
but not delay the upgrade out of fear.
I'm also involved in maintaining a small number of public tools because I
started them. Bugs due to incompatibilities with newer versions of OCaml
have been reported and sometimes fixed by others, and while this is great,
I'd rather (1) not have these bugs in the first place and (2) fix the bugs
before they start affecting users. I must say I really lack the motivation
for such maintenance, compared to developing a new feature that I need.
What could be useful is a few user-visible stats on what's installable and
what's usable in opam, for a given OCaml version. It's the same idea as
labeling an OCaml/opam distribution as beta, but finer-grained. Each
package known to have version-specific problems would be labeled as such. A
package could be in one of a few states such as: OK, Usable but has some
version-specific bugs, Uninstallable. For the opam user wondering whether
they should upgrade OCaml, it would be useful to get the percentage of
packages in each state for a given OCaml version, as well as the list of
packages in each state and links to the relevant bug reports. Hopefully
this would help users be more adventurous and make informed decisions on
whether to upgrade.

Post by Gabriel Scherer
Dear platform,
TL;DR: I propose that we change the OCaml release process to remain in
"beta" stage while there remain OPAM packages that are broken by the new
releases and their version bounds have not been updated.
I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.
A large chunk of my personal hacking time following the release was
spent on updating Batteries, with a 4.03-compatible release happening on
May 12, so two weeks after the release -- a time I would still consider
too long for our downstream users waiting to try the 4.03 goodness but
held back by a Batteries dependency. I recently helped make atdgen
compatible with 4.03 (it's a dependency of IOCaml for example), and just
today found out that Coq releases (or its CoqIDE part) did not work on
4.03 (with no upper-bound in opam, it would just miscompile).
I find it frustrating because that means that there is a large time
window where we advertise (externally, to beginners, etc.) that the best
OCaml versions ever is 4.03.0, while running opam from this switch to
install any major software whatsoever feels like playing a rigged
russian roulette. I feel bad whenever I'm informed that my software (or
anyone else's, in fact) does not work by end-users that would rather
have spent their day doing something else than trying broken software.
Do you agree that this is a problem? What could we do to fix that?
When I started writing this email I intended to send it to Damien
Doligez, as the release manager of the compiler distribution, with the
following suggestion: maybe we could have a *much longer* beta-testing
period for OCaml versions that would be used not (only) for finding bugs
in the new version, but also to let important pieces of the ecosystem
update to the new version. Say, at least a month. We could have a fixed
set of software that we monitor, or in fact all of what's in OPAM, and
delay the release until they get updated/fixed or their OPAM version
bounds get fixed. (Aha, a Platform indeed!)
Does that sound like a reasonable idea to you? Should we monitor all
OPAM, or only a subset?
This suggestion is rooted in the idea that when we announce the release
to our end-users, stuff should just work -- or refuse to install itself.
Maybe this is an old idea of the SVN times and we're now in the era of
continuously-released, always-broken stuff?
Finally, there has been all this work on the "OPAM weather services",
continuous build of opam packages etc., plus the work that Damien is
doing around release time. It doesn't seem to have worked very well for
4.03. Who is monitoring this stuff? Do we need to fix them to make them
work better, or to fix our process to use them better? To be honest, I'm
never looking at these services -- I just don't think of using them. Can
I get an email whenever they detect that my stuff is broken? Say I'm a
new package manager, are there guidelines somewhere on what the best
practices are and what I should do?
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Daniel Bünzli]

1. We need to have a "test:" field in OPAM metadata and get packages to use it.

This already exists. But in a broken form in my opinion; see build-test: field. I hope OPAM v2 will propose something better (simply build: and test:), I talked with Louis about this.

However note that most failures are build failures anyways — notably because a lot of the packages are affected, sometimes without knowing (rec deps) — by the pre-processing cancer that `ppx` and `camlp4` are.

Also one other thing to note is that people stupidly have -warn-error activated in their tarballs. It should be communicated more effectively that this should absolutely not be done as it is difficult to check this by the OPAM repository maintainers.

3. If such a weekly build is effective, the policy of "stay in beta as long as allegedly-installable OPAM packages fail to build on the new release" would be much easier to accept.

I don't think that's reasonable. The responsability here is the one of the package developers — refrain from using hacks that are not able to pass a major version of the OCaml compiler — and the package maintainers (often the same person) who need to actually test and fix the packages they maintain once an OCaml beta is out; this should be especially true for people who distribute cancer inducing substances (but yes it's nice if we can be notified automatically). I don't think that irresponsible people should be allowed to hold the whole eco-system back because of their behaviour.

FWIW AFAIK none of ~20 packages I maintain needed any kind of fixing for 4.03.

Best,

Daniel

[Gabriel Scherer]

Post by Daniel Bünzli
I don't think that irresponsible people should be allowed to hold the
whole eco-system back because of their behaviour.

If it seems to costly to fix a given package, we can always add an upper
bound and declare it done. My condition was that no package that pretends
to be installable should fail to build, not that all packages must be fixed.

Note that I would disapprove of "let's just automatically add an upper
bound on everything that fails at beta time" as a process to meet this
requirement. I would like this to be a human decision, to make sure that we
actually consider fixing them before making them uninstallable.

If specific software is found to be too fragile/unreliable to resist to
OCaml version changes (this arguably is the case of Batteries by design as
its testsuite breaks when the stdlib is extended and Batteries isn't to
match it up), we can decide to always add pessimistic upper bounds on its
packaged versions (Batteries, camlp5 do that). Then it's up to its
maintainers and users to relax the bounds after someone manually checked
that the package works correctly.

Post by Daniel Bünzli

Post by Gabriel Scherer
1. We need to have a "test:" field in OPAM metadata and get packages to

use it.
field. I hope OPAM v2 will propose something better (simply build: and
test:), I talked with Louis about this.
However note that most failures are build failures anyways — notably
because a lot of the packages are affected, sometimes without knowing (rec
deps) — by the pre-processing cancer that `ppx` and `camlp4` are.
Also one other thing to note is that people stupidly have -warn-error
activated in their tarballs. It should be communicated more effectively
that this should absolutely not be done as it is difficult to check this by
the OPAM repository maintainers.

Post by Gabriel Scherer
3. If such a weekly build is effective, the policy of "stay in beta as

long as allegedly-installable OPAM packages fail to build on the new
release" would be much easier to accept.
I don't think that's reasonable. The responsability here is the one of the
package developers — refrain from using hacks that are not able to pass a
major version of the OCaml compiler — and the package maintainers (often
the same person) who need to actually test and fix the packages they
maintain once an OCaml beta is out; this should be especially true for
people who distribute cancer inducing substances (but yes it's nice if we
can be notified automatically). I don't think that irresponsible people
should be allowed to hold the whole eco-system back because of their
behaviour.
FWIW AFAIK none of ~20 packages I maintain needed any kind of fixing for 4.03.
Best,
Daniel

[Daniel Bünzli]

If specific software is found to be too fragile/unreliable to resist to OCaml version changes (this arguably is the case of Batteries by design as its testsuite breaks when the stdlib is extended and Batteries isn't to match it up), we can decide to always add pessimistic upper bounds on its packaged versions (Batteries, camlp5 do that).

This should not be done as it makes it needlessly difficult for compiler and runtime system developers/researchers to test their experimental changes on large real-world subsets of the ocaml repo. (This is also the reason why I'm pushing to have actually working byte-code only switches).

Daniel

[Yaron Minsky]

I do think that PPX rewriters are going to be an ongoing difficulty with
keeping OPAM building with trunk. That's because changes to OCaml's AST
often break rewriters in a way that isn't always trivial to fix. For 4.03,
there was a fair amount of pain for us in providing versions of the PPX
rewriters that worked for both 4.03 and 4.02.

We've contemplated writing a layer that would allow most PPX rewriters to
be written against a stable API that could work across OCaml versions, but
it's real effort.

4.03 has been a bit of a long haul for us internally too --- we're hoping
to release it finally tomorrow morning! I'm hoping that the new 6-month
release cycle will help make releases smaller, and therefore easier to keep
up with. Maybe as OCaml's evolution accelerates, we can make the cycle
faster (though it's hard to imagine having releases more than quarterly
makes any sense.) This would presumably require more and better automation
to really be tenable.

y

Post by Gabriel Scherer

Post by Gabriel Scherer
If specific software is found to be too fragile/unreliable to resist to

OCaml version changes (this arguably is the case of Batteries by design as
its testsuite breaks when the stdlib is extended and Batteries isn't to
match it up), we can decide to always add pessimistic upper bounds on its
packaged versions (Batteries, camlp5 do that).
This should not be done as it makes it needlessly difficult for compiler
and runtime system developers/researchers to test their experimental
changes on large real-world subsets of the ocaml repo. (This is also the
reason why I'm pushing to have actually working byte-code only switches).
Daniel
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Alain Frisch]

OCamlPro's opam builder (
http://opam.ocamlpro.com/builder/html/report-last.html ) really needs to
be more advertised and its UI a bit polished to encourage people to
actually look at it. Also a report per maintainer (with emails sent
automatically?) could be useful.

The release cycle is certainly important for this matter, and I hope the
new policy (smaller and more regular releases) will help significantly.
Simply announcing the next release date (ans sticking to it) sends a
signal to package authors about when they should start checking their
code works with trunk. Of course, smaller releases also make it simpler
to update the code and less risky to upgrade to for larger organization
(and thus packages maintained by them are more likely to be fixed
quickly). Hopefully the 4.03 experience was a bit exceptional (increased
development pace, very long release, moving release date, breaking
changes in the last months).

There is certainly things to learn and do to minimize the breakage of
ppx and their consequences. Introducing abstraction/compatibility
layers should help (either things such as Ast_convenience.Constant in
ppx_tools, or more ambitious stable APIs over the Parsetree).

It would also be interesting to understand in more details the reasons
of the breakages for so many packages in 4.03. It'd be a tedious work
to collect this information, but it will probably guide both core
maintainers and the community on how to improve the process. There are
two really important pieces of information: which changes in OCaml
cause the more breakage, and which broken packages cause the more
transitive breakage.

I've to say that I'm also a bit scared by the inflation of dependencies
across packages, including for relatively core packages that could
themselves potentially block many others. It's not because OPAM makes
it easy to depend on external packages that it's always a good idea to
do so. As an example, see
https://github.com/ocsigen/js_of_ocaml/blob/master/opam, and this does
not account for transitive dependencies, which include e.g. oasis (whose
own dependencies are themselves a bit scary). More atomic packages
could help in some respect (e.g. a package with only the js_of_ocaml
compiler, with precompiled menhir parsers, could only depnd on cmdliner,
I guess). Same for ocamlnet (which is still not available for 4.03, and
probably causing some further breakages; if it were split into smaller
packages, at least some of them should work on 4.03 already).



Alain

Post by Gabriel Scherer
Dear platform,
TL;DR: I propose that we change the OCaml release process to remain in
"beta" stage while there remain OPAM packages that are broken by the new
releases and their version bounds have not been updated.
I'm a bit frustrated with the time it takes for the OCaml ecosystem to
catch up with the 4.03 release -- which has been released two months ago
now, on April 25.
A large chunk of my personal hacking time following the release was
spent on updating Batteries, with a 4.03-compatible release happening on
May 12, so two weeks after the release -- a time I would still consider
too long for our downstream users waiting to try the 4.03 goodness but
held back by a Batteries dependency. I recently helped make atdgen
compatible with 4.03 (it's a dependency of IOCaml for example), and just
today found out that Coq releases (or its CoqIDE part) did not work on
4.03 (with no upper-bound in opam, it would just miscompile).
I find it frustrating because that means that there is a large time
window where we advertise (externally, to beginners, etc.) that the best
OCaml versions ever is 4.03.0, while running opam from this switch to
install any major software whatsoever feels like playing a rigged
russian roulette. I feel bad whenever I'm informed that my software (or
anyone else's, in fact) does not work by end-users that would rather
have spent their day doing something else than trying broken software.
Do you agree that this is a problem? What could we do to fix that?
When I started writing this email I intended to send it to Damien
Doligez, as the release manager of the compiler distribution, with the
following suggestion: maybe we could have a *much longer* beta-testing
period for OCaml versions that would be used not (only) for finding bugs
in the new version, but also to let important pieces of the ecosystem
update to the new version. Say, at least a month. We could have a fixed
set of software that we monitor, or in fact all of what's in OPAM, and
delay the release until they get updated/fixed or their OPAM version
bounds get fixed. (Aha, a Platform indeed!)
Does that sound like a reasonable idea to you? Should we monitor all
OPAM, or only a subset?
This suggestion is rooted in the idea that when we announce the release
to our end-users, stuff should just work -- or refuse to install itself.
Maybe this is an old idea of the SVN times and we're now in the era of
continuously-released, always-broken stuff?
Finally, there has been all this work on the "OPAM weather services",
continuous build of opam packages etc., plus the work that Damien is
doing around release time. It doesn't seem to have worked very well for
4.03. Who is monitoring this stuff? Do we need to fix them to make them
work better, or to fix our process to use them better? To be honest, I'm
never looking at these services -- I just don't think of using them. Can
I get an email whenever they detect that my stuff is broken? Say I'm a
new package manager, are there guidelines somewhere on what the best
practices are and what I should do?
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Sylvain Le Gall]

Post by Alain Frisch
I've to say that I'm also a bit scared by the inflation of dependencies
across packages, including for relatively core packages that could
themselves potentially block many others. It's not because OPAM makes
it easy to depend on external packages that it's always a good idea to
do so. As an example, see
https://github.com/ocsigen/js_of_ocaml/blob/master/opam, and this does
not account for transitive dependencies, which include e.g. oasis (whose
own dependencies are themselves a bit scary).

OASIS 0.4.7, next version to be released, will kill the scariest
dependencies of all (camlp4 and ocaml-data-notation). Some simple packages
will remain (ocamlify and ocamlmod).

[Alain Frisch]

Post by Alain Frisch
OCamlPro's opam builder (
http://opam.ocamlpro.com/builder/html/report-last.html ) really needs to
be more advertised and its UI a bit polished to encourage people to
actually look at it. Also a report per maintainer (with emails sent
automatically?) could be useful.

Here is some wishlist for this tool:

- Add trunk testing: this is what will put pressure for people to
continuously upgrade their packages. Ideally, it would also show the
(date of the) latest successful build on trunk. Same for release
branches once they are forked.

- Make the grid more narrow to fit more versions on a screen (e.g.
abbreviate Broken Deps to "Deps") and show more recent versions
(including trunk) to the left.

- By default, show only the latest version of each package.

- Add a mode where packages are grouped by maintainer.

- Try to make the errors more visible. E.g. for
http://opam.ocamlpro.com/builder/html/github/github.1.0.0/3b48ac1bc360931d68afa4e126d96b44,
the section "=== ERROR while compiling github.1.0.0 ==" could be moved
to the top of the screen.

- There could also be heuristics to detect the error and show
directly in the main report grid the category. For instance,
http://opam.ocamlpro.com/builder/html/acgtk/acgtk.1.0b/764d15d569e342c5ee761551e398fb38
is simply about a new warning and "-warn-error A" in the package.

- It would be useful to force building package even if they claim they
are not compatible with the latest version (or trunk). For instance, I
was curious about why ocamlnet.4.0.4 did not build with 4.03. (In case
people wonder, the first problem comes from the new error on multiple
definitions of exceptions with the same name; this error was introduced
on trunk in 2014 IIRC. The fix for ocamlnet is just to remove one line.)


Alain

[Fabrice Le Fessant]

If you look at the announce of opam-builder, you will notice that it was
monitoring 4.03.0+beta1 from the beginning:
http://lists.ocaml.org/pipermail/opam-devel/2016-March/001412.html

4.03.0+beta2 was monitored too, as soon as it was released.

But indeed, opam-builder's website is far from perfection, it could be
faster to load, send notifications by email, recognize and display error
messages (and much more, as the information sleeping there is huge...). We
will try to improve it when we have time, but for now, it is on our
spare-time list of projects.

I agree with Yaron that PPX are more and more going to become a problem for
upgrading to the latest OCaml, as Camlp4 was before (and is still,
unfortunately). Many packages now depend on ppx-rewriters, that will almost
never be in sync with trunk, which makes testing trunk very hard, even if
we patch OPAM not to use the upper-bounds on ocaml-version in package
descriptions. Soon, OCaml users will probably ask us to merge some
ppx-rewriters in the OCaml distribution to force the sync, as they asked
for Camlp4 long ago.

--Fabrice

Post by Alain Frisch

Post by Alain Frisch
OCamlPro's opam builder (
http://opam.ocamlpro.com/builder/html/report-last.html ) really needs to
be more advertised and its UI a bit polished to encourage people to
actually look at it. Also a report per maintainer (with emails sent
automatically?) could be useful.

- Add trunk testing: this is what will put pressure for people to
continuously upgrade their packages. Ideally, it would also show the
(date of the) latest successful build on trunk. Same for release
branches once they are forked.
- Make the grid more narrow to fit more versions on a screen (e.g.
abbreviate Broken Deps to "Deps") and show more recent versions
(including trunk) to the left.
- By default, show only the latest version of each package.
- Add a mode where packages are grouped by maintainer.
- Try to make the errors more visible. E.g. for
http://opam.ocamlpro.com/builder/html/github/github.1.0.0/3b48ac1bc360931d68afa4e126d96b44
,
the section "=== ERROR while compiling github.1.0.0 ==" could be moved
to the top of the screen.
- There could also be heuristics to detect the error and show
directly in the main report grid the category. For instance,
http://opam.ocamlpro.com/builder/html/acgtk/acgtk.1.0b/764d15d569e342c5ee761551e398fb38
is simply about a new warning and "-warn-error A" in the package.
- It would be useful to force building package even if they claim they
are not compatible with the latest version (or trunk). For instance, I
was curious about why ocamlnet.4.0.4 did not build with 4.03. (In case
people wonder, the first problem comes from the new error on multiple
definitions of exceptions with the same name; this error was introduced
on trunk in 2014 IIRC. The fix for ocamlnet is just to remove one line.)
Alain
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

[Yaron Minsky]

Including ppx-rewriters in the distribution would be the wrong resolution,
I think. The modularity afforded by the current world is worth a lot,
letting independent sets of people to work on different pieces of the
infrastructure, and making the work of the core team as simple as possible.
I think the core team should be loathe to give this up.

Having some kind of compatibility story that most PPXs could use which
would make PPXs compatible across compiler versions could help a great
deal, though. Maybe that's where the effort can be best spent. Such a thing
could even be done outside of the core distribution, though it would have
to be among the very first packages ported. Indeed, given that it would
need to work with multiple compiler versions, it is perhaps best for it to
be developed outside.

y

On Thu, Jun 30, 2016 at 4:57 AM Fabrice Le Fessant <

Post by Fabrice Le Fessant
If you look at the announce of opam-builder, you will notice that it was
http://lists.ocaml.org/pipermail/opam-devel/2016-March/001412.html
4.03.0+beta2 was monitored too, as soon as it was released.
But indeed, opam-builder's website is far from perfection, it could be
faster to load, send notifications by email, recognize and display error
messages (and much more, as the information sleeping there is huge...). We
will try to improve it when we have time, but for now, it is on our
spare-time list of projects.
I agree with Yaron that PPX are more and more going to become a problem
for upgrading to the latest OCaml, as Camlp4 was before (and is still,
unfortunately). Many packages now depend on ppx-rewriters, that will almost
never be in sync with trunk, which makes testing trunk very hard, even if
we patch OPAM not to use the upper-bounds on ocaml-version in package
descriptions. Soon, OCaml users will probably ask us to merge some
ppx-rewriters in the OCaml distribution to force the sync, as they asked
for Camlp4 long ago.
--Fabrice

Post by Alain Frisch

Post by Alain Frisch
OCamlPro's opam builder (
http://opam.ocamlpro.com/builder/html/report-last.html ) really needs

to

Post by Alain Frisch
be more advertised and its UI a bit polished to encourage people to
actually look at it. Also a report per maintainer (with emails sent
automatically?) could be useful.

- Add trunk testing: this is what will put pressure for people to
continuously upgrade their packages. Ideally, it would also show the
(date of the) latest successful build on trunk. Same for release
branches once they are forked.
- Make the grid more narrow to fit more versions on a screen (e.g.
abbreviate Broken Deps to "Deps") and show more recent versions
(including trunk) to the left.
- By default, show only the latest version of each package.
- Add a mode where packages are grouped by maintainer.
- Try to make the errors more visible. E.g. for
http://opam.ocamlpro.com/builder/html/github/github.1.0.0/3b48ac1bc360931d68afa4e126d96b44
,
the section "=== ERROR while compiling github.1.0.0 ==" could be moved
to the top of the screen.
- There could also be heuristics to detect the error and show
directly in the main report grid the category. For instance,
http://opam.ocamlpro.com/builder/html/acgtk/acgtk.1.0b/764d15d569e342c5ee761551e398fb38
is simply about a new warning and "-warn-error A" in the package.
- It would be useful to force building package even if they claim they
are not compatible with the latest version (or trunk). For instance, I
was curious about why ocamlnet.4.0.4 did not build with 4.03. (In case
people wonder, the first problem comes from the new error on multiple
definitions of exceptions with the same name; this error was introduced
on trunk in 2014 IIRC. The fix for ocamlnet is just to remove one line.)
Alain
_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform

_______________________________________________
Platform mailing list
http://lists.ocaml.org/listinfo/platform