Prevent future cookiecutter update merge conflict with physical file separation

[tlvu]

With my recent cookiecutter refresh work in Raven, Finch, Flyingpigeon and Emu (issue #61), it strikes me that the 2 files with the most merge conflicts are the Makefile and the .travis.yml file.

These 2 files are essential for dev workflow and testing automation and would benefit most from staying in sync effortlessly without merge conflicts with all the latest improvements.

I am proposing we split these files into a "generic" file and an "override" file. The intention is to provide a clear physical boundary indication that if the "generic" file changes, the change should be backported to the cookiecutter to be synced to other projects.

Right now, I think there are differences because it's not clear which changes are generic and should be backported and which are not.

For the Makefile, all the generic targets and variables stay there and all the project specific targets and variables will go into a Makefile.inc and we can include into the Makefile. This is a strategy documented in Scaraplate https://scaraplate.readthedocs.io/en/latest/template.html?highlight=makefile#include-files

For the .travis.yml, it looks like we can use import statement to import snippets of generic code into the main .travis.yml file, see https://docs.travis-ci.com/user/build-config-imports.

I think other files would also benefit from this physical file boundary separation but let's just start with just these 2 most useful files and see how it goes.

@cehbrecht @Zeitsperre I would like to have your opinion whether it's something desirable and doable before I spend time on this.

Another very closely related issue is I think we should invoke as much as possible make targets from Travis-CI so we can test those make targets that a real dev will use and by the same token make those Travis-CI setup steps available to the dev via make targets so dev can also reproduce problems found on Travis-CI locally.

So I am proposing we migrate all the setup and testing steps in the .travis.yml file into make targets and call those make targets from the .travis.yml file.

Again, @cehbrecht @Zeitsperre I would like your opinion on this before proceeding.

[Zeitsperre]

Good point that we need to settle this before you advance. As it stands, there's quite a few differences for many of the birds' .travis.yml files due to a few differences in approaches between the services:

• Does it call make recipes?
• Are the make the same?
• Does it manage the installed Ana-/mini- conda binary?
• Does it run special pytest ensembles, doctests, or notebook-based tests?

Personally, I would like to see a standard slew of recipes in the top half of makefile configurations and special recipes for particular birds added to the bottom. For Anaconda configurations, much of the work is simply in the .travis.yml files but I would like it if we had one makefile recipe shared between all birds to install+activate and deactivate+uninstall anaconda, but that's just me.

I agree with you that having .travis.yml configurations that call make commands only would greatly simply the management, and seeing that build environments for all birds can be shared (Linux/macOS + Python3.6 are easy to manage in Travis CI), I think making a cross-cutting makefile with recipes common to all birds and an example makefile.inc for special recipes (thoughts to raven) would be a great approach.

[cehbrecht]

@tlvu my thoughts ... I would prefer a standard Makefile (maintained by the cookiecutter) that is suitable for most birds. Customizations can be added to the makefile.inc.

We once had the conda setup in the makefile ... but it got to complicated (users have different shells: bash, zsh, ...)... so we skipped it. We could add it again as an optional target. We could also have travis CI targets to make the .travis.yml more readable ... maintainable.

[tlvu]

@Zeitsperre

Does it run special pytest ensembles, doctests, or notebook-based tests?

Travis-CI should run as much as possible all the various make targets to test them out. So the answer is yes to all the above and more if we can squeeze in.

Personally, I would like to see a standard slew of recipes in the top half of makefile configurations and special recipes for particular birds added to the bottom.

Can special recipes for particular birds be out to a separate Makefile.inc file. The separation effect is greater with a physical file boundary.

For Anaconda configurations, much of the work is simply in the .travis.yml files but I would like it if we had one makefile recipe shared between all birds to install+activate and deactivate+uninstall anaconda, but that's just me.

We can have install and uninstall. Not sure about activate, deactivate as it would have problems with the different shells again. But we can add detection that it is not activated and display instructions to user.

I agree with you that having .travis.yml configurations that call make commands only would greatly simply the management, and seeing that build environments for all birds can be shared (Linux/macOS + Python3.6 are easy to manage in Travis CI)

Thinking the same here.

@cehbrecht

I would prefer a standard Makefile (maintained by the cookiecutter) that is suitable for most birds. Customizations can be added to the makefile.inc.

Thinking the same here.

We once had the conda setup in the makefile ... but it got to complicated (users have different shells: bash, zsh, ...)... so we skipped it. We could add it again as an optional target.

So I was thinking to keep the conda install part in the Makefile but leave out the activate part because activation is shell dependent. User is responsible for activating for their respective shell.

That's what is left in Raven right now https://github.com/Ouranosinc/raven/blob/master/Makefile#L72-L98

Other targets do not depend on those conda targets so user have to manually activate before using other targets https://github.com/Ouranosinc/raven/blob/ee6fdc151ad50b7faac1daaf4a018eee1f6945c7/Makefile#L138-L147

We could also have travis CI targets to make the .travis.yml more readable ... maintainable.

Thinking the same here.

[Zeitsperre]

This seems pretty reasonable of an approach. Feel free to tag me in on anything relevant to the discussion here and I can give it a look. I personally would love to be able to run a few tox-based recipes for local testing of builds and teardowns without touching the base environment, but that's a bit "pie-in-the-sky" for now. Let's get something in common to all the birds first.