Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add generic_bird optional component #41

Merged
merged 8 commits into from
Apr 15, 2020
Merged

Conversation

tlvu
Copy link
Collaborator

@tlvu tlvu commented Apr 9, 2020

Could be used as work-around for issue bird-house/finch#121 so we have a Finch dedicated for async request only.

This component is generic and configurable enough it could even accommodate any WPS. It has its own Postgres instance so we can use it later to experiment with different version of Postgres.

Test server: https://lvupavics-lvu.pagekite.me/twitcher/ows/proxy/finchasync, https://lvupavics-lvu.pagekite.me/canarie/node/service/status

See the README.md update for more info.

Note the docker image, service name and port is customizable (my test server do not use the default service name).

It starts out as a Finch2 but now its rather a generic skeleton to plugin any birds we need. I wish I had this when I was testing out the new Thunderbird from PCIC.

At some point in the past, we had the idea of 2 simultaneous FlyingPigeon because the new one is so different we wanted to keep the old one around for a short while. This PR could have helped.

So do not view this PR as Finch 2 anymore, view this PR as a mechanism to quickly deploy another bird, any bird, without having to change code.

This opens up many options, testing is just one of the possible usage.

tlvu added 2 commits April 8, 2020 21:57
See README.md for more details.
Bonus, this change also prevent the same value to be hardcoded multiple
times at various locations.

Optional components should probably not hook directly into `common.env`
and `pavics-compose.sh`.  Another refactoring for another day.
@tlvu tlvu requested review from huard and tlogan2000 April 9, 2020 02:58
tlvu added 4 commits April 9, 2020 13:31
Default was SQLite in-memory and it does not clear completed async wps
requests so we quickly run into the default 2 max concurrent wps
requests !

Use a separate DB so this optional component do not leave traces when we
disable it.

This will also allow us to test new different version of Postgres DB if
needed.  Default version is latest Postgres, not the old version
currently in used.
… Finch

We can plug in other WPS docker images in this skeleton/generic
component.  All the elements that need to change are configurable (was
just missing the FINCH2_INTERNAL_PORT).

The HOSTNAME and HTTP_PORT env var are only for older birds so disabled
them.

Set WPS logging to a more suitable production logging level, can easily
be changed later.
Copy link
Collaborator

@huard huard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm uncomfortable with this. I suppose it's ok as a temporary measure, but I don't like the idea of having multiple versions of birds without a clear purpose and framework.

We now have a secondary server for thredds and finch. Should we uniformize how we deal with this ? Ie thredds-test, finch-test, etc.
Should test servers inherit their config from their parent ?
How do these test servers differ from our staging environment ?

@tlvu
Copy link
Collaborator Author

tlvu commented Apr 15, 2020

I'm uncomfortable with this. I suppose it's ok as a temporary measure, but I don't like the idea of having multiple versions of birds without a clear purpose and framework.

It's an optional generic component, if this looks confusing, we don't need to enable it.

We now have a secondary server for thredds and finch. Should we uniformize how we deal with this ? Ie thredds-test, finch-test, etc.

It starts out as a Finch 2 but now its rather a generic skeleton to plugin any birds we need. I wish I had this when I was testing out the new Thunderbird from PCIC.

At some point in the past, we had the idea of 2 simultaneous FlyingPigeon because the new one is so different we wanted to keep the old one around for a short while. This PR could have helped.

So do not view this PR as Finch 2 anymore, view this PR as a mechanism to quickly deploy another bird, any bird, without having to change code.

Should test servers inherit their config from their parent ?

No, this is a standalone bird.

How do these test servers differ from our staging environment ?

Good question, same reason why we have Test Thredds: the staging and prod environment is not exactly identical, not same data on Thredds.

This opens up many options, testing is just one of the possible usage.

@tlvu tlvu changed the title Add finch2 optional component Add generic_bird optional component Apr 15, 2020
@tlvu tlvu merged commit 5c58136 into master Apr 15, 2020
@tlvu tlvu deleted the add-finch2-optional-component branch April 15, 2020 14:05
@tlvu
Copy link
Collaborator Author

tlvu commented Apr 17, 2020

Auto deployed to prod:

triggerdeploy finished START_TIME=2020-04-16T05:07:01-0400
triggerdeploy finished   END_TIME=2020-04-16T05:08:25-0400

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants