Server monitoring and Data download exception handling

[bezunakarmi]

• update existing bash script to generate new sqlite during import and then replace existing sqlite after successful import
• setup uptimerobot for open.undg.org
• password protected static status page listing the log from last import to see the possible errors if any

Cron job for fetching organization data is currently not working properly. This is due to API provided by IATI registry is not working properly. eg: fetching data for UNDP https://iatiregistry.org/api/3/action/package_search?fq=organization:undp&rows=1000000
responds with following empty results data

So, what we should do for now is

• If an organization data is not fetched then the Cron process of re-creating database should be aborted.
• A log should be maintained to review the process, which should be accessible by /logs url.
• Log should clearly mention the date and organization name.

[anjesh]

@bezunakarmi any updates on the updating bash script during data fetch. I see that few agencies have 0 data when i am looking at the pages. Shall we expedite atleast the bash script issue. Thanks.

[anjesh]

As discussed, we might need to follow these steps for this. @bezunakarmi

• setup a second instance of this app in different folder

There should be a bash which performs the followings

• run the xml-data-import code only to different database name
• if success, backup the first-instance-db and replace that with second-instance-db
• restart the first-instance-app

lets discuss if there's any confusion.