Giga Maps Backend

Table of Contents

1. About Giga
2. What is Giga Maps
   • Why it is important?
   • Github Repositories of Giga Maps
3. School Location
4. School Connectivity
5. Infrastructure
6. Getting Started
7. Code of Conduct
8. Contribution Guidelines
9. License/Legal
10. Contact
11. Acknowledgements

About Giga

Giga is a UNICEF-ITU global initiative to connect every school to the Internet and every young person to information, opportunity, and choice. By connecting all schools to the Internet, we ensure that every child has a fair shot at success in an increasingly digital world.

What is Giga Maps?

Giga Maps aims to build a global, dynamic map that showcases the status of school connectivity in the world. This tool helps to identify gaps in school connectivity, create transparency, estimate the capital investment needed to connect unconnected schools, develop plans, attract the necessary funding, and track progress towards achieving universal school Internet connectivity.

Why it is important?

Mapping schools is important because we can't solve a problem if we don't know the problem. Giga's mission is to connect every school to the internet and for that, we need to know accurate info on each school location, their connectivity and their infrastructure.

We need to put it on an open map because this is a global project with multiple stakeholders like governments, donors, investors, UNICEF implementation country offices, Internet service providers, partners, schools, open source community and of course, Giga. All of them need to have a common tool through which they can see the progress in connecting schools, highlight its importance and bring actionable clarity and transparency into the process. So, Giga Maps is this common narrative tool for all Giga members and stakeholders.

Github Repositories of Giga Maps

• Giga Maps Frontend
• Giga Maps Backend
• Giga – documentation

School Location

Locate schools through our open map

2.1M schools location collected from 50 government bodies, Open data sources like OSM and Giga's AI model.

School Connectivity

Monitor real-time connectivity of schools

Real time connectivity monitored through GigaCheck Desktop App and chrome extension, Brazil's nic.br app and multiple ISP collaborations.

Infrastructure

Understand connectivity infrastructure available around schools

Infrastructure data contributed by ~10 partners like ITU, Meta, GSMA, and government bodies.

Tasks

Sr No	Task Name	Path	Description	Frequency
1.	update_all_cached_values	proco.utils.tasks.update_all_cached_values	Update cached values in Backend Redis	4:00 AM Every day
2.	rebuild_school_index	proco.utils.tasks.rebuild_school_index	Create the Cognitive Search Index	2:00 AM Every day
3.	clean_old_realtime_data	proco.connection_statistics.tasks.clean_old_realtime_data	Delete more than 30 days old data from RealTimeConnectivity table	5:00 AM Every day
4.	update_school_records	proco.schools.tasks.update_school_records	Populate the connectivity_status and coverage_status field in School table	1:00 AM, 1:00 PM Every day
5.	cleanup_school_master_rows	proco.data_sources.tasks.cleanup_school_master_rows	Cleanup School Master data	1:40 AM, 3:40 PM Every day
6.	update_static_data	proco.data_sources.tasks.update_static_data	Fetch the data from School Master data source	3:00 AM Every day
7.	update_live_data	proco.data_sources.tasks.update_live_data	Fetch the live data from PCDC and QoS data source	2:10 AM, 8:10 AM, 2:10 PM , 8:10 PM Every day
8.	update_live_data_and_aggregate_yesterday_data	proco.data_sources.tasks.update_live_data	Fetch the latest live data and aggregate the previous daya data	00:30 AM every day
9.	populate_school_registration_data	proco.utils.tasks.populate_school_registration_data	Populate the RT status for newly added schools in Live/Static data source	2:40 AM Every day
10.	handle_published_school_master_data_row	proco.data_sources.tasks.handle_published_school_master_data_row	Handle Published School Master data records	Every 4 hour at 10th minute
11.	handle_deleted_school_master_data_row	proco.data_sources.tasks.handle_deleted_school_master_data_row	Delete the School from Proco DB if deleted by School Master data source	00:10 AM every day
12.	email_reminder_to_editor_and_publisher_for_review_waiting_records	proco.data_sources.tasks.email_reminder_to_editor_and_publisher_for_review_waiting_records	Send reminder mail to Editors and Publishers to review the School Master records if any	08:10 AM every day
13.	clean_old_live_data	proco.data_sources.tasks.clean_old_live_data	Cleanup for QoS and PCDC source data table for data older than 30 days	05:10 AM every day

Getting Started

Prerequisites

Python: version 3.8

Installation

Build the project from source:

1. Clone the giga-maps-backend repository:

❯ git clone https://github.com/unicef/giga-maps-backend

2. Navigate to the project directory:

❯ cd giga-maps-backend

3. Install the required dependencies:

You can create a requirements.txt file first with the following command

❯ pip freeze > requirements.txt

Then you can convert a Pipfile and Pipfile.lock into a requirements.txt.

❯ pipenv lock -r > requirements.txt

After that, you can install all your modules in your python virtual environment by doing the following:

❯ pip install -r requirements.txt

Usage

To run the project, execute the following command:

❯ python main.py

Tests

Execute the test suite using the following command:

❯ pytest

---

Repository Structure

└── giga-maps-backend/
    ├── .github
    │   ├── dependabot.yml
    │   └── workflows
    ├── Dockerfile
    ├── Pipfile
    ├── Pipfile.lock
    ├── README.md
    ├── azure-pipeline-dev.yml
    ├── celery-dev.sh
    ├── celery.sh
    ├── celerybeat.sh
    ├── config
    │   ├── __init__.py
    │   ├── settings
    │   ├── urls.py
    │   ├── utils.py
    │   └── wsgi.py
    ├── data_sources.json
    ├── deploy
    │   └── server.py
    ├── docker
    │   ├── Dockerfile
    │   ├── Dockerfile-db
    │   ├── Dockerfile-db15
    │   ├── Dockerfile-dev
    │   ├── docker-compose.dev.yml
    │   ├── docker-compose.test.yml
    │   ├── docker-compose.yml
    │   ├── initdb copy.sh
    │   └── initdb.sh
    ├── hello.py
    ├── manage.py
    ├── proco
    │   ├── __init__.py
    │   ├── about_us
    │   ├── accounts
    │   ├── assets
    │   ├── background
    │   ├── connection_statistics
    │   ├── contact
    │   ├── context_processors.py
    │   ├── core
    │   ├── custom_auth
    │   ├── data_sources
    │   ├── fixtures
    │   ├── locations
    │   ├── mailing
    │   ├── proco_data_migrations
    │   ├── schools
    │   ├── taskapp
    │   ├── templates
    │   └── utils
    ├── scripts
    │   ├── runtests.sh
    │   └── wait-for-it.sh
    ├── sshd_config
    ├── web-flask.sh
    └── web-worker.sh

---

In the root directory of the application, you will find the configuration files for the development and build process. These include configuration files for code linting, the Jest configuration for the UI testing tool, and files for deployment with Docker. The source code is located in the src/ directory. The webpack/ directory contains the environment configuration and webpack configuration files.

Additionally, the jest.config.js file is used to configure Jest for UI testing. It specifies the test environment, the test files to be run, and the module name mapping.

Code of Conduct

At Giga, we're committed to maintaining an environment that's respectful, inclusive, and harassment-free for everyone involved in our project and community. We welcome contributors and participants from diverse backgrounds and pledge to uphold the standards.

Click here for detailed Code of Conduct

Contribution Guidelines

Contributions are welcome! Here are several ways you can contribute:

• Report Issues: Submit bugs found or log feature requests for the giga-maps-backend project.
• Submit Pull Requests: Review open PRs, and submit your own PRs.
• Join the Discussions: Share your insights, provide feedback, or ask questions.

If you feel impacted by the work we are doing and would like to contirbute through monetary means then consider donating at https://give.giga.global

Contributing Guidelines

1. Fork the Repository: Start by forking the project repository to your github account.
2. Clone Locally: Clone the forked repository to your local machine using a git client.

   git clone https://github.com/unicef/giga-maps-backend

3. Create a New Branch: Always work on a new branch, giving it a descriptive name.

   git checkout -b new-feature-x

4. Make Your Changes: Develop and test your changes locally.
5. Commit Your Changes: Commit with a clear message describing your updates.

   git commit -m 'Implemented new feature x.'

6. Push to github: Push the changes to your forked repository.

   git push origin new-feature-x

7. Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
8. Review: Once your PR is reviewed and approved, it will be merged into the main branch. Congratulations on your contribution!

Contributor Graph

---

License / Legal

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. See LICENSE for more information.

For the plain text version please see https://www.gnu.org/licenses/agpl-3.0.txt

Contact

GigaMaps Project Lead: Shilpa Arora: [email protected]

GigaMaps Project Member: Vipul Bhavsar: [email protected]

Giga Open-Source Community Manager: David Njagah: [email protected]

Giga Website: https://giga.global/contact-us/

Acknowledgments💜

• Thanks to the Nagarro team for helping build this application!