From 89650a977e59c1f01bb1f0af118dfc2c4e33043f Mon Sep 17 00:00:00 2001 From: Holly Date: Wed, 25 Sep 2024 21:49:28 -0400 Subject: [PATCH] Lab template updates --- README.md | 25 ++++++---------- docs/1-intro-to-data-lab.md | 46 ++++++++++++++++++++++++++++++ docs/2-analytics.md | 43 ++++++++++++++++++++++++++++ docs/_toc.yml | 3 +- docs/introduction-to-data-goods.md | 23 --------------- docs/team.md | 4 +-- 6 files changed, 102 insertions(+), 42 deletions(-) create mode 100644 docs/1-intro-to-data-lab.md create mode 100644 docs/2-analytics.md delete mode 100644 docs/introduction-to-data-goods.md diff --git a/README.md b/README.md index 317e6dc..a165c89 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,12 @@ The Myanmar Prosperity team is responsible for undertaking macroeconomic surveillance of Myanmar and conducting analytical studies to guide economic policy and development, including the bi-annual Myanmar Economic Monitor. However, published data can be delayed or unreliable, making traditional macroeconomic monitoring a challenge. The Lab team was asked to investigate different sources of alternative data to help fill data gaps, covering such areas as conflict monitoring, crop production monitoring, analyzing trends in nighttime lights, and understanding changing patterns of population movement. +## Contents + +```{tableofcontents} + +``` + ## Data @@ -19,20 +25,9 @@ The following datasets are used in this assignmet. Further details about each da +#### Data Availability Statement -## Reusable Data Science Products - -The following data products are included in this project: - -* [Conflict Monitoring](https://datapartnership.org/myanmar-economic-monitor/notebooks/conflict/acled.html) - -* [Nighttime Lights Monitoring](https://datapartnership.org/myanmar-economic-monitor/notebooks/nighttime-lights/analysis-2023/README.html) - -* [Crop Monitoring](https://datapartnership.org/myanmar-economic-monitor/notebooks/vegetation-conditions/README.html) - -* [Activity Index](https://datapartnership.org/myanmar-economic-monitor/notebooks/mobility/activity.html#) - - +Restrictions may apply to the data that support the findings of this study. Data received from the private sector through the Development Data Partnership are subject to the terms and conditions of the data license agreement and the “Official Use Only” data classification. These data are available upon request through the [Development Data Partnership](https://datapartnership.org/). Licensing and access information for all other datasets are included in the documentation. @@ -41,8 +36,6 @@ The following data products are included in this project: This projects is licensed under the [**Mozilla Public License**](https://opensource.org/license/mpl-2-0/) - see the [LICENSE](LICENSE) file for details. - - ## Code of Conduct -The template used to create this project maintains a [Code of Conduct](docs/CODE_OF_CONDUCT.md) to ensure an inclusive and respectful environment for everyone. Please adhere to it in all interactions within our community. +The World Bank Data Lab template used to create this project maintains a [Code of Conduct](docs/CODE_OF_CONDUCT.md) to ensure an inclusive and respectful environment for everyone. Please adhere to it in all interactions within our community. diff --git a/docs/1-intro-to-data-lab.md b/docs/1-intro-to-data-lab.md new file mode 100644 index 0000000..2d0b375 --- /dev/null +++ b/docs/1-intro-to-data-lab.md @@ -0,0 +1,46 @@ +# Introduction to the Data Lab + +The Data Lab supports World Bank operations -- lending, technical assistance, and economic reporting -- by coordinating ad-hoc teams of data analysts and specialists from across our organization. Through the Lab, teams solve global challenges using best practices in coding, code documentation, and data visualization. + +Unlike a traditional data analysis, which results in a single-use report or visualization, Data Lab products are designed to be customized, reused, and updated, thereby building the capacity of the World Bank and partner organizations to quickly deliver complex data science solutions to pressing global challenges. + +Data Lab-supported projects may include: + +1. **Data**. Data Lab teams provide guidance on how to access the data underpinning all analyses, indicators, and insights. This transparency in data sources supports reproducibility and, critically, re-use in new countries and contexts, over time. Data may include: + +> Existing Data. Each project may include a curation of datasets -- public and private -- that will support project objectives. The team prepares this curated list as a table, which includes data type, update frequency, access links, and contact information. + +> Digitized Government Data. Where needed, a project may also include guidance on government data digitalization and/or management, leveraging AI methods to make disaggregated government records readily searchable and usable. + +> New Data Collection. A Data Lab project may also incude a field data collection plan (and implementation of that plan, as needed) that includes some combination of household surveys, remote sensing (including drones), and crowdsourcing. Projects may also include guidance (and again, implementation of that guidance) on processing, storage, and cataloguing of all collected data. +> +> All Bank-produced datasets as part of the project can be hosted as a special collection on the World Bank's Data Catalogue, managed by the Development Economics Data Group (DECDG). The Catalogue receives more than 14 million unique users per month and will ensure value of the investment in data collection will be multiplied. + +2. **Analytics**. Leveraging curated datasets, the team conducts analytics across a range of topics (e.g., understanding population movement in response to a crisis or monitoring trends in nighttime lights). Each analysis will include original code, documentation, links to original data sources (and/or information on how to access them), and a description of their limitations. Reference resources are also cited, where relevant. + + + +3. **Additional Resources.** Links and descriptions of additional resources for each project may include: + - Description of common baseline data used to support the analyses -- administrative boundaries, population, infrastructure, etc. + + - Project SharePoint where original data and documents are maintained. + + - Additional static images and data visualizations. + + + +4. **Project Team**. For each project, the [World Bank Data Lab](https://wbdatalab.org/) recruits colleagues from throughout the World Bank, pooling our collective data talents in support of our lending and technical assistance operations. Project packages include names and contact information for the unique teams that prepared the analytics. + + + +## How Data Lab Projects are Managed + +1. **Dynamic, Web-Hosted Documentation**. Unless specified otherwise, all code and documentation used to produce the analytics is hosted in a project GitHub repository to facilitate reuse for future updates and projects, as well as to support collaboration and capacity building activities. + + + +2. **Data Catalogue**. Where possible, all datasets used in the production of Data Goods are added as entries to the World Bank’s [Development Data Hub](https://datacatalog.worldbank.org/home), where they are tagged with meta data, license attributes, and access information. + + + +3. **Internal Project Management and File Sharing System**. To facilitate project management across teams, the Lab creates a Project SharePoint, which includes project management information (work plan, milestones, check-in slides, log of hours charged, final report), related literature, data files, indicator tables, and links to resources, such as this documentation. The advantage of SharePoint for World Bank usage is that all contents are automatically encrypted and tagged as Official Use Only. The project SharePoint is accessible to project team members and, with permission, can be replicated as a basis for future project updates or for similar projects. diff --git a/docs/2-analytics.md b/docs/2-analytics.md new file mode 100644 index 0000000..d4aa072 --- /dev/null +++ b/docs/2-analytics.md @@ -0,0 +1,43 @@ +# Analytics: Introduction + +All Data Lab analytics include information on data sources, as well as original code and documentation. All analytics are presented in a web book format, per the following outline: + +1. **Overview** + + Summary of the analytical challenge. + + + +2. **Data Description** + Everything a user would need to access and use the data that supports the analysis. For each source, we include: + + - Description + + - License + + - Frequency of Access + + - Access Instructions + + - Point of Contact + + + +3. **Methodology** + All analytses include step-by-step directions, code snippets, links to complete code, and notes on any critical dependencies. The user should be able to fully understand how the analytical results were achieved and be able to replicate them by following the methodology. + + + +4. **Findings** + + This section includes initial results, including statistics, graphs, and maps to illustrate findings. + + + +5. **Limitations** + + It is critical that all analyses are accompanied by a detailed description of limitations of the data and methodology for interpreting or reproducing results. + + + +6. **References and Works Cited** diff --git a/docs/_toc.yml b/docs/_toc.yml index 4490233..3790d81 100644 --- a/docs/_toc.yml +++ b/docs/_toc.yml @@ -6,8 +6,9 @@ parts: numbered: False chapters: - file: reports/myanmar-economic-monitor-strategic-brief.md - - caption: Data Products + - caption: Analytics chapters: + - file: notebooks/2-analytics.md - file: notebooks/relative-wealth/relative-wealth-index.ipynb - file: notebooks/vegetation-conditions/README - file: notebooks/nighttime-lights/analysis-2023/README.md diff --git a/docs/introduction-to-data-goods.md b/docs/introduction-to-data-goods.md deleted file mode 100644 index e40b3a8..0000000 --- a/docs/introduction-to-data-goods.md +++ /dev/null @@ -1,23 +0,0 @@ -# Introduction to Data Goods - -**Data Goods** are comprised of data, reproducible methods (code), documentation, and sample insights. Unlike a traditional data analysis, which results in a single-use report or visualization, Data Goods are designed to be re-used for future updates and projects, thereby building the capacity of the World Bank and partner organizations to quickly and effectively deliver complex data science solutions to pressing global challenges. - -Data Goods packages include: - -1. [**Datasets**](docs/data.md): Datasets comprise *all* datasets used to prepare the Data Goods. To support replication and re-use of the Data Goods, the documentation includes a description of each datasource, including data type, update frequency, access links (including to the World Bank's data catalogue, the [Development Data Hub](https://datacatalog.worldbank.org/home)) and contact information. -

-2. **Data Products**: These are analytical products derived from the Foundational Datasets, which can be further used to generate indicators and insights. All data products include documentation, links to original data sources (and/or information on how to access them), and a description of their limitations. Reference resources are also cited, where relevant. In the documentation, each Data Product has it's own "chapter", generated through use of a Jupyter notebook. -

-3. **Insights and Indicators**: Each Data Goods package may also include additional analytical work, such as dynamic maps, data visualizaations, and/or sample indicators. Indicators can be derived from a combination of **Foundational Datasets** and **Data Products**. By combining these two inputs, teams are empowered to develop a large array of indicators to meet their project needs. Indicators can be presented side-by-side in an Excel workbook -- a format that is generally accessible to the widest audience. Because all indicators are based on the same underlying data, they are comparable with each other, across geographies and across time. -

-4. **Data Lab Team**: For each project, the [World Bank Data Lab](https://wbdatalab.org/) recruits colleagues from throughout our organization, pooling our collective great talents in support of our lending and technical assistance operations. Data Goods documentation includes a list and contact information for the unique team that prepared the Goods. -

- -## How Data Goods are Managed - -1. **Dynamic, Web-Hosted Documentation**. Unless specified otherwise, all code and documentation used to produce the Data Goods is hosted in a project GitHub repository, to facilitate reuse for future updates and projects, as well as to support collaboration and capacity building activities. -

-2. **Data Catalogue**. Where possible, all datasets used in the production of Data Goods are added as entries to the World Bank's [Development Data Hub](https://datacatalog.worldbank.org/home), where they are tagged with meta data, license attributes, and access information. -

-3. **Internal Project Management and File Sharing System**. To facilitate project management across teams, the Lab creates a Project SharePoint, which includes project management information (work plan, milestones, check-in slides, log of hours charged, final report), related literature, data files, indicator tables, and links to resources, such as this documentation. The advantage of SharePoint for World Bank usage is that all contents are automatically encrypted and tagged as Official Use Only. The project SharePoint is accessible to project team members and, with permission, can be replicated as a basis for future project updates or for similar projects. -

diff --git a/docs/team.md b/docs/team.md index f350309..64c2be4 100644 --- a/docs/team.md +++ b/docs/team.md @@ -1,6 +1,6 @@ -# Data Goods Team and Acknowledgements +# Project Team and Acknowledgements -The Data Lab would like to express our sincere gratutude and appreciation for the colleagues who worked together to prepare this Data Goods package: +The Data Lab would like to express our sincere gratutude and appreciation for the colleagues who worked together to prepare this project: | **Name** | **Role** | **Team** | | -------------------------------------------------- | ------------------------------------------------ | --------------------------- |