Skip to content

Commit

Permalink
Adds subsection on pickle and refers to it when pickle is mentioned
Browse files Browse the repository at this point in the history
  • Loading branch information
spjuhel committed Aug 21, 2024
1 parent 2736e62 commit dfff57d
Show file tree
Hide file tree
Showing 7 changed files with 32 additions and 11 deletions.
28 changes: 23 additions & 5 deletions doc/guide/Guide_Py_Performance.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,17 @@
"\n",
"This guide covers the following recommendations:\n",
"\n",
"⏲️ **Use profiling tools** to find and assess performance bottlenecks. \n",
"🔁 **Replace for-loops** by built-in functions and efficient external implementations. \n",
"📝 **Consider algorithmic performance**, not only implementation performance. \n",
"🧊 **Get familiar with NumPy:** vectorized functions, slicing, masks and broadcasting. \n",
"⚫ **Miscellaneous:** sparse arrays, Numba, parallelization, huge files (xarray), memory. \n",
"⏲️ **Use profiling tools** to find and assess performance bottlenecks.\n",
"🔁 **Replace for-loops** by built-in functions and efficient external implementations.\n",
"📝 **Consider algorithmic performance**, not only implementation performance.\n",
"🧊 **Get familiar with NumPy:** vectorized functions, slicing, masks and broadcasting.\n",
"⚫ **Miscellaneous:** sparse arrays, Numba, parallelization, huge files (xarray), memory, pickle format.\n",
"⚠️ **Don't over-optimize** at the expense of readability and usability."
]
},
{
"cell_type": "markdown",
"id": "33f970c4",
"metadata": {
"slideshow": {
"slide_type": "slide"
Expand Down Expand Up @@ -1095,6 +1096,23 @@
},
{
"cell_type": "markdown",
"id": "0ca35e9e",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"(saving-with-pickle)=\n",
"### Using pickle to save python objects\n",
"`pickle` is the standard python library [serialization module](https://docs.python.org/3/library/pickle.html).\n",
"It has the nice feature of being able to save most of python objects (standard and user defined) using simple methods.\n",
"However, pickle is *transient*, i.e. it has limited Portability: Pickle files are specific to the Python environment they were created in. This means that Pickle files may not be compatible across different Python versions or environments, which can make it challenging to share data between systems. As such it should only be used for temporary storage and not for persistent one."
]
},
{
"cell_type": "markdown",
"id": "de8dca2c",
"metadata": {
"slideshow": {
"slide_type": "slide"
Expand Down
2 changes: 1 addition & 1 deletion doc/tutorial/1_main_climada.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1082,7 +1082,7 @@
"\n",
">**Exercise:** Plot the impacts of Hurricane Maria. To do this you'll need to set `save_mat=True` in the earlier `ImpactCalc.impact()`.\n",
"\n",
"We can save our variables in pickle format using the `save` function and load them with `load`. This will save your results in the folder specified in the configuration file. The default folder is a `results` folder which is created in the current path (see default configuration file `climada/conf/defaults.conf`). However, we recommend to use CLIMADA's writers in `hdf5` or `csv` whenever possible."
"We recommend to use CLIMADA's writers in `hdf5` or `csv` whenever possible. It is also possible to save our variables in pickle format using the `save` function and load them with `load`. This will save your results in the folder specified in the configuration file. The default folder is a `results` folder which is created in the current path (see default configuration file `climada/conf/defaults.conf`). The pickle format has a [transient format](saving-with-pickle) and should be avoided when possible."
]
},
{
Expand Down
3 changes: 2 additions & 1 deletion doc/tutorial/climada_entity_DiscRates.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -179,9 +179,10 @@
},
{
"cell_type": "markdown",
"id": "37bf8fe8",
"metadata": {},
"source": [
"Pickle can always be used as well:"
"Pickle can always be used as well, but note that pickle has a [transient format](saving-with-pickle) and should be avoided when possible:"
]
},
{
Expand Down
3 changes: 2 additions & 1 deletion doc/tutorial/climada_entity_Exposures.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1757,9 +1757,10 @@
},
{
"cell_type": "markdown",
"id": "5d078d09",
"metadata": {},
"source": [
"Finallly, as with any Python object, use climada's save option to save it in pickle format."
"Finally, as with any Python object, use climada's save option to save it in pickle format. Note however, that pickle has a transient format and should be avoided when possible."
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion doc/tutorial/climada_entity_ImpactFuncSet.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,7 @@
"source": [
"#### Alternative saving format\n",
"\n",
"Alternatively, users may also save the impact functions into [pickle format](https://docs.python.org/3/library/pickle.html), using CLIMADA in-built function `save()`."
"Alternatively, users may also save the impact functions into [pickle format](https://docs.python.org/3/library/pickle.html), using CLIMADA in-built function `save()`. Note that pickle has a [transient format](saving-with-pickle) and should be avoided when possible."
]
},
{
Expand Down
3 changes: 2 additions & 1 deletion doc/tutorial/climada_entity_MeasureSet.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -620,9 +620,10 @@
},
{
"cell_type": "markdown",
"id": "adb8d606",
"metadata": {},
"source": [
"Pickle can always be used as well:"
"Pickle can be used as well, but note that pickle has a [transient format](saving-with-pickle) and should be avoided when possible:"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion doc/tutorial/climada_hazard_Hazard.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Pickle will work as well:"
"Pickle will work as well, but note that pickle has a [transient format](saving-with-pickle) and should be avoided when possible:"
]
},
{
Expand Down

0 comments on commit dfff57d

Please sign in to comment.