Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link 'Advice on writing efficient data...' in BigData:Data_storage to related pages #12

Open
hot007 opened this issue Sep 17, 2021 · 4 comments

Comments

@hot007
Copy link
Contributor

hot007 commented Sep 17, 2021

I suspect there will be other pages in this book and in https://acdguide.github.io/Governance/ that will have related material that should be linked under https://acdguide.github.io/BigData/data_storage.html#advice-on-writing-large-datasets-efficiently

@paolap
Copy link
Contributor

paolap commented Nov 8, 2021

Not much point in duplicate material, likewise Governance should refer here to show examples of what can happen when you don't do something.

@paolap
Copy link
Contributor

paolap commented Mar 10, 2022

There's another aspect to this, as writing the results of your computations to file can become a bottleneck,

see for examples climtas and this training

This could probably be added to the chunk section

@paigem
Copy link
Contributor

paigem commented Apr 14, 2022

Those are great resources @paolap - I agree that they should probably be added to the in-the-works chunking section. I'll link to the chunking issue for reference: #48

@hot007
Copy link
Contributor Author

hot007 commented Nov 24, 2022

I'm not sure I can find the relevant page anymore following restructure - however I think we should link to https://acdguide.github.io/Governance/introduction.html perhaps from the data intro page https://acdguide.github.io/BigData/data/data-intro.html, what do others think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants