Skip to content

Remove set_meta methods that read whole files #20750

@mvdbeek

Description

@mvdbeek

or otherwise consume exorbitant amounts of memory or CPU time.

As an example here's the biom1 method:

def set_meta(self, dataset: DatasetProtocol, overwrite: bool = True, **kwd) -> None:
"""
Store metadata information from the BIOM file.
"""
if dataset.has_data():
with open(dataset.get_file_name()) as fh:
try:
json_dict = json.load(fh)
except Exception:
return

@ksuderman reported this one on matrix.

It's highly questionable if we want to determine metadata values that are very rarely, if ever, consumed in tools or the tool interface. Even allocating more memory to particular tools won't solve the issue, as you might create files via cat1 or similar general purpose tools. What's even worse that this is wrapped in try/except, so it's not even a guarantee that those values are really there, so what's the point ?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions