Skip to content

improve behavior across multiple session #17

@piccolbo

Description

@piccolbo

What happens right now all tbls become stale, src becomes stale and one needs to refresh by hand.
Possible steps:

  • src stores creation call, uses it to refresh itself
  • tbl can temporary (TT) or persistent (PT). TT are just a backing for an in-memory object, PT have a life of their own. The table behind a TT should be destroyed at gc time, the one behind PT should outlive it. When a TT is saved, meaning it has a chance to be loaded in another session it should become a PT. The latter is problematic because the risk of leaving behind, but it mimicks the behavior for regular objects.
> z = data.frame(col1, col2)
> quit()
save? N
$ R
> #z is gone
> z = data.frame(col1, col2)
> quit()
save? Y
$ R
> print(z)
> #z is still there
> rm(z) #z gone

There's a third possibility where you can save z to a file and load it later. Now in an ideal world we can go through the same use case with a tbl_SparkSQL and never have z in some butchered state where it exists but is invalid. This may be a pipe-dream because I don't think save is a generic so that we can do something with it, but I thought I'd write it down.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions