Skip to content

Add the Dataptr* methods to ALTREP representation #286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 30, 2025

Conversation

tzakharko
Copy link
Contributor

When processing large amounts on data using the parallel package, the session occasionally errors out complaining about missing Dataptr method. It is not easily reproducible, and happens during the collection phase. This patch adds the missing methods.

Copy link
Collaborator

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

It's tricky to test this, but I believe that the_list[[0]] <- the_list[[1]] will trigger SetElt. For strings sort() and unique() will trigger Dataptr() but I think those will just error here. Perhaps a stub Rcpp function whose only job is to call DATAPTR_RO(input) would be a way to make sure this works from our test suite.

}

static void* s2_altrep_Dataptr(SEXP obj, Rboolean writable) {
if (writable) return NULL;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure that you can return NULL here. I don't think it's harmful to skip this check, though, since we allocate all of the lists that we wrap in this ALTREP class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like a defect in R spec to me. Technically, DATAPTR is non-api (so I can't use it here), and returning writeable pointers for ALTREP lists is not supported at the API level. Since we seem to be entering the realm of undefined behavior, I think that returning NULL is fine. I also dislike casing away const pointer, but I don't really see a way around it in the current API.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely a defect/inconsistency in R. Instead of returning NULL (which might trigger a segfault in current or future code), I think Rf_error() is a better choice.

@tzakharko
Copy link
Contributor Author

I've added some simple tests.

}

static void* s2_altrep_Dataptr(SEXP obj, Rboolean writable) {
if (writable) return NULL;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely a defect/inconsistency in R. Instead of returning NULL (which might trigger a segfault in current or future code), I think Rf_error() is a better choice.

return (double) ((uintptr_t) DATAPTR_RO(obj));
}")

expect_no_error(get_dataptr(as_s2_geography("POINT (0 0)")))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
expect_no_error(get_dataptr(as_s2_geography("POINT (0 0)")))
expect_true(get_dataptr(as_s2_geography("POINT (0 0)")) != 0)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the test function result to boolean directly as this is more portable

Copy link
Collaborator

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@paleolimbot paleolimbot merged commit 4f2ffae into r-spatial:main Jun 30, 2025
7 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants