Get all output as data frame

[jmgirard]

Description

It seems I can get item-level responses from shiny using sd_output() and I can pull all the data from the database using sd_get_data(). However, is there a convenient way to access just the current session's data as a data frame (i.e., the current user's row from sd_get_data())? Perhaps I can tap into what sd_output() is drawing from or modify the database query for sd_get_data()? My use case is that I want to collect data from a participant and then, on the final page, make a report describing their scores.

[jhelvy]

The unique identifier for each respondent is the session_id, which is just taken from session$token. So in your server() function you could try something like this:

data <- sd_get_data(db, refresh_interval = 5) # Refreshes the data every 5 seconds

# Do something with the data, like make a summary plot
output$plot <- renderPlot({
    data() |> # Note the () here, as this is a reactive expression
        filter(session_id == session$token) |>
        # More code to make a plot
})

Since data is a reactive expression, it has to be called inside a reactive environment, like a render function (e.g. renderPlot()) or reactive(), etc. In the code above I'm assuming you might make some sort of summary plot.

If you didn't set a refresh_interval, then the data would be just a static read, which wouldn't contain any of the current respondent's responses yet.

[jmgirard]

I figured there was something like that. This works well for the online setting, where we are writing to the database and can call from it. But I suppose this wouldn't work for the local/CSV version. But perhaps we could modify it in that case to read the CSV file. Is there an easy way to check if we are in "online" or "local" version, other than to parameterize it?

[jhelvy]

The words "local" and "online" are a little ambiguous here, because you can locally preview the app versus have it hosted online (e.g. on shinyapps.io), and you can have a database connection or not (with the app running locally or hosted online).

If you're talking about the case where you have the data stored locally in a data.csv file, then that means you're not actually storing anything other than the data for a single session. The only reason anything is written to data.csv is if you do not provide a database connection object to sd_server(). So for example, if you just set sd_server(db = NULL), the data will be written to the local data.csv file. Or if you set ignore = TRUE in sd_database(), then any of your database connection parameters will be ignored.

The only reason we write to data.csv is for you to be able to quickly preview how the data will look once it's stored in the database. But if you're trying to develop locally to do something interactive, you can still go ahead and set up a database connection with sd_database() and preview your app locally. You'll be able to see the data being stored in the database (e.g. on supabase), and you can also test the data being refreshed in the server.

[jmgirard]

I meant if ignore = TRUE, then we can't demo/test features that rely on sd_get_data(). Was thinking we could maybe pull from the csv in such cases. But the desire for this may be niche.

[jhelvy]

Ah okay I understand. Yeah that's not how we intended the data.csv file to work. It was just meant to allow people to preview responses. It only stores one row. We could try to modify it to update just like the database, but that's probably more work than it's worth.

I would just suggest working with a live database connection for local testing. That way you can get a better sense of what the user might see when the app is hosted live somewhere. You can work with any table name you want when setting up sd_database(), then when you're ready to go live just use a different table name. It will create a new table and your testing table can be ignored, no other changes are needed in the database.

Another reason it's probably better to use a live database connection for local testing is that when you first launch your survey, there won't be any data, which might cause an error. So you'll probably need some sort of logic for the very first time you run the app with a new table. Like, if you read in the data and nrow(data) == 0, then do something else, otherwise do the usual behavior. But it depends on what you're doing. In our reactive plotting demo, the plot isn't shown until a second page, so because the data is a reactive object and is continuously updated, you actually do get data after answering a question on the first page. So if you're doing something like presenting a summary of all their responses, then that should work so long as the summary doesn't trigger until the person answers some questions.

[jhelvy]

We changed how the local data works. It now saves as a file called preview_data.csv, making it a little more clear that this is for previewing purposes only. We also modified it so that multiple sessions will be stored in it, so if you restart your survey you should see a new row appear in the local file.