How to generate the comparison viewer dashboard from predictions saved in DuckDB?

[adelinor]

Hi,

I would like to generate a comparison viewer dashboard from prediction results saved in a DuckDB file. Below are options I unsuccessfully explored. Is that possible at all?

Loading data from duckdb

PREDICT_TABLE='__splink__df_predict_123abcd'
con = duckdb.connect('mypredictionresultsdb')
df_predict = con.table(PREDICT_TABLE).df()

df = ... # Load data used to generate predictions
linker = DuckDBLinker(df, connection='mypredictionresults')
linker.load_settings_from_json('settings.json') 

linker.comparison_viewer_dashboard(df_predict, path, overwrite=True)

This fails because df_predict is not a SplinkDataFrame. Is there a way to create a SplinkDataFrame from a pandas data frame?

linker.query_sql

Creating a linker as before, the data gets loaded as a SplinkDataFrame via the linker:

df_predict = linker.query_sql(f"select * from {PREDICT_TABLE}", output_type='splink_df')

The linker.comparison_viewer_dashboard call fails with a different error this time:

duckdb.CatalogException: Catalog Error: Table with name __splink__df_predict does not exist!
Did you mean "__splink__df_predict_123abcd"?

Any guidance is very much appreciated.

Kind regards

[RossKen]

Hey @adelinor

I'm not 100% sure, but It looks like you may want the register table function.

So you would have:

df = ...
linker.register_table(df, "df")
linker = DuckDBLinker(df, connection='mypredictionresults')
linker.load_settings_from_json('settings.json') 

linker.comparison_viewer_dashboard(df_predict, path, overwrite=True)