How to create an agent and use with spark table and reuse agent

[joscani]

I would like create an agent with some validation rules and reuse agent for check a spark table. something like that

agent   <-  
  create_agent(tbl = name_of_table_or_conexion, 
               label = "valid_report") %>%
  col_vals_in_set(vars(segmento), set = c("No_way", "Neut", "Best")) %>%
  col_vals_between(vars(valor_cliente), left = 0, right = 4)

I would like save my agent validation rules and apply this rules to my_table. Ideally table change data every month and I would like to check if new data pass the rules

Something like that is possible ?

my_table <- sc %>% tbl("table_in_spark")

## apply agent 

agent %>% interrogate(my_table) 

Thanks

[rich-iannone]

There’s a function available for this exact use: set_tbl() (https://rich-iannone.github.io/pointblank/reference/set_tbl.html). With that you could define a different table for the agent object and interrogate again with the new table data.

Hope this helps!

[joscani]

Nice. Is there any way to create an angen without table in first instance. I mean, create agent only with validation rules , write in yml file and use this later where I set tbl with set_tbl. Thanks,

[joscani]

I've just seen x_write_disk and x_read_disk and it's useful to me . thanks