Fairmodels produce issue for output variable as protected variable. #45
Description
Good day everyone
I am using the output variable of my data (isKilled= yes/no) as a protected variable and “yes” as a priviliged value. But every time I run the code, I just get a few metrics calculated and get the output as shown at the end.
Metric calculation : 2/12 metrics calculated for all models ( 10 NA created )
I used the following code, so could you please suggest where is the problem :
data = readARFF("apns.arff")
na.omit(data)
index= sample(1:nrow(data), 0.7*nrow(data))
train= data[index,]
test= data[-index,]
task = TaskClassif$new("data", backend = train, target = "isKilled")
learner= lrn("classif.randomForest", predict_type = "prob")
model= learner$train(task )
explainer = explain_mlr3(model,
data = test[,-15],
y = as.numeric(test$isKilled)-1,
label="RF")
prot <- ifelse(test$isKilled == 'no', 1, 0)
privileged <- '1'
privileged %in% as.factor(prot)
fc= fairness_check(explainer,
protected = prot,
privileged = privileged)
plot(fc)
msfc <- metric_scores(fc)
plot(msfc)
msfc$metric_scores_data
Output are as below:
msfc$metric_scores_data
score subgroup metric model
1 NA 0 TPR RF
2 0.90967742 1 TPR RF
5 0.00000000 0 PPV RF
6 1.00000000 1 PPV RF
11 0.06214689 0 FPR RF
12 NA 1 FPR RF
19 0.06214689 0 STP RF
20 0.90967742 1 STP RF
21 0.93785311 0 ACC RF
22 0.90967742 1 ACC RF