Why I rebuilt LIME and Anchors in R
If you want to see an explanation (draw it, put it in a tour, check it against the data), you need access to everything the explainer computed along the way. When I started my PhD, the existing R tools for two of the most popular local explainers didn’t give me that. So I wrote my own: kumquat for LIME and kultarr for Anchors.
These are chapters 4 and 5 of my thesis.
kumquat: a simpler lime
The lime package is a careful port of the original method, and it handles many data types. That flexibility comes at a cost: the intermediate pieces (the perturbed data, the weights given to each point, the fitted local model) are hidden. When an explanation looks odd, it’s hard to tell whether the model is odd or the explainer is.
kumquat is deliberately smaller. For tabular data it:
- samples perturbations from a small neighbourhood around the observation,
- gives every perturbed point a weight of 1, and
- fits an unregularised GLM (logistic regression for classification) as the local model, so every variable gets an importance score, however small.
What you get back is everything: the perturbation distribution, the black-box predictions, the local model’s predictions, the GLM object and its coefficients. You can audit exactly where an explanation came from.
It’s a separate package rather than a change to lime, so existing users’ results don’t shift under them. kumquat is meant for understanding and demonstration, not production.
install.packages("kumquat")
kultarr: Anchors, revisited
Anchors explains a prediction with a rule, such as “if x > 0.5 and y > 0.5, the model predicts U”, that holds with high precision near the observation. The existing R route to Anchors calls out to a Java implementation. Computation happens in a separate environment, which makes it slow to iterate on and hard to debug.
kultarr is Anchors in pure R:
- S7 classes represent anchors, so the data structure is explicit and easy to extend.
- Memoisation (
memoise) caches rewards for states it has already visited, which cuts repeated work on large datasets. - Parallel processing (
furrr) explains each observation independently. - Visualisation helpers bundle the bounds and aesthetic settings needed to draw anchors as boxes, including in high-dimensional tours.
pak::pak("janithwanni/kultarr")
library(kultarr)
library(randomForest)
set.seed(145)
train_data <- data.frame(x = runif(100), y = runif(100), z = runif(100))
train_data[21, ] <- c(0.5, 0.5, 0.5)
train_data$cls <- factor(ifelse(train_data$x > 0.5 & train_data$y > 0.5, "U", "D"))
rf_model <- randomForest(cls ~ ., data = train_data)
model_func <- carrier::crate(function(data) {
randomForest:::predict.randomForest(!!rf_model, data)
})
final_bounds <- make_anchors(
dataset = train_data,
cols = c("x", "y", "z"),
instance = train_data[21, ],
model_func = model_func,
class_col = "cls",
verbose = FALSE
)
final_bounds$final_anchor # bounds plus reward, precision and coverage
The point of it all
Neither package tries to be the fastest or most general explainer. They’re built so that nothing is hidden, because you can’t visualise or question what you can’t see. That’s what makes the geometric pictures in Seeing what the explainers see possible.
Feedback, issues and pull requests are very welcome on GitHub.