Janith Wanniarachchi
← Blog

Why I rebuilt LIME and Anchors in R

· 3 min read

If you want to see an explanation (draw it, put it in a tour, check it against the data), you need access to everything the explainer computed along the way. When I started my PhD, the existing R tools for two of the most popular local explainers didn’t give me that. So I wrote my own: kumquat for LIME and kultarr for Anchors.

These are chapters 4 and 5 of my thesis.

kumquat: a simpler lime

The lime package is a careful port of the original method, and it handles many data types. That flexibility comes at a cost: the intermediate pieces (the perturbed data, the weights given to each point, the fitted local model) are hidden. When an explanation looks odd, it’s hard to tell whether the model is odd or the explainer is.

kumquat is deliberately smaller. For tabular data it:

What you get back is everything: the perturbation distribution, the black-box predictions, the local model’s predictions, the GLM object and its coefficients. You can audit exactly where an explanation came from.

It’s a separate package rather than a change to lime, so existing users’ results don’t shift under them. kumquat is meant for understanding and demonstration, not production.

install.packages("kumquat")

kultarr: Anchors, revisited

Anchors explains a prediction with a rule, such as “if x > 0.5 and y > 0.5, the model predicts U”, that holds with high precision near the observation. The existing R route to Anchors calls out to a Java implementation. Computation happens in a separate environment, which makes it slow to iterate on and hard to debug.

kultarr is Anchors in pure R:

pak::pak("janithwanni/kultarr")

library(kultarr)
library(randomForest)

set.seed(145)
train_data <- data.frame(x = runif(100), y = runif(100), z = runif(100))
train_data[21, ] <- c(0.5, 0.5, 0.5)
train_data$cls <- factor(ifelse(train_data$x > 0.5 & train_data$y > 0.5, "U", "D"))

rf_model <- randomForest(cls ~ ., data = train_data)
model_func <- carrier::crate(function(data) {
  randomForest:::predict.randomForest(!!rf_model, data)
})

final_bounds <- make_anchors(
  dataset = train_data,
  cols = c("x", "y", "z"),
  instance = train_data[21, ],
  model_func = model_func,
  class_col = "cls",
  verbose = FALSE
)
final_bounds$final_anchor  # bounds plus reward, precision and coverage

The point of it all

Neither package tries to be the fastest or most general explainer. They’re built so that nothing is hidden, because you can’t visualise or question what you can’t see. That’s what makes the geometric pictures in Seeing what the explainers see possible.

Feedback, issues and pull requests are very welcome on GitHub.