Skip to content

LENS Project

Thomas Fel, Thomas Serre
{thomas_fel}@brown.edu

Carney Institute for Brain Science, Brown University (Providence, USA),
DEEL Team - Artificial and Natural Intelligence Toulouse Institute

👋 Start exploring »

Goldfish concepts
Ostrich concepts
Loggerhead concepts

This project is the result of several articles, the most notable ones being
CRAFT · MACO · Holistic

This project aims to characterize the strategies, identify key features used by state-of-the-art models trained on ImageNet, and detect biases using the latest explainability methods: Concept-based explainability, Attribution methods, and Feature Visualization. We show that these approaches, far from being antagonistic, can be complementary in helping better understand models.

The illustrated model in this project is a ResNet50, where each class in ImageNet has its dedicated page highlighting the concepts used by the model to classify that particular class.

A normalized importance score is calculated for each concept, indicating the concept's significance for the class. For example, an importance level of 0.30 means that the concept contributes 30% of the sum of logits for all points classified as that class. The "LENS Method" page provides an introduction explaining how to interpret the results.

🤝 Contributors

This interactive website relies on numerous published studies, with each member considered a contributor to the project.

CRAFT: Thomas Fel, Agustin Picard, Louis Béthune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène & Thomas Serre.

MACO: Thomas Fel, Thibaut Boissin, Victor Boutin, Agustin Picard, Paul Novello, Julien Colin, Drew Linsley, Tom Rousseau, Rémi Cadène, Laurent Gardes & Thomas Serre.

Holistic: Thomas Fel, Victor Boutin, Mazda Moayeri, Rémi Cadène, Louis Béthune, Léo Andeol, Mathieu Chalvidal & Thomas Serre.

👀 See Also:

Furthermore, this work heavily builds on seminal research in explainable AI, specifically the work on concepts by Been Kim et al.1 and ACE2 for the automatic extraction of concept activation vectors (CAVs). More recently, the research on invertible concepts3 and their impressive human experiments.

Regarding the feature visualization, this work builds on the insightful articles published by the Clarity team at OpenAI4, notably the groundbreaking work by Chris Olah et al5. Similarly, their recent work on mechanistic interpretability7 and the concept of superposition6 has motivated us to explore dictionary learning methods.

Several articles have greatly inspired the development of the attribution method11 and importance estimation, ranging from attribution metrics10 12 13 to more recent theoretical insights 8 9.

A more comprehensive list of this foundational body of work is discussed in the three articles that form the foundation of our project.

🗞️ Citation

If you are using LENS as part of your workflow in a scientific publication, please consider citing one of the articles we build on:

@inproceedings{fel2023craft,
      title     = {CRAFT: Concept Recursive Activation FacTorization for Explainability},
      author    = {Thomas Fel and Agustin Picard and Louis Bethune and Thibaut Boissin
                  and David Vigouroux and Julien Colin and Rémi Cadène and Thomas Serre},
      year      = {2023},
      booktitle = {Proceedings of the IEEE Conference on Computer Vision and 
                   Pattern Recognition (CVPR)},
}
@article{fel2023holistic,
  title         = {A Holistic Approach to Unifying Automatic Concept Extraction 
                  and Concept Importance Estimation},
  author        = {Thomas Fel and Victor Boutin and Mazda Moayeri and Rémi Cadène and Louis Bethune
                  and Léo andéol and Mathieu Chalvidal and Thomas Serre},
  journal       = {Advances in Neural Information Processing Systems (NeurIPS)},
  year          = {2023}
}
@article{fel2023unlocking,
  title         = {Unlocking Feature Visualization for Deeper Networks with 
                  MAgnitude Constrained Optimization},
  author        = {Thomas Fel and Thibaut Boissin and Victor Boutin and Agustin Picard and
                  Paul Novello and Julien Colin and Drew Linsley and Tom Rousseau and
                  Rémi Cadène and Laurent Gardes, Thomas Serre},
  journal       = {Advances in Neural Information Processing Systems (NeurIPS)},
  year          = {2023}
}

📝 License

The package is released under MIT license.