OpenAI just published a new research on interpretability of large language models (LLM), quoting its title, TLDR, and a brief summary:
"Title:
Language models can explain neurons in language models

TLDR:
We use GPT-4 to automatically write explanations for the behavior of neurons in large language models and to score those explanations. We release a dataset of these (imperfect) explanations and scores for every neuron in GPT-2.

Summary:
We propose an automated process that uses GPT-4 to produce and score natural language explanations of neuron behavior and apply it to neurons in another language model.
This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself. A promising aspect of this approach is that it scales with the pace of AI development. As future models become increasingly intelligent and helpful as assistants, we will find better explanations."

Later, someone commented: "Nice try. Understanding magic with the help of magic, you will get double portions of truth. [NAME_1]"

Questions:
1. Summarize the new research.
2. Give an in-depth explanation of the comment.
3. What is your opinion on this approach of LLM interpretability? There should be exactly 3 paragraphs separated by the markdown divider: ***.
user