Non-Contextual Embeddings from Contextual Language Models

Making this topic for discussion on ways to get single-word embeddings from BERT and similar language models.