Posts by Tags

NAACL 2022

• We expand the scope of analysis from attention block in Transformers to the whole encoder.
• Our method significantly improves over existing techniques for quantifying global token attributions.
• We qualitatively demonstrate that the attributions obtained by our method are plausibly interpretable.
  
  read more read paper

EMNLP 2021 (BlackboxNLP)
In this work, we extend the probing studies to ELECTRA and XLNet, showing that variations in pre-training objectives can result in different behaviors in encoding linguistic information. We show that

• Weight mixing results in edge probing does not lead to reliable conclusions in layer-wise cross model analysis studies and MDL probing is more informative in this setup.
• XLNet accumulates linguistic knowledge in the earlier layers than BERT, whereas that of ELECTRA is in the final layers.
• ELECTRA undergoes a slight change during fine-tuning, whereas XLNet experiences significant adjustments.
  
  read more read paper poster

NAACL 2022

• We expand the scope of analysis from attention block in Transformers to the whole encoder.
• Our method significantly improves over existing techniques for quantifying global token attributions.
• We qualitatively demonstrate that the attributions obtained by our method are plausibly interpretable.
  
  read more read paper

EMNLP 2021 (BlackboxNLP)
In this work, we extend the probing studies to ELECTRA and XLNet, showing that variations in pre-training objectives can result in different behaviors in encoding linguistic information. We show that

• Weight mixing results in edge probing does not lead to reliable conclusions in layer-wise cross model analysis studies and MDL probing is more informative in this setup.
• XLNet accumulates linguistic knowledge in the earlier layers than BERT, whereas that of ELECTRA is in the final layers.
• ELECTRA undergoes a slight change during fine-tuning, whereas XLNet experiences significant adjustments.
  
  read more read paper poster

EMNLP 2021 (BlackboxNLP)
In this work, we extend the probing studies to ELECTRA and XLNet, showing that variations in pre-training objectives can result in different behaviors in encoding linguistic information. We show that

• Weight mixing results in edge probing does not lead to reliable conclusions in layer-wise cross model analysis studies and MDL probing is more informative in this setup.
• XLNet accumulates linguistic knowledge in the earlier layers than BERT, whereas that of ELECTRA is in the final layers.
• ELECTRA undergoes a slight change during fine-tuning, whereas XLNet experiences significant adjustments.
  
  read more read paper poster

NAACL 2022

• We expand the scope of analysis from attention block in Transformers to the whole encoder.
• Our method significantly improves over existing techniques for quantifying global token attributions.
• We qualitatively demonstrate that the attributions obtained by our method are plausibly interpretable.
  
  read more read paper