11월 7일, 8일 주제

11월 7일, 8일 주제 - ELECTRA

Data Scientist

by 자색 고구마칩 2021. 11. 7. 16:34

https://arxiv.org/abs/2003.10555

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Masked language modeling (MLM) pre-training methods such as BERT corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens. While they produce good results when transferred to downstream NLP tasks, the

arxiv.org

https://www.youtube.com/watch?v=BGRculoppT8&ab_channel=%EB%94%A5%EB%9F%AC%EB%8B%9D%EB%85%BC%EB%AC%B8%EC%9D%BD%EA%B8%B0%EB%AA%A8%EC%9E%84