#topic/nlp

0.5 Tokenization

Tokenization Tokenization is a term for how a computer assigns a numerical value to an associated word. Because a deep learning model is unable to comprehend text directly, it must convert the…