From the OpenNLP documents.
e.g. Tokenizer
"A tokenizer instance is not thread safe. For each thread one tokenizer must be instantiated which can share one
Actually, the whole point is
(1) Use a single instance (singleton) of Model file (e.g., TokenModel or POSModel POSModel pModel = new POSModel(new FileInputStream(posModelFile));) , which is shared by multiple threads
(2) In each thread, you can create multiple tokenizers or POStaggers ((e.g., POSTaggerME POSTagger = new POSTaggerME(pModel)). Their creation is light weight.
e.g. Tokenizer
"A tokenizer instance is not thread safe. For each thread one tokenizer must be instantiated which can share one
TokenizerModel
instance to safe memory."Actually, the whole point is
(1) Use a single instance (singleton) of Model file (e.g., TokenModel or POSModel POSModel pModel = new POSModel(new FileInputStream(posModelFile));) , which is shared by multiple threads
(2) In each thread, you can create multiple tokenizers or POStaggers ((e.g., POSTaggerME POSTagger = new POSTaggerME(pModel)). Their creation is light weight.
Comments
Post a Comment