Nvtext API

组 NVText

目录

• Nvtext Ngrams
  • generate_ngrams()
  • generate_character_ngrams()
  • hash_character_ngrams()
  • ngrams_tokenize()
• Nvtext 规范化
  • normalize_spaces()
  • normalize_characters()
  • create_character_normalizer()
  • normalize_characters()
  • nvtext::character_normalizer
• Nvtext 词干提取
  • letter_type
  • is_letter()
  • is_letter()
  • porter_stemmer_measure()
• Nvtext 编辑距离
  • edit_distance()
  • edit_distance_matrix()
• Nvtext 分词
  • load_merge_pairs()
  • byte_pair_encoding()
  • load_vocabulary_file()
  • subword_tokenize()
  • tokenize()
  • tokenize()
  • count_tokens()
  • count_tokens()
  • character_tokenize()
  • detokenize()
  • load_vocabulary()
  • tokenize_with_vocabulary()
  • load_wordpiece_vocabulary()
  • wordpiece_tokenize()
  • nvtext::bpe_merge_pairs
  • nvtext::hashed_vocabulary
  • nvtext::tokenizer_result
  • nvtext::tokenize_vocabulary
  • nvtext::wordpiece_vocabulary
• Nvtext 替换
  • replace_tokens()
  • filter_tokens()
• Nvtext Minhash
  • minhash()
  • minhash64()
  • minhash_ngrams()
  • minhash64_ngrams()
• Nvtext Jaccard
  • jaccard_index()