tokenizers.bpe

R 패키지 메타데이터와 수집 신호를 모아 봅니다.

Packages / CRAN / tokenizers.bpe

tokenizers.bpe

v0.1.5

tokenizers...

Repository CRANLicense MPL-2.0Lifecycle activeNeeds compilation yes

Package page Reference Source DOI README NEWS

DOI

10.32614/CRAN.package.tokenizers.bpe

Task views

Natural Language Processing

Core Signals

첫 화면에서 판단해야 할 수집 신호를 먼저 배치합니다.

Task views

Natural Language Processing

Supported Backends

DESCRIPTION에서 감지한 backend 관련 package입니다.

backend package 신호가 없습니다.

Quick Facts

기본 메타데이터를 작은 카드와 토큰으로 압축합니다.

profile

Repository

CRAN

Version

0.1.5

License

MPL-2.0

Lifecycle

active

Needs compilation

yes

Last observed

2026-05-30

CRAN: cran.r-project.org/package=tokenizers.bpe

Build fields

LinkingTo

Rcpp

수집 소스별 패키지 정보

1개 소스

CRAN

0.1.5

2026-05-30

License

MPL-2.0

Depends

R (>= 2.10)

Imports

Rcpp (>= 0.11.5)

LinkingTo

Rcpp

Needs compilation

yes

Lifecycle

active

Last observed

2026-05-30 10:45:11

이 패키지가 의존하는 패키지

2개 표시전체 2개

Package	Type	Spec
Rcpp CRAN · 0.1.5 · 2026-05-30	Imports	Rcpp (>= 0.11.5)
Rcpp CRAN · 0.1.5 · 2026-05-30	LinkingTo	Rcpp
검색 결과가 없습니다.

1 / 1

이 패키지를 쓰는 패키지

3개 표시전체 3개

Package	Type	Spec
doc2vec 0.2.2 CRAN · 2026-05-30	Suggests	tokenizers.bpe
sentencepiece 0.2.5 CRAN · 2026-05-30	Suggests	tokenizers.bpe
textrecipes 1.1.0 CRAN · 2026-05-30	Suggests	tokenizers.bpe
검색 결과가 없습니다.

1 / 1

Reverse dependency summary

1 types

Type	Packages
Suggests	3

패키지 페이지

Reverse suggests

All links

Repository: CRAN
Version: 0.1.4
Collected: 2026-05-19 20:35:12
Package page: https://cran.r-project.org/web/packages/tokenizers.bpe/index.html
DOI: 10.32614/CRAN.package.tokenizers.bpe
CRAN checks: https://cran.r-project.org/web/checks/check_results_tokenizers.bpe.html
README: https://cran.r-project.org/web/packages/tokenizers.bpe/readme/README.html
NEWS: https://cran.r-project.org/web/packages/tokenizers.bpe/news/news.html
Reference HTML: https://cran.r-project.org/web/packages/tokenizers.bpe/refman/tokenizers.bpe.html
Reference PDF: https://cran.r-project.org/web/packages/tokenizers.bpe/tokenizers.bpe.pdf
Source package: https://cran.r-project.org/src/contrib/tokenizers.bpe_0.1.4.tar.gz
Archive: https://CRAN.R-project.org/src/contrib/Archive/tokenizers.bpe

In views

NaturalLanguageProcessing

Page fields

Author: Jan Wijffels [aut, cre, cph] (R wrapper), BNOSAC [cph] (R wrapper), VK.com [cph], Gregory Popovitch [ctb, cph] (Files at src/parallel_hashmap (Apache License, Version 2.0), The Abseil Authors [ctb, cph] (Files at src/parallel_hashmap (Apache License, Version 2.0), Ivan Belonogov [ctb, cph] (Files at src/youtokentome (MIT License))
CRAN Checks: tokenizers.bpe results [issues need fixing before 2026-06-05]
DOI: 10.32614/CRAN.package.tokenizers.bpe
In Views: NaturalLanguageProcessing
License: MPL-2.0
LinkingTo: Rcpp
Maintainer: Jan Wijffels <jwijffels at bnosac.be>
Materials: README , NEWS
NeedsCompilation: yes
Old Sources: tokenizers.bpe archive
Package Source: tokenizers.bpe_0.1.4.tar.gz
Published: 2025-09-05
Reference Manual: tokenizers.bpe.html , tokenizers.bpe.pdf
Reverse Suggests: doc2vec , sentencepiece , textrecipes
URL: https://github.com/bnosac/tokenizers.bpe
Version: 0.1.4
Windows Binaries: r-devel: tokenizers.bpe_0.1.4.zip , r-release: tokenizers.bpe_0.1.4.zip , r-oldrel: tokenizers.bpe_0.1.4.zip
MacOS Binaries: r-release (arm64): tokenizers.bpe_0.1.4.tgz , r-oldrel (arm64): tokenizers.bpe_0.1.4.tgz , r-release (x86_64): tokenizers.bpe_0.1.4.tgz , r-oldrel (x86_64): tokenizers.bpe_0.1.4.tgz

Version

0.1.4

LinkingTo

Rcpp

Published

2025-09-05

DOI

10.32614/CRAN.package.tokenizers.bpe

Author

Jan Wijffels [aut, cre, cph] (R wrapper), BNOSAC [cph] (R wrapper), VK.com [cph], Gregory Popovitch [ctb, cph] (Files at src/parallel_hashmap (Apache License, Version 2.0), The Abseil Authors [ctb, cph] (Files at src/parallel_hashmap (Apache License, Version 2.0), Ivan Belonogov [ctb, cph] (Files at src/youtokentome (MIT License))

Maintainer

Jan Wijffels <jwijffels at bnosac.be>

License

MPL-2.0

URL

https://github.com/bnosac/tokenizers.bpe

NeedsCompilation

yes

Materials

README , NEWS

README NEWS

In Views

NaturalLanguageProcessing

CRAN Checks

tokenizers.bpe results [issues need fixing before 2026-06-05]

tokenizers.bpe results

Reference Manual

tokenizers.bpe.html , tokenizers.bpe.pdf

tokenizers.bpe.html tokenizers.bpe.pdf

Package Source

tokenizers.bpe_0.1.4.tar.gz

Windows Binaries

r-devel: tokenizers.bpe_0.1.4.zip , r-release: tokenizers.bpe_0.1.4.zip , r-oldrel: tokenizers.bpe_0.1.4.zip

tokenizers.bpe_0.1.4.zip tokenizers.bpe_0.1.4.zip tokenizers.bpe_0.1.4.zip

MacOS Binaries

r-release (arm64): tokenizers.bpe_0.1.4.tgz , r-oldrel (arm64): tokenizers.bpe_0.1.4.tgz , r-release (x86_64): tokenizers.bpe_0.1.4.tgz , r-oldrel (x86_64): tokenizers.bpe_0.1.4.tgz

tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz

Old Sources

tokenizers.bpe archive

Reverse Suggests

doc2vec , sentencepiece , textrecipes

doc2vec sentencepiece textrecipes

Page sections 4

Documentation

Heading

Documentation

Links

[{"label":"tokenizers.bpe.html","section":"","type":"","url":"https://cran.r-project.org/web/packages/tokenizers.bpe/refman/tokenizers.bpe.html"},{"label":"tokenizers.bpe.pdf","section":"","type":"","url":"https://cran.r-project.org/web/packages/tokenizers.bpe/tokenizers.bpe.pdf"}]

Text

Reference manual: tokenizers.bpe.html , tokenizers.bpe.pdf

Downloads

Heading

Downloads

Links

[{"label":"tokenizers.bpe_0.1.4.tar.gz","section":"","type":"","url":"https://cran.r-project.org/src/contrib/tokenizers.bpe_0.1.4.tar.gz"},{"label":"tokenizers.bpe_0.1.4.zip","section":"","type":"","url":"https://cran.r-project.org/bin/windows/contrib/4.7/tokenizers.bpe_0.1.4.zip"},{"label":"tokenizers.bpe_0.1.4.zip","section":"","type":"","url":"https://cran.r-project.org/bin/windows/contrib/4.6/tokenizers.bpe_0.1.4.zip"},{"label":"tokenizers.bpe_0.1.4.zip","section":"","type":"","url":"https://cran.r-project.org/bin/windows/contrib/4.5/tokenizers.bpe_0.1.4.zip"},{"label":"tokenizers.bpe_0.1.4.tgz","section":"","type":"","url":"https://cran.r-project.org/bin/macosx/sonoma-arm64/contrib/4.6/tokenizers.bpe_0.1.4.tgz"},{"label":"tokenizers.bpe_0.1.4.tgz","section":"","type":"","url":"https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.5/tokenizers.bpe_0.1.4.tgz"},{"label":"tokenizers.bpe_0.1.4.tgz","section":"","type":"","url":"https://cran.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.6/tokenizers.bpe_0.1.4.tgz"},{"label":"tokenizers.bpe_0.1.4.tgz","section":"","type":"","url":"https://cran.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/tokenizers.bpe_0.1.4.tgz"},{"label":"tokenizers.bpe archive","section":"","type":"","url":"https://CRAN.R-project.org/src/contrib/Archive/tokenizers.bpe"}]

Text

Package source: tokenizers.bpe_0.1.4.tar.gz Windows binaries: r-devel: tokenizers.bpe_0.1.4.zip , r-release: tokenizers.bpe_0.1.4.zip , r-oldrel: tokenizers.bpe_0.1.4.zip macOS binaries: r-release (arm64): tokenizers.bpe_0.1.4.tgz , r-oldrel (arm64): tokenizers.bpe_0.1.4.tgz , r-release (x86_64): tokenizers.bpe_0.1.4.tgz , r-oldrel (x86_64): tokenizers.bpe_0.1.4.tgz Old sources: tokenizers.bpe archive

Reverse dependencies

Heading

Reverse dependencies

Links

[{"label":"doc2vec","section":"","type":"","url":"https://cran.r-project.org/web/packages/doc2vec/index.html"},{"label":"sentencepiece","section":"","type":"","url":"https://cran.r-project.org/web/packages/sentencepiece/index.html"},{"label":"textrecipes","section":"","type":"","url":"https://cran.r-project.org/web/packages/textrecipes/index.html"}]

Text

Reverse suggests: doc2vec , sentencepiece , textrecipes

Linking

Heading

Linking

Links

[{"label":"https://CRAN.R-project.org/package=tokenizers.bpe","section":"","type":"","url":"https://CRAN.R-project.org/package=tokenizers.bpe"}]

Text

Please use the canonical form https://CRAN.R-project.org/package=tokenizers.bpe to link to this page.

Materials 2

README NEWS

Documentation 2

tokenizers.bpe.html tokenizers.bpe.pdf

Downloads 9

tokenizers.bpe_0.1.4.tar.gz tokenizers.bpe_0.1.4.zip tokenizers.bpe_0.1.4.zip tokenizers.bpe_0.1.4.zip tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz tokenizers.bpe_0.1.4.tgz tokenizers.bpe archive

All page links 23

버전 이력

Repository	Version	Published	First seen	Last seen	Docs
CRAN	0.1.5		2026-05-29	2026-05-30
CRAN	0.1.4		2026-05-21	2026-05-21

보안

표시할 OSV 데이터가 없습니다.

문헌 신호

표시할 OpenAlex 데이터가 없습니다.

CRAN Task Views

Natural Language Processing · 2026-05-30