gpt-tokenizer

v3.0.1

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

Install

npm install gpt-tokenizer

Bundle size

Dependencies

Dependencies(0)

Peer Dependencies(0)

Dev Dependencies(8)