~lobsters | Bookmarks (3)

clear filters

go × thegreenplace.net ×

Tokens for LLMs: Byte Pair Encoding in Go - Eli Bendersky's website

thegreenplace.net

April 25, 2024 at 06:34 Tags Go , Machine Learning , WebAssembly A basic unit of...
April 25, 2024 at 06:34 Tags Go , Machine Learning , WebAssembly A basic unit of currency in modern LLMs is the token; exciting new models have long context windows of millions of tokens. API pricing for the large providers is per-token. We're even seeing the invention of new, derived units like TPM (tokens per minu...

ai go
2
gemini-cli: Access Gemini models from the command-line - Eli Bendersky's website

thegreenplace.net

ai go
2
Sign in with Google in Go - Eli Bendersky's website

thegreenplace.net

go
2