Use more text than one token to avoid neuralese

• AI News Team
• February 14, 2026

​Published on February 13, 2026 9:09 PM GMTYou want to relay the contents of a transformer’s output vector to the next input: next_input = encode(decode(output)).You’re currently using next_input = embed(sample_token(output)) to do this.This compresses the output to one row of a lookup table. That’s a pretty big squash. Too big, surely — there must be a better way.Enter neuralese. If you make next_input=output (or something like that) you lose no bandwidth at all. The bad news is that these vectors no longer correspond to natural language.But you can add more bandwidth by funneling each vector through more text. That way, encode(decode(output)) doesn’t lose too much information.You could have each vector decode to multiple tokens. Or even a cleverly chosen patch of bytes. Perhaps whole sentences, paragraphs, essays one day.I don’t know which is best, but you do have options here, so it’s not obvious to me that “you need more bandwidth” implies “you must abandon natural language” — unless you’re forcing it through a text intermediate as impoverished as, well, a lookup table.Discuss ​Read More