shiki-stream

Streaming highlighting with Shiki. Useful for highlighting text streams like LLM outputs.

Live Demo

Usage

Create a transform stream with CodeToTokenTransformStream and .pipeThrough your text stream:

import { createHighlighter, createJavaScriptRegexEngine } from 'shiki'
import { CodeToTokenTransformStream } from 'shiki-stream'

// Initialize the Shiki highlighter somewhere in your app
const highlighter = await createHighlighter({
  langs: [/* ... */],
  themes: [/* ... */],
  engine: createJavaScriptRegexEngine()
})

// The ReadableStream<string> you want to highlight
const textStream = getTextStreamFromSomewhere()

// Pipe the text stream through the token stream
const tokensStream = textStream
  .pipeThrough(new CodeToTokenTransformStream({
    highlighter,
    lang: 'javascript',
    theme: 'nord',
    allowRecalls: true, // see explanation below
  }))

allowRecalls

Due fact that the highlighting might be changed based on the context of the code, the themed tokens might be changed as the stream goes on. Because the streams are one-directional, we introduce a special "recall" token to notify the receiver to discard the last tokens that has changed.

By default, CodeToTokenTransformStream only returns stable tokens, no recalls. This also means the tokens are outputted less fine-grained, usually line-by-line.

For stream consumers that can handle recalls (e.g. our Vue / React components), you can set allowRecalls: true to get more fine-grained tokens.

Typically, recalls should be handled like:

const receivedTokens: ThemedToken[] = []

tokensStream.pipeTo(new WritableStream({
  async write(token) {
    if ('recall' in token) {
      // discard the last `token.recall` tokens
      receivedTokens.length -= token.recall
    }
    else {
      receivedTokens.push(token)
    }
  }
}))

Consume the Token Stream

Manually

tokensStream.pipeTo(new WritableStream({
  async write(token) {
    console.log(token)
  }
}))

Or in Node.js

for await (const token of tokensStream) {
  console.log(token)
}

Vue

<script setup lang="ts">
import { ShikiStreamRenderer } from 'shiki-stream/vue'

// get the token stream
</script>

<template>
  <ShikiStreamRenderer :stream="tokensStream" />
</template>

React

import { ShikiStreamRenderer } from 'shiki-stream/react'

export function MyComponent() {
  // get the token stream
  return <ShikiStreamRenderer stream={tokensStream} />
}

Sponsors

License

MIT License © Anthony Fu