Transformers.js 4 in a Vue app: embeddings and classification in the browser

Some AI features do not need a server. Telling whether a product review is positive, or finding the FAQ answer closest to a question, can run on the user's own device with a model of a few dozen megabytes. Transformers.js is the library that does this in JavaScript: it runs Hugging Face models through ONNX Runtime, on WebAssembly or on the GPU through WebGPU. Version 4.0.0 was published to npm on 30 March.

You will build a Vue app that classifies a review and ranks FAQ answers by meaning. Both models run in a web worker and download only when the user clicks, with a progress bar while they load. The project is a Vite app from npm create vue@latest with Vue 3.5.

What version 4 changes

The release notes lead with a new WebGPU runtime, rewritten in C++ together with the ONNX Runtime team and tested across about 200 model architectures. The same code now runs WebGPU-accelerated in browsers and in Node, Bun and Deno. The notes report a speed-up of about four times for BERT-based embedding models from a new attention operator, and support for models above 8 billion parameters.

The other changes you will notice:

• A ModelRegistry API that lists a pipeline's files, their sizes and whether they are cached, and a new progress_total event that reports loading progress across all files at once.
• New env settings: useWasmCache caches the runtime's WebAssembly files, fetch swaps in your own fetch function, and logLevel controls console output. ONNX Runtime's WebGPU warnings are now hidden by default.
• Tokenisation moved into its own package, @huggingface/tokenizers, which Transformers.js depends on.
• The build moved from webpack to esbuild, and transformers.web.js, the default browser bundle, is 53 percent smaller.

Some things stay as they were in 3.x:

• The same 25 pipeline tasks exist.
• In the browser the default device is still WebAssembly.
• The default precision is still q8 on WebAssembly and fp32 everywhere else.

One default did move: pipeline('feature-extraction') without a model name now loads onnx-community/all-MiniLM-L6-v2-ONNX instead of Xenova/all-MiniLM-L6-v2. This tutorial names its models, so the change does not affect it.

The release notes list no breaking changes, so upgrading is npm install @huggingface/transformers@4.

Sizes, limits and where the data goes

Two models from the Hugging Face Hub do the work. Each comes in several precisions, called dtype: fp32 stores each weight (a number the model learned) in 32 bits, fp16 in 16 bits, and q8 squeezes it into 8. The precision sets the download size:

• Xenova/distilbert-base-uncased-finetuned-sst-2-english classifies text as positive or negative: 68 MB as q8, 134 MB as fp16, 268 MB as fp32.
• Xenova/all-MiniLM-L6-v2 turns text into embeddings, lists of 384 numbers that place similar meanings close together: 23 MB as q8, 45 MB as fp16, 90 MB as fp32.

On top of that comes the ONNX Runtime WebAssembly file, about 23 MB (under 6 MB compressed), which Transformers.js fetches from the jsDelivr CDN. The model files come from huggingface.co. Everything is stored in the browser's Cache Storage under transformers-cache, so the second visit downloads nothing. The text the user types never leaves the device.

The limits are real:

• Both models are English.
• Input longer than 512 tokens (the word pieces a model reads) is cut off.
• The classifier only knows two labels. With the q8 model, a mixed review such as "The zip broke after a week, but customer service sent a new one straight away." comes back NEGATIVE with a score of 0.998. The score is the model's confidence in its label, not a measure of how mixed the text is.
• Neither is a chat model. They classify and compare; they do not write.

Use WebGPU only when the GPU supports fp16

fp16 halves the size of fp32 and is the natural choice on a GPU. It is also the easiest way to break the app:

const extractor = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', {
  device: 'webgpu',
  dtype: 'fp16',
})

On a GPU without the shader-f16 feature (support for 16-bit numbers) this throws The device (webgpu) does not support fp16., and in a browser without WebGPU it cannot run at all: the error there starts Unsupported device: "webgpu". Ask the GPU first, and fall back to WebAssembly with q8:

async function runtime() {
  const adapter = await navigator.gpu?.requestAdapter()
  return adapter?.features.has('shader-f16')
    ? ({ device: 'webgpu', dtype: 'fp16' } as const)
    : ({ device: 'wasm', dtype: 'q8' } as const)
}

A GPU without fp16 support falls back to WebAssembly with q8 rather than staying on WebGPU with fp32, on purpose: the fp32 files are four times the size (268 MB against 68 MB for the classifier).

navigator.gpu is typed in the DOM library of TypeScript 6.0, released on 23 March. On TypeScript 5.9, install the @webgpu/types package and add "@webgpu/types" to compilerOptions.types in tsconfig.app.json; TypeScript does not pick it up on its own.

Run the models in a web worker

Running a model blocks the thread it runs on for as long as it takes. On the main thread that freezes the page, so the models live in a web worker. Install the library first:

npm install @huggingface/transformers@4

Then create the worker file. It includes runtime() from the previous section:

import { pipeline, type ProgressInfo } from '@huggingface/transformers'

export type JobInput = { task: 'classify'; input: string } | { task: 'embed'; input: string[] }
export type Job = JobInput & { id: number }

async function runtime() {
  const adapter = await navigator.gpu?.requestAdapter()
  return adapter?.features.has('shader-f16')
    ? ({ device: 'webgpu', dtype: 'fp16' } as const)
    : ({ device: 'wasm', dtype: 'q8' } as const)
}

function onProgress(info: ProgressInfo) {
  if (info.status === 'progress_total') postMessage({ type: 'progress', percent: Math.round(info.progress) })
}

const loadClassifier = async () =>
  pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', { ...(await runtime()), progress_callback: onProgress })
const loadExtractor = async () =>
  pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', { ...(await runtime()), progress_callback: onProgress })

let classifier: ReturnType<typeof loadClassifier> | undefined
let extractor: ReturnType<typeof loadExtractor> | undefined

self.onmessage = async ({ data: job }: MessageEvent<Job>) => {
  try {
    let output: unknown
    if (job.task === 'classify') {
      classifier ??= loadClassifier()
      output = await (await classifier)(job.input)
    } else {
      extractor ??= loadExtractor()
      output = (await (await extractor)(job.input, { pooling: 'mean', normalize: true })).tolist()
    }
    postMessage({ type: 'result', id: job.id, output })
  } catch (error) {
    if (job.task === 'classify') classifier = undefined
    else extractor = undefined
    postMessage({ type: 'error', id: job.id, message: String(error) })
  }
}

What the less obvious lines do:

• Each model loads on the first job that needs it, and the worker keeps the loading promise, so a second job waits for the same download instead of starting another.
• If a job fails, the worker drops that model's promise so the next click can try again.
• progress_total arrives with a progress value from 0 to 100 that covers every file of the model.
• For embeddings, pooling: 'mean' averages the per-token vectors into one vector per text, normalize: true scales each to length 1, and tolist() turns the result into plain arrays that can cross to the main thread.

Wrap the worker in a composable

The component side needs three things: send a job, get its answer back as a promise, and see the progress. A composable wraps that, and it only creates the worker on the first job:

import { onScopeDispose, ref } from 'vue'
import type { Job, JobInput } from '@/ai/worker'

type Reply =
  | { type: 'progress'; percent: number }
  | { type: 'result'; id: number; output: unknown }
  | { type: 'error'; id: number; message: string }

export function useLocalModel() {
  const progress = ref(0)
  const pending = new Map<number, { resolve: (value: unknown) => void; reject: (error: Error) => void }>()
  let worker: Worker | undefined
  let nextId = 0

  function start() {
    const w = new Worker(new URL('../ai/worker.ts', import.meta.url), { type: 'module' })
    w.onmessage = ({ data }: MessageEvent<Reply>) => {
      if (data.type === 'progress') {
        progress.value = data.percent
        return
      }
      const job = pending.get(data.id)
      pending.delete(data.id)
      if (data.type === 'error') job?.reject(new Error(data.message))
      else job?.resolve(data.output)
    }
    return w
  }

  function send<T>(job: JobInput) {
    worker ??= start()
    const id = nextId++
    worker.postMessage({ ...job, id } satisfies Job)
    return new Promise<T>((resolve, reject) => {
      pending.set(id, { resolve: value => resolve(value as T), reject })
    })
  }

  onScopeDispose(() => worker?.terminate())

  return {
    progress,
    classify: (text: string) => send<{ label: string; score: number }[]>({ task: 'classify', input: text }),
    embed: (texts: string[]) => send<number[][]>({ task: 'embed', input: texts }),
  }
}

new Worker(new URL('../ai/worker.ts', import.meta.url), { type: 'module' }) is the pattern Vite recognises: it bundles the worker file and its imports separately. Vite 8 builds a worker that imports Transformers.js 4 with its default settings, no extra config.

Each job gets an id, and pending holds that job's resolve and reject until the worker replies with the same id, so each answer reaches the job that asked for it.

Because nothing is created until send() runs, opening the page costs nothing. Each component that calls useLocalModel() gets its own worker, and onScopeDispose terminates it when the component goes away, which also frees the model's memory.

Classify the sentiment of a review

The first component sends the review to classify() and shows the top label with its score:

<script setup lang="ts">
import { ref } from 'vue'
import { useLocalModel } from '@/composables/useLocalModel'

type Sentiment = { label: string; score: number }

const review = ref('')
const result = ref<Sentiment | null>(null)
const busy = ref(false)
const ready = ref(false)
const failed = ref(false)
const { progress, classify } = useLocalModel()

async function analyse() {
  busy.value = true
  failed.value = false
  try {
    const [top] = await classify(review.value)
    result.value = top ?? null
    ready.value = true
  } catch {
    failed.value = true
  } finally {
    busy.value = false
  }
}
</script>

<template>
  <label for="review">Your review</label>
  <textarea id="review" v-model="review" rows="4" />
  <button type="button" :disabled="busy || !review.trim()" @click="analyse">
    {{ ready ? 'Analyse' : 'Download the model and analyse' }}
  </button>
  <progress v-if="busy && !ready" :value="progress" max="100">{{ progress }}%</progress>
  <p v-if="failed" role="alert">The model could not run in this browser.</p>
  <p v-else-if="result">{{ result.label === 'POSITIVE' ? 'Positive' : 'Negative' }}, {{ Math.round(result.score * 100) }}% confident</p>
</template>

The classifier returns an array with the top label first, such as [{ label: 'POSITIVE', score: 0.9989 }] for "Not bad at all.", so it does handle simple negation. The button says what the first click will do. Put the size next to it in your own copy: 68 MB on WebAssembly, 134 MB on a GPU with fp16.

Rank FAQ answers by meaning

Semantic search compares meaning rather than words. Every FAQ answer becomes an embedding once, the question becomes one too, and the answers are ranked by how close their vectors are:

<script setup lang="ts">
import { computed, ref } from 'vue'
import { useLocalModel } from '@/composables/useLocalModel'

const faqs = [
  'You can return any item within 30 days for a full refund.',
  'Standard shipping takes three to five working days.',
  'Reset your password from the sign-in page with "Forgot password".',
  'We ship to the UK, the EU and the United States.',
]

const question = ref('')
const scores = ref<number[]>([])
const busy = ref(false)
const { progress, embed } = useLocalModel()
let faqVectors: number[][] | undefined

const dot = (a: number[], b: number[]) => a.reduce((sum, x, i) => sum + x * (b[i] ?? 0), 0)

async function search() {
  busy.value = true
  try {
    faqVectors ??= await embed(faqs)
    const [q] = await embed([question.value])
    if (q) scores.value = faqVectors.map(v => dot(q, v))
  } finally {
    busy.value = false
  }
}

const ranked = computed(() =>
  faqs.map((text, i) => ({ text, score: scores.value[i] ?? 0 })).sort((a, b) => b.score - a.score))
</script>

<template>
  <form @submit.prevent="search">
    <label for="faq-q">Ask a question</label>
    <input id="faq-q" v-model="question" type="search">
    <button type="submit" :disabled="busy || !question.trim()">Search</button>
  </form>
  <progress v-if="busy && !faqVectors" :value="progress" max="100">{{ progress }}%</progress>
  <ol v-if="scores.length">
    <li v-for="item in ranked" :key="item.text">{{ item.text }} ({{ item.score.toFixed(2) }})</li>
  </ol>
</template>

Cosine similarity measures the angle between two vectors. Because the worker normalised every vector to length 1, cosine similarity is simply the dot product, which is all dot computes.

With the q8 model, "Do you deliver to Germany?" scores 0.54 against the answer about shipping countries and 0.30 or less against the rest, though the two share no words. "I forgot my login" scores 0.60 against the password answer. Scores on a GPU with fp16 differ slightly in the second decimal.

Four answers fit in memory without thinking. A few thousand still do: each vector is 384 numbers, so a thousand answers take about 3 MB as plain JavaScript arrays. Embed them once, keep the vectors, and only embed the question on each search.