How Vite Works Under the Hood | York's Blog

Introduction

Vite is a development server based on the browser's native ES imports. It uses the browser to parse imports, compiles and returns them on-demand from the server, completely bypassing the concept of bundling, and starts the server for immediate use.

Today we are going to implement a mini-vite from scratch. The source code can be found here.

Implementation Steps

Project setup
Implement CLI
Start a static server, use nodemon to watch for file changes, and execute the Vite command
Handle index.html
Handle JS, handle the import of node_modules
Split middleware
Handle React files

Project Structure

├── \_example
├── cli
│ └── index.js
├── src
│ └── index.js

_example is a Vite project created using npx create-vite-app, used for comparison with mini-vite.

npx create-vite _example --template react

Implementing CLI

Create a new file cli/index.js

#! /usr/bin/env node
console.log("mini-vite!");

mini-vite/package.json

{
  "bin": "cli/index.js"
}

Link the CLI globally using yarn link

#  _demo/mini-vite directory
yarn link

Link in _example

#  _demo/mini-vite/_example directory
yarn link mini-vite

Add a script command in package.json

{
  "scripts": {
    "dev:mini-vite": "mini-vite"
  }
}

Run the dev:mini-vite command to see the output "mini-vite!" in the console.

Starting a Static Server

Install dependencies

yarn add koa koa-static

Create a new file index.js in the src directory

// src/index.js
const Koa = require('koa')
const KoaStatic = require('koa-static')

const app = new Koa()

// Path of the command execution
const rootPath = process.cwd()
app.use(KoaStatic(rootPath))

app.listen(8000, () => {
  console.log('mini-vite server started successfully!')
})

Add a script command in the _example package.json

{
  "scripts": {
    "dev:mini-vite": "nodemon -w ../ --exec mini-vite",
    "mini-vite": "mini-vite"
  }
}

Install nodemon

# mini-vite/_example
yarn add nodemon -D

Run the command and see the output "mini-vite server started successfully!" in the console. Open http://localhost:8000/ in the browser to see the project running. (Port 8000 is used here because the default port for create-vite-app is 3000, so we use 8000 here)

Modifying index.js will also show the change in the terminal.

Handling JSX

Now we can return static files, but after returning index.html, the browser immediately requests src/main.jsx

<script type="module" src="/src/main.jsx"></script>

This results in an error because the browser cannot parse JSX files. We need to handle .jsx by converting src/main.jsx to src/main.js.

main.jsx:1 Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/jsx". Strict MIME type checking is enforced for module scripts per HTML spec.

First, convert JSX.

Install dependencies in the mini-vite directory:

# mini-vite
yarn add  @babel/core @babel/plugin-transform-react-jsx

Add the transformJsx function

function transformJsx(jsxCode) {
  const babel = require('@babel/core')

  const options = {
    // presets: ['@babel/preset-env'], // Do not use @babel/preset-env here as it will convert all code to ES5, including imports
    plugins: [
      [
        '@babel/plugin-transform-react-jsx',
        {
          pragma: 'React.createElement',
          pragmaFrag: 'React.Fragment',
        },
      ],
    ],
  }

  const { code } = babel.transform(jsxCode, options)

  return code
}

Modify src/index.js, refactor slightly, and add middleware mechanism

// index.js
const Koa = require('koa')
const KoaStatic = require('koa-static')

function createServer() {
  const app = new Koa()

  const context = {
    app,
    rootPath: process.cwd(),
  }
  const resolvePlugins = [moduleRewirePlugin, serverStaticPlugin]

  resolvePlugins.forEach((plugin) => plugin(context))
}

createServer()

function serverStaticPlugin({ app, rootPath }) {
  app.use(KoaStatic(rootPath))
  app.use(KoaStatic(rootPath, '/public'))

  app.listen(8000, () => {
    console.log('mini-vite server started successfully!')
  })
}

function moduleRewirePlugin({ app, context }) {
  app.use(async (ctx, next) => {
    await next()
    if (ctx.body && ctx.response.is('jsx')) {
      // Initial ctx.body is a Readable stream, needs to be converted to a string
      const jsxCode = await readBody(ctx.body)
      // Convert JSX code with Babel
      const transformedCode = transformJsx(jsxCode)

      ctx.type = 'application/javascript'
      ctx.body = transformedCode
    }
  })
}

function transformJsx(jsxCode) {
  const babel = require('@babel/core')

  const options = {
    // presets: ['@babel/preset-env'], // Do not use @babel/preset-env here as it will convert all code to ES5, including imports
    plugins: [
      [
        '@babel/plugin-transform-react-jsx',
        {
          pragma: 'React.createElement',
          pragmaFrag: 'React.Fragment',
        },
      ],
    ],
  }

  const { code } = babel.transform(jsxCode, options)

  return code
}

function readBody(stream) {
  return new Promise((resolve, reject) => {
    if (!stream.readable) {
      resolve(stream)
    } else {
      let res = ''
      stream.on('data', (data) => {
        res += data
      })
      stream.on('end', () => {
        resolve(res)
      })
      stream.on('error', (err) => {
        reject(err)
      })
    }
  })
}

You can see that the browser successfully requested main.js, and our JSX syntax has been converted to React.createElement

But now the browser reports an error:

Uncaught TypeError: Failed to resolve module specifier "react". Relative references must start with either "/", "./", or "../".

The reason is that we imported react in main.js, but the browser cannot parse modules in node_modules, so we need to handle modules in node_modules.

Handling node_modules

Add a custom Babel plugin

module.exports = function ({ types: t }) {
  return {
    visitor: {
      ImportDeclaration(path, state) {
        const { node } = path
        const id = node.source.value
        // Simplified scenario: anything not starting with / . is a third-party module, without considering alias and other cases
        if (/^[^\/\.]/.test(id)) {
          node.source = t.stringLiteral('/@modules/' + id)
        }
      },
    },
  }
}

Corresponding handling on the server side

const customAliasPlugin = require('./babel-plugin-custom-alias')

const regex = /^\/@modules\//
function moduleResolvePlugin({ app, context }) {
  app.use(async (ctx, next) => {
    if (!regex.test(ctx.path)) {
      return next()
    }
    const id = ctx.path.replace(regex, '')
    console.log('id', id)

    const mapping = {
      // Read esm fields from package.json, simplified here, normally read esm exports from package.json
      react: path.resolve(process.cwd(), 'node_modules/react/index.js'),
      'react-dom/client': path.resolve(process.cwd(), 'node_modules/react-dom/client.js'),
    }

    ctx.type = 'application/javascript'

    const content = fs.readFileSync(mapping[id], 'utf-8')

    ctx.body = content
  })
}

The above operation encounters a problem: React does not provide an esm version!

Looking at the export field in React's official package.json

{
  "exports": {
    ".": {
      "react-server": "./react.shared-subset.js",
      "default": "./index.js"
    },
    "./package.json": "./package.json",
    "./jsx-runtime": "./jsx-runtime.js",
    "./jsx-dev-runtime": "./jsx-dev-runtime.js"
  }
}

Find the corresponding index.js

'use strict'

if (process.env.NODE_ENV === 'production') {
  module.exports = require('./cjs/react.production.min.js')
} else {
  module.exports = require('./cjs/react.development.js')
}

So now we have two choices:

1. Find a version with esm, such as https://github.com/esm-bundle/react
1. Use the original package but handle the cjs package on the server side to convert it to an esm package

Let's see how vite-plugin-react handles this issue. Go back to the browser and check the normal Vite packaged files.

import __vite__cjsImport0_react_jsxDevRuntime from '/node_modules/.vite/deps/react_jsx-dev-runtime.js?v=78b1e259'
const jsxDEV = __vite__cjsImport0_react_jsxDevRuntime['jsxDEV']
import __vite__cjsImport1_react from '/node_modules/.vite/deps/react.js?v=78b1e259'
const React = __vite__cjsImport1_react.__esModule
  ? __vite__cjsImport1_react.default
  : __vite__cjsImport1_react
import __vite__cjsImport2_reactDom_client from '/node_modules/.vite/deps/react-dom_client.js?v=78b1e259'
const ReactDOM = __vite__cjsImport2_reactDom_client.__esModule
  ? __vite__cjsImport2_reactDom_client.default
  : __vite__cjsImport2_reactDom_client
import App from '/src/App.jsx'
// ReactDOM.createRoot(document.getElementById("root")).render

Looking at node_modules/.vite/deps/react.js, it indeed copied the code and converted the cjs package to an esm package. This process is called optimizeDeps in Vite.

Handling CommonJS

Vite internally uses esbuild for handling this, but we will use Babel for this purpose.

yarn add @babel/core babel-plugin-transform-commonjs

🚧🚧🚧 Note: There are two packages

@babel/plugin-transform-modules-commonjs: Converts esm to cjs
@babel/plugin-transform-commonjs: Converts cjs to esm

Add a setupDevDepsAssets process when starting the service.

setupDevDepsAssets(process.cwd())

/**
 * Pre-build dependencies: convert third-party libraries such as react, react-dom, scheduler into ES Modules and write to a development temporary folder.
 * @param {*} rootPath
 */
function setupDevDepsAssets(rootPath) {
  // Check node_modules/.mini-vite
  const tempDevDir = path.resolve(rootPath, 'node_modules', '.mini-vite')

  if (!fs.existsSync(tempDevDir)) {
    fs.mkdirSync(tempDevDir)
  }

  // Convert third-party libraries like react, react-dom, scheduler into ES Modules and write to node_modules/.mini-vite directory
  // Simplified here; normally, start recursive dependency search from index.html and then convert
  const mapping = {
    react: {
      sourcePath: path.resolve(rootPath, 'node_modules/react/cjs/react.development.js'),
      targetPath: path.resolve(tempDevDir, 'react.js'),
    },
    'react-dom/client': {
      sourcePath: path.resolve(rootPath, 'node_modules/react-dom/cjs/react-dom.development.js'),
      targetPath: path.resolve(tempDevDir, 'react-dom.js'),
    },
    scheduler: {
      sourcePath: path.resolve(rootPath, 'node_modules/scheduler/cjs/scheduler.development.js'),
      targetPath: path.resolve(tempDevDir, 'scheduler.js'),
    },
  }

  Object.keys(mapping).forEach((key) => {
    const { sourcePath, targetPath } = mapping[key]
    transformCjsToEsm(sourcePath, targetPath)
  })

  /**
   * Convert CommonJS to ES Module; some third-party libraries, like React, do not provide ES Module versions.
   * @param {*} sourcePath
   * @param {*} targetPath
   */
  function transformCjsToEsm(sourcePath, targetPath) {
    const content = fs.readFileSync(sourcePath, 'utf-8')
    const babel = require('@babel/core')

    // Convert CommonJS code to esm
    const transformedCode = babel.transform(content, {
      plugins: ['transform-commonjs'],
    }).code

    // Path rewrite: convert require('react') to require('/@modules/react')
    // TODO: Merge the two code sections
    const pathRewritedCode = babel.transform(transformedCode, {
      plugins: [customAliasPlugin],
    }).code

    fs.writeFileSync(targetPath, pathRewritedCode)
  }
}

After adding this, check the network requests and you can see that react.js and react-dom.js have been successfully requested.

Console shows an error because React is not imported in App.js.

Automatic React Import

For those familiar with React, before React 17, we needed to manually import React when using React because JSX syntax would be converted to React.createElement.

import React from 'react'
function App() {
  return <div>hello world1</div>
}

However, after React 17, we no longer need to manually import React. You can check out the official introduction. React will automatically inject into the global scope, so we need to add the import for React in App.js.

Install package:

yarn add  @babel/plugin-transform-react-jsx-development

Add the logic for automatic import in the code transformation.

const transformedCode = babel.transform(jsxCode, {
  plugins: [
    '@babel/plugin-transform-react-jsx-development', // Import JSX
    customAliasPlugin,
  ],
}).code

You can see that the code has been successfully transformed.

Next, handle import { jsxDEV as _jsxDEV } from "/@modules/react/jsx-dev-runtime";

Add jsx-dev-runtime to the original mapping

mapping = {
  react: {
    sourcePath: path.resolve(rootPath, 'node_modules/react/cjs/react.development.js'),
    targetPath: path.resolve(tempDevDir, 'react.js'),
  },
  ['react/jsx-dev-runtime']: {
    sourcePath: path.resolve(
      rootPath,
      'node_modules/react/cjs/react-jsx-dev-runtime.development.js'
    ),
    targetPath: path.resolve(tempDevDir, 'jsx-dev-runtime.js'),
  },
}

You can see "hello world1" has been successfully rendered on the page.

References

vite-plugin-react

The demo corresponding to this section can be found here.