BuildTools

Build tools

Our build tools support the development and publishing process. They are developed in a separate repository: https://github.com/open-pioneer/trails-build-tools ↗ (opens in a new tab)

Packages

@open-pioneer/build-support

This is a public package used directly in used code (import { defineBuildConfig } ...).

Defines the defineBuildConfig function used by the build.config.mjs configuration file. The provided types are the important bit here: they define the schema for configuration objects and support the developer with documentation.

@open-pioneer/vite-plugin-pioneer

This is a public package used directly in user code (in the vite.config.ts).

The central vite plugin required by our framework. This plugin does most of the heavy lifting when it comes to code generation and application metadata.

@open-pioneer/build-package-cli

Implements the build-pioneer-package CLI tool, which is used to compile a package into a publishable form.

@open-pioneer/build-package

Internal implementation of build-package. This is a library that could - in theory - be used to script the build process. It is just an internal artifact for now, however.

@open-pioneer/build-common

Internal helpers used by the vite plugin and the build-package tool. Reading and validation of the build.config.mjs are implemented here, as well as the metadata format used by build-package.

Build process

Overview

Every local package in an Open Pioneer Trails application contains a build.config.mjs and a package.json file, together defining the package’s metadata. Package metadata includes:

• The packages name, version, etc.
• Service(s) defined by the package
• Service(s) required by the package
• Supported languages and the location of the translation files
• Associated css styles
• …

All these data points (for every package!) are required by the framework at runtime in one way or another.

Thus, the framework’s job is, starting from an application, to detect the application’s dependencies, gather their metadata and prepare all information in a format suitable for the runtime environment (implemented in @open-pioneer/runtime).

Walkthrough

We will demonstrate the workings of the build plugin by using a simple example.

Consider an extremely simple application without any dependencies:

app.js

import { createCustomElement } from "@open-pioneer/runtime";
import * as appMetadata from "open-pioneer:app";

const Element = createCustomElement({
    appMetadata
});

customElements.define("empty-app", Element);

createCustomElement is a normal function that creates a custom element class, but it expects application metadata in a certain format (see typings). In a normal application, metadata will always be generated by the build plugin using the virtual module "open-pioneer:app". It can also be specified manually: this is done in the tests of the runtime package, for example.

The build plugin generates the application’s metadata by providing "open-pioneer:app" as a dynamic, virtual JavaScript module. When the virtual module is imported, the plugin is triggered (the resolveId and load hooks are called, see Documentation ↗ (opens in a new tab)). The plugin will then detect the location of the import (in resolveId()). This is the dynamic part, the content of the module depends on the importing app, i.e. the package the importing app.js belongs to.

From there on, the build plugin can trigger the analysis of the package itself (read package.json, build.config.js) and its entire dependency tree (walk through the dependencies declared in package.json, and their dependencies etc.). After that process, the build plugin knows the metadata about the entire application.

It will return a file such as the (simplified) one below from (load()), which delegates the actual code generation to other, also virtual modules. Note that all import paths and the precise format of the application’s metadata are an implementation detail; they are not stable.

// (simplified) content of "open-pioneer:app" for a specific application
import { createBox } from "/packages/runtime/metadata/index.ts";
import packages from "/apps/i18n-app/app.ts?open-pioneer-packages";
import stylesString from "/apps/i18n-app/app.ts?open-pioneer-styles&inline&lang.css";
import { locales, loadMessages } from "/apps/i18n-app/app.ts?open-pioneer-i18n-index";
const styles = createBox(stylesString);
export { packages, styles, locales, loadMessages };

As you can see, the plugin only outputs 4 different exports:

packages

This is a simple JSON-like structure containing metadata for every relevant package in the application.

Data points included are any service definitions, their classes (resolved via import) and their declared dependencies, which will eventually be used by Service Layer to start all required services.

A simplified packages structure looks like this:

// i18n-app/app.ts?open-pioneer-app
import { ExternalEventServiceImpl as _open_pioneer_integration_ExternalEventServiceImpl } from "/packages/integration/index.ts";
export default {
    // key: package name
    "i18n-app": {
        name: "i18n-app",
        services: {},
        ui: {
            // services referenced from the UI, from build.config.mjs
            references: [
                {
                    name: "integration.ExternalEventService",
                    qualifier: void 0,
                    all: false
                }
            ]
        },
        // Properties from build.config.mjs
        properties: {}
    },
    "@open-pioneer/integration": {
        name: "@open-pioneer/integration",
        // Service definitions from build.config.mjs, but class has been resolved (import is above)
        services: {
            ExternalEventServiceImpl: {
                name: "ExternalEventServiceImpl",
                clazz: _open_pioneer_integration_ExternalEventServiceImpl,
                provides: [
                    {
                        name: "integration.ExternalEventService",
                        qualifier: void 0
                    }
                ],
                references: {
                    ctx: {
                        name: "runtime.ApplicationContext",
                        qualifier: void 0,
                        all: false
                    }
                }
            }
        },
        ui: {
            references: []
        },
        properties: {}
    }
    // ...
};

styles

The build plugin, knowing all packages that make up the application, combines their style definitions into a single css value. That value is exported as a string from the virtual module, e.g.

// i18n-app/app.ts?open-pioneer-styles&inline&lang.css
export default ".sample-component-with-css {\n  color: red;\n  font-size: 1.25em;\n}\n";

The string is wrapped in an ObservableBox and not exported directly. During development, the box supports altering the inner value (and emits events when that occurs). That feature is used to support hot reloading for CSS during development and has no relevance for apps during production.

locales and loadMessages

The build plugin determines the set of supported languages for an app by inspecting the apps’ build.config.mjs. Those languages are transported as an array to the runtime environment. The runtime will then determine the application’s language and call loadMessages(lang) to load the i18n messages in the appropriate lang.

For example:

// i18n-app/app.ts?open-pioneer-i18n-index
export const locales = ["de", "en"];
export function loadMessages(locale) {
    switch (locale) {
        case "de":
            return import("i18n-app/app.ts?open-pioneer-i18n&locale=de").then((mod) => mod.default);
        case "en":
            return import("i18n-app/app.ts?open-pioneer-i18n&locale=en").then((mod) => mod.default);
    }
    throw new Error(`Unsupported locale: '${locale}'`);
}

The loadMessages function is generated from the same dataset (the array of supported languages) and contains a dynamic import ↗ (opens in a new tab) for every language.

When the application launches with language en, the function will trigger an import of the (again: virtual) module that contains the appropriate messages. This is done lazily through an import expression to ensure that no unneeded messages are loaded.

The module imported module for en looks like this:

const messages = JSON.parse(
    `{"i18n-app":{"content.header":"I18N Sample","content.description":"This example demonstrates an app that supports multiple languages.\\nThe app's actual language can be inferred from the user's environment (browser language) or by\\nexplicitly forcing the locale during application start.\\n","content.open":"Open dialog","picker.choose":"Select a language:","picker.default":"automatic (default)"}}`
);
export default messages;

Essentially, the i18n messages from every package for that language have been combined into a large JSON structure. That JSON structure will ultimately be used by the runtime to initialize the I18N framework.

Conclusion

You have now seen all the relevant pieces generated by the build plugin. Essentially, we have just extended vite with one important virtual module that provides the values required by our framework at runtime. Most complexity within the plugin comes from parsing, validating and caching the required bits and pieces. Vite handles everything else.

Build plugin internals

The build plugin is organized into two important subdirectories:

1. metadata

   Implements parsing, validation and caching for all relevant “facts” about applications and packages. The central class here is the MetadataRepository which provides access to package metadata, application metadata, and i18n files. It implements caching for an efficient development workflow: metadata and i18n files are only re-parsed when their underlying files have changed.

2. codegen

   Implements source code generation for the virtual modules provided by the plugin. We rely heavily on @babel/generator ↗ (opens in a new tab) and @babel/template ↗ (opens in a new tab) to achieve generation of valid JavaScript output.

   The input to the codegen functions is always one of the metadata types computed for application or package.

Further reading

• Rollup’s plugin API ↗ (opens in a new tab)
• Vite’s plugin API ↗ (opens in a new tab)
• Vite’s HMR API ↗ (opens in a new tab)