Creating Interactive VR Panels with ThreeMeshUI in Three.js

Virtual Reality (VR) experiences in Three.js can be significantly enhanced by adding interactive UI elements. ThreeMeshUI is a powerful library that enables the creation of responsive, lightweight, and flexible 3D user interfaces in WebXR environments.

In this article, we will explore how ThreeMeshUI is initialized and used to create interactive panels in a VR scene.

1. Setting Up ThreeMeshUI in Three.js

1.1 Importing Required Libraries

Before using ThreeMeshUI, ensure you have the necessary imports:

import * as THREE from "three";
import ThreeMeshUI from "three-mesh-ui";
import FontJSON from "/Static/Fonts/Roboto-msdf.json";
import FontImage from "/Static/Fonts/Roboto-msdf.png";

ThreeMeshUI relies on multi-channel signed distance field (MSDF) fonts for rendering text efficiently in 3D. The font JSON and texture image files are necessary to define the appearance of the text.

2. Creating a VR Panel with ThreeMeshUI

2.1 Initializing the Panel Data

A list of UI panels is stored in an array, panelData, containing details about each panel’s position, header text, correct answer, and interactivity.

let panelData = [
  {
    position: [-15, 5, -12],
    header: "Extension Board",
    correctBtn: "Parallel",
    lockPanel: false,
  },
  {
    position: [-5, 5, -12],
    header: "Fuse",
    correctBtn: "Series",
    lockPanel: false,
  },
  {
    position: [5, 5, -12],
    header: "Decoration Lights",
    correctBtn: "Series",
    lockPanel: false,
  },
  {
    position: [15, 5, -12],
    header: "Switches",
    correctBtn: "Parallel",
    lockPanel: false,
  },
];

2.2 Creating the Panels

Each panel consists of a header and interactive buttons. The createMeshPanel function iterates through the panelData and generates a UI panel for each item.

function createMeshPanel() {
  panelData.forEach((data, index) => {
    let { header, container, buttons } = meshPanelConfig(
      data.position,
      data.header,
      "Series",
      "Parallel",
      () => {
        let btnKey = `Series ${index + 1}`;
        if (!clickedButtons.has(btnKey)) {
          clickedButtons.add(btnKey);
          clicked = true;
          let btnClicked = "Series";
          scoreCalculator(index + 1, data.header, btnClicked);
        }
      },
      () => {
        let btnKey = `Parallel ${index + 1}`;
        if (!clickedButtons.has(btnKey)) {
          clickedButtons.add(btnKey);
          clicked = true;
          let btnClicked = "Parallel";
          scoreCalculator(index + 1, data.header, btnClicked);
        }
      }
    );

    scene.add(header);
    scene.add(container);
    objsToTest.push(...buttons);
  });
}

2.3 Configuring Each Panel

The meshPanelConfig function generates a ThreeMeshUI block containing a title and two answer buttons.

function meshPanelConfig(
  position,
  headerText,
  label1,
  label2,
  callback1,
  callback2
) {
  let header = new ThreeMeshUI.Block({
    width: 3,
    height: 0.5,
    borderRadius: 0.25,
    justifyContent: "center",
    fontFamily: FontJSON,
    fontTexture: FontImage,
    fontSize: 0.4,
  }).add(
    new ThreeMeshUI.Text({
      content: headerText,
    })
  );

  header.position.set(...position);
  header.rotation.y = Math.PI / 6;

  let container = new ThreeMeshUI.Block({
    width: 3,
    height: 1,
    padding: 0.1,
    borderRadius: 0.25,
    justifyContent: "center",
    fontFamily: FontJSON,
    fontTexture: FontImage,
    fontSize: 0.3,
    flexDirection: "row",
    backgroundOpacity: 0.5,
  });

  container.position.set(position[0], position[1] - 1, position[2]);
  container.rotation.y = Math.PI / 6;

  let btn1 = new ThreeMeshUI.Block({
    width: 1.2,
    height: 0.4,
    justifyContent: "center",
    borderRadius: 0.2,
    backgroundColor: new THREE.Color(0x00ff00),
  }).add(
    new ThreeMeshUI.Text({
      content: label1,
    })
  );

  let btn2 = new ThreeMeshUI.Block({
    width: 1.2,
    height: 0.4,
    justifyContent: "center",
    borderRadius: 0.2,
    backgroundColor: new THREE.Color(0xff0000),
  }).add(
    new ThreeMeshUI.Text({
      content: label2,
    })
  );

  btn1.setupState({
    state: "hovered",
    attributes: {
      backgroundColor: new THREE.Color(0x88ff88),
    },
  });

  btn2.setupState({
    state: "hovered",
    attributes: {
      backgroundColor: new THREE.Color(0xff8888),
    },
  });

  btn1.addEventListener("click", callback1);
  btn2.addEventListener("click", callback2);

  container.add(btn1, btn2);

  return { header, container, buttons: [btn1, btn2] };
}

3. Adding Interaction to UI Buttons

ThreeMeshUI allows interactive states like hover and click. The setupState function changes the button color on hover.

btn1.setupState({
  state: "hovered",
  attributes: {
    backgroundColor: new THREE.Color(0x88ff88),
  },
});

btn2.setupState({
  state: "hovered",
  attributes: {
    backgroundColor: new THREE.Color(0xff8888),
  },
});

Adding an event listener to buttons enables interaction:

btn1.addEventListener("click", callback1);
btn2.addEventListener("click", callback2);

When a button is clicked, it triggers the appropriate callback function to check the answer.

4. Rendering the UI in the Animation Loop

To ensure ThreeMeshUI elements update correctly in Three.js, add this inside the render loop:

function animate() {
  requestAnimationFrame(animate);
  ThreeMeshUI.update();
  renderer.render(scene, camera);
}

Conclusion

Using ThreeMeshUI in a Three.js VR scene allows for intuitive and immersive interactions. This guide covered:

• Importing and setting up ThreeMeshUI.
• Creating interactive panels.
• Configuring buttons with event listeners.
• Managing UI updates in the render loop.

This approach can be extended to more complex VR UI components, enhancing interactivity in your Three.js applications.