Need help integrating Three.js with Deck.gl

[ssriv48]

Been desperately trying to integrate three.js in one of the deck.gl custom layer for the purpose of a PoC. Basic idea is to create a cube in a custom layer using three.js and render on top of other deck.gl layers showing tiles, icons, links etc. as well as being able to integrate with the cube.

When the deck.gl layers are a more than three, the cube renders for a fraction of a second and then disappears. When there are fewer layers, the cube renders fine but as soon as I interact with the cube (rotate), the cube again disappears.

There are no errors on the console, however I get GL_INVALID_OPERATION: Insufficient buffer size warning. On further investigation, they seems to originate in the gl.drawElement in three.js.

My draw function in the custom layer is very simple, all copied from (almost non existing) material available regarding integration of three.js with deck.gl, I tried fixing this issue however I started getting another issue, location is not from current program. Trying to fix that issue, only throws more warning.

draw() {
    const {context: {gl, timeline: {time}}} = this
    const buffer = gl.createBuffer();
    // Update Three.js scene
    this.cube.rotation.x += 0.01;
    this.cube.rotation.y += 0.01;

    // Render Three.js scene
    this.renderer.state.reset(); // Reset WebGL state
    gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, buffer)
    this.renderer.render(this.scene, this.camera);
  }

[Pessimistress]

• You are not uploading any data to buffer which is the cause of the error message
• You should not call gl.createBuffer() in draw(). This method is called frequentlty and you end up with a massive perf hit + memory leak by recreating resources at every animation frame.
• You should not store any thing on this (layer instance). When layer prop is updated the instance is discarded with any custom property on it.

The skeleton of your layer class should look like this:

import {Layer} from '@deck.gl/core';

class MyWebGLLayer extends Layer {

  initializeState() {
    // Create resources
    const {gl} = this.context
    this.state = {
      buffer: gl.createBuffer()
    }
  }

  updateState({changeFlags}) {
    if (changeFlags.propsChanged) {
      // Update resources
      const {buffer} = this.state
      // Update data
      const bufferData = new Float32Array(...)
      gl.bindBuffer(gl.ARRAY_BUFFER, buffer)
      gl.bufferData(gl.ARRAY_BUFFER, bufferData, gl.STATIC_DRAW)
    }
  }

  draw() {
    // Use resources
    const {buffer} = this.state
    gl.bindBuffer(gl.ARRAY_BUFFER, buffer)
    // Render something
    gl.drawElement(...)
  }

  finalizeState() {
    // Delete resources
    const {buffer} = this.state
    gl.deleteBuffer(buffer)
  }
}

[ssriv48]

Thanks, trying the above code, still getting the issue. Do we need to be aware of underlying Webgl issues when working with combination of three and deck.gl. Would it be possible to provide a working example of rendering three.js in a custom deck.gl layer. Anything simple like just rendering a cube on top of icons layer sort of thing and being able to interact with it. Thanks again for your help.

[mrtakethatriskk]

try this just enter your specifications :

import { Deck } from '@deck.gl/core';
import { ThreeLayer } from '@deck.gl/layers';
import { PerspectiveCamera, Scene, WebGLRenderer, BoxGeometry, MeshBasicMaterial, Mesh } from 'three';

class ThreeCustomLayer extends ThreeLayer {
initializeState() {
const { gl } = this.context;

// Initialize Three.js scene, camera, and renderer
const scene = new Scene();
const camera = new PerspectiveCamera(75, gl.canvas.width / gl.canvas.height, 0.1, 1000);
camera.position.z = 5; // Set camera position
const renderer = new WebGLRenderer({ context: gl });

// Create cube
const geometry = new BoxGeometry();
const material = new MeshBasicMaterial({ color: 0x00ff00 });
const cube = new Mesh(geometry, material);
scene.add(cube);

this.setState({ scene, camera, renderer, cube });

}

draw() {
const { gl } = this.context;
const { scene, camera, renderer, cube } = this.getState();

// Update Three.js scene
cube.rotation.x += 0.01;
cube.rotation.y += 0.01;

// Render Three.js scene
renderer.state.reset(); // Reset WebGL state
renderer.render(scene, camera);

// Call super draw function to trigger deck.gl rendering
super.draw();

}
}

// Create deck.gl deck
const deck = new Deck({
canvas: 'deck-canvas',
width: '100%',
height: '100%',
layers: [
new ThreeCustomLayer({
id: 'three-layer',
}),
// Add other deck.gl layers here
],
});

// Render deck.gl deck
deck.setProps({
viewState: {
latitude: 37.7749, // Latitude of the center of the map
longitude: -122.4194, // Longitude of the center of the map
zoom: 12, // Initial zoom level
pitch: 0, // Pitch of the map
bearing: 0 // Bearing of the map
}
});

// Render deck.gl deck
deck.render();

[xiaomo1q]

import { ThreeLayer } from '@deck.gl/layers'; Does not exist in layers. The direction of drawing the threejs camera does not change with the OrbitView

[grahambates-code]

I think that was ChatGPT generated, it relies on things that don't exist.

https://zhuanlan.zhihu.com/p/634390324

I have had success with this approach however!

[charlieforward9]

https://www.npmjs.com/package/@deck.gl-community/three has a good example of how to do this