Introduction
Three.js does a lot and can be somewhat confusing how it relates to other 3D fields. Its scope is also an ever-evolving thing, so it's not very easy to summarize, and these observations are subjective.
JavaScript 3D library
The goal of this project is to create a lightweight, easy-to-use 3D library. This library provides renderings , , CSS3D and WebGL.
This is the official description from the github repository. It actually sums it up pretty well, but each topic in this sentence is a broad topic on its own, and that's not all three.js does.
The library itself is written in JavaScript and is intended for use in a JavaScript environment. In most cases this means it runs on the client side – in a web browser on some device. But with node.js and headless browsers it can also be used server side. The first thought that comes to mind is rendering – maybe some preview images on the server, but it could just be 3D calculations, as three.js has a rich math library. This is an incredibly broad term. 3D can mean a lot. In most cases we think of “graphics”.
Most three.js projects we see involve real-time 3D graphics, where user interaction results in immediate visual feedback. Other types of 3D graphics are either various effects or artificial characters in movies, or various «renderings» that you might see in print or in a web catalog (for example, IKEA’s website is full of 3D graphics, as all of their product photos are computer-generated).
A subset of all of this is 3D math. 3D graphics can't be done without math, and computer languages don't understand 3D concepts by default. This is where a library comes in, abstracting those math operations, perhaps optimizing them, and exposing a high-level interface like Matrix4 or .dot().
Three.js comes with its own math library with classes specific to 3D math. There are standalone libraries that deal with this math alone, but with Three, it's just a subset of a much larger system.
Rendering is another big responsibility of the library, but here too things get a little complicated. WebGL is very specific and stands out from the crowd.
With canvas, svg and css, the responsibility for rendering is entirely 3D. These APIs have many libraries for drawing non-3D things, or actually do it by default (css draws 2D rectangles, canvas draws various 2D shapes), but they require a magic touch and 3D math to actually do 3D rendering.
The magic touch comes mostly in the form of interface abstraction. For example, managing the 3D state of an element. <div> Converting to 3D via CSS is very difficult. It takes a lot of logic to get the Canvas API to draw something that looks like 3D. WebGL is much more involved.
Three abstracts all of these APIs into something as simple as render() , but to do so it needs a generic representation of what the "3D world" is.
Scene diagram
One can identify an area of three.js that acts as this general abstraction of the «3D world». A scene graph is a data structure used to describe how objects in some 3D scene (world) relate to each other. It doesn’t actually have to be 3D, as this is a convenient way to describe any vector graphics hierarchy. It is more specifically a “tree” branching out from “nodes” with a “root node”. In three.js the base class for this data structure is Object3D.
This is almost exactly the same as the DOM tree. THREE.Scene is similar <body> will be and everything else is a branch. In the DOM we can put things in place but they are relatively limited. Rotation usually happens around an axis and we move things left/right or up/down. In a 3D scene graph we have more degrees of freedom.
Scene three is more like a virtual DOM. We perform our operations and set state on that tree, and when we want to have a visual snapshot of that state (say in a continuous loop, or some user interaction/state change), we call render (scene). When something changes, you don't want to update the entire DOM tree, while with the element , we have to clear the entire view, then redraw everything, even if only one element changes its position.
One <div> In one <div> Similar to the parent-child relationship THREE.Mesh('sun')->THREE.Mesh('earth'). A CSS rule could be similar to a THREE.Material where a description like color:'red' makes the magic happen and something turns red. Finally, the call to threeRenderer.render(scene) could be similar to the browser loading some html page with some CSS rules.
Mesh, Scene, Camera, and Light are all subclasses of this generic class. This is what allows you to add () a “box” to the “scene”, or a “light” to the “camera”.
A simple structure can be very flat. The root node can be seen as "world" and can have "ground", "house", "sun" and "camera" as its children.
THREE.Scene('world')
|-THREE.Mesh('ground')
|-THREE.Mesh('house')
|-THREE.Light('sun')
|-THREE.Camera('main')This is enough information to feed into a renderer to get a visual result. For some scenes, there are two grids representing different things, the ground and the house on the hill. A light that defines how they are lit (morning vs. noon vs. flash light) and an object (the camera) that defines our point of view, our view of the world.
The results may vary, for example CSS is limited to a very light rendering, while WebGL can provide shadows and generally high fidelity.
Through this structure, the 3D world is managed. If we want to simulate how daylight affects a house during different seasons, we programmatically change the position and direction of the light in the world. The scene graph's job is to expose this "position" hook, but to actually animate it, you need to implement your own logic. A simple way to animate a 3D scene is three.js with a "tweening" library.
All of this is probably only true in theory, and you may not be able to change the scene renders at will. But this is mostly due to the overlap of "materials" with the scene graph and their coupling with renders. For example, there is no way to shade a <div> Or it doesn't appear as metal, which is something a WebGL material can describe, but it can be made red, which is something all materials can do.
Underneath it all Object3D is still generic, and the spatial and hierarchical relationship of nodes to each other is described through a “scene graph”.
In plain English, this is the scene variable you end up with after calling scene.add(my_foo) multiple times.
WebGL
Webgl is incredibly specific and is probably used in something like 99% of existing three.js applications. It's a big topic, so it might be worth doing a quick overview of the alternatives first.
canvas, css, svg
These are all APIs. This is an interface that you as a programmer can use to tell the browser to draw certain things. CSS is the most common interface on the web because without it, everything looks like plain text. Historically it had nothing to do with 3D.
Canvas actually uses the same WebGL element for drawing, but has a different context. The context is actually called “2d”, but since 3d is fake anyway and we are always drawing some kind of 2D surface, whether real or virtual, we can also use this context to draw 3D graphics.
SVG is another non-3D API that is typically used to describe things like logos or icons. However, since it can describe basic things like lines, it can also be represented in a 3D context (such as an overlay on a map, or spatially aware UI or HUD elements).
A common theme here is that none of these are intended for 3D use. Another important feature is that they are all high-level – they were previously intended to do something else. For example, all three know how to draw a “circle”. With Canvas, this is an explicit shape, with CSS you may have to use border radii, but the end result is a very direct access to a circle.
WebGL is very low-level, it doesn't know much about the concept of 3D graphics. 3D graphics require doing specific mathematical calculations, and it requires a lot of them. Just think for a second about a high-resolution screen and the number of pixels it has. If you have to do calculations for each pixel to determine how a certain amount of light affects the surface, and you have to do this 60 times a second, that number adds up.
Graphics processor
Most computers, such as laptops, desktops, mobile phones, and even watches, have some kind of hardware device that can efficiently calculate these 3D operations and allow for the display of interactive graphics. This is called a graphics processing unit.
It is different from the main processor because it is built for different purposes – specific mathematical operations that are executed in parallel.
Just as we use JavaScript for browser programming, we use WebGL for graphics card programming.
Well, that's true in concept, but in practice they are two very different beasts. WebGL is made up of both JavaScript code (the instructions) and a completely different language that actually does the calculations (GLSL). You can draw some parallels between HTML and JavaScript and how they work together on a page.
Two-dimensional and three-dimensional
It's not just 3D that benefits from this hardware acceleration. Video processing is also a good candidate. You can program the graphics card to change colors or change the image in a live video feed.
WebGL, which is very low level, is generic. It doesn't know about 2d or 3d, but it knows about memory, buffers, command queues, shaders, etc.
Dealing with parallel programming is different from how you would program in JavaScript. A common problem is how different threads access a shared variable.
This different paradigm means that there's a whole other language called GLSL. It's a shading language that's kind of in every low-level graphics API. It's where you write the actual logic for these big numbers, and the only benefit is that you don't have to write machine code.
The other part of the WebGL API is the JavaScript bindings through which you tell the GPU to do things. A shader "do calculations A" and a binding "run a million times".
It's up to the programmer to calculate what A is. It could be something 3D related or it could be a kernel that blurs the video.
When you start abstracting these calculations and these commands, you end up with three.js.
Renders that work together
One use case that makes a lot of sense is using a combination of renderers to draw things that are good in "3D". WebGL can crunch a lot of numbers and create really realistic images, but it's weak at handling text and even some lines. An extra layer of rendered text can be handled via CSS and canvas renderers, while paths and various lines can be handled via SVG.
THREE.WebGLRenderer
All of this low-level stuff is abstracted away through a three.js WebGLRenderer class. This is what turns the cube into a bunch of numbers in GPU memory.
Ironically, it's the only three.js renderer that doesn't need to do 3D graphics, but it's the best option for it. The rest fake 3D using 2D APIs, WebGL intentionally does 3D using a generic parallel computing API. But that still doesn't rule out a scenario where you could use it exclusively to process that live video stream. It abstracts away enough from WebGL to be useful for this, but you're probably using a third of the library. You could build a super responsive UI layer with WebGL, or a Super Mario-type platform game where three.js would still be a great tool.
The fact that you're only using a third of the library means that a different tool could be better suited for that use, or that you could just build a subset of three.js. Both the Super Mario and video processing examples might only need PlaneGeometry and maybe a material type.
three-math
JavaScript code that performs certain mathematical operations in 3D. JS has Math.pow() by default but does not have Quaternion.inverse(). With these classes we can write algorithms that do not require rendering – for example a game server that verifies who is shooting who is doing a lot of rays but not drawing anything.
three scene-graph
A family of Object3D subclasses that form a tree data structure that describes a "3D world" and the relationships of objects within it. It is conceptually abstract, but can be somewhat rendered with specific rendering, once you dive into the code.
three renders
The layer that translates that generic graph into a visual representation on the screen or some buffer (e.g. you create it on the server side). It uses various technologies to achieve this goal.
THREE-WebGLRenderer
A special renderer that allows for hardware acceleration and knows a lot of 3D concepts but can also be used for 2D (and even general computing). These are the main building blocks of three.js IMO. I tend to replace “3D” with “Graphics”, but that only applies to the WebGLRenderer.
Three.js is not a game engine.
Not everyone who needs 3D (or graphics) on the web is making games. Game engines typically do a lot of optimizations beyond describing and displaying 3D worlds. Different games have different needs, and physics and rendering systems for real-time strategy and first-person shooters are likely to have very different needs. All of this means more code, and for someone who just wants to spin up a 3D model as part of a product catalog, this is not only unnecessary but undesirable. Of course, you could build a game engine and use three for rendering and for building engine blocks.
Three.js is not loading much
Sure, the core has multiple loaders for some assets, but all the common formats like gltf or fbx are independent. Threejs doesn't care how you get your assets, as long as you parse them properly and create a 3D object. As far as Threejs is concerned, there is no difference between a mesh from a gltf file and a procedural sphere. Many creative examples use cubes and spheres and don't load anything other than Three.js itself. The core loaders are very generic, loading images and files and having a direct representation of a Threejs object like a Material or Texture. Format-specific loaders are built with these building blocks.
