Lecture 6 + Lecture 6 (IDK)

A huge dataset with over 10 million 3D objects

A large library of 3D objects, mostly chairs, tables, and cars

Another large library of 3D objects for 3D shape research

Point cloud, Mesh, Voxel grid

A 3D representation made of dots in space (x,y,z) outlining the surface of an object without faces or textures

They are lightweight and good for capturing the overall shape of objects

They don’t store surfaces or textures, appearing like a hollow skeleton

A 3D representation using tiny cubes (voxels) in a grid to show which parts of space are solid or empty

They’re easy for computers to handle and capture inside and outside of objects

They require a lot of memory for high resolution and can look blocky if the grid is coarse

A representation of an object’s surface using vertices connected by edges to form triangles that wrap around the object

They are efficient and produce smooth, detailed surfaces used in games and 3D modeling

They are harder for neural networks to generate due to their complex structure

Signed Distance Function

For every point in space, it gives the distance to the object’s surface—negative inside, zero on the surface, positive outside

They give smooth, continuous shapes and are more compact than voxel grids

By finding all points where the distance equals zero (using algorithms like marching cubes)

A neural network that represents a 3D scene by mapping each 3D point and view direction to color and density, enabling realistic images from any angle

A 3D coordinate (x,y,z) and a viewing direction

Density (how solid the point is) and radiance/color (light emitted toward the camera)

It casts rays from the camera, samples many points along each ray, predicts color and density for each, then blends them using volume rendering

A one-shot process where the model directly generates a 3D shape or image in a single forward pass without iterative refinement

3D GANs or triplane diffusion models that combine 2D top-down, front, and side views into a 3D model

An approach that starts with a rough shape and iteratively adjusts it to match data or a prompt by minimizing a loss function

DreamFusion, which optimizes a 3D shape guided by a 2D diffusion model’s feedback

It often produces high-quality, accurate results tailored to specific input data

It is slower because it optimizes each new object or scene from scratch

The task of generating realistic images of a scene or object from new, unseen camera angles based on one or a few input photos

Because it can hallucinate or imagine plausible unseen parts of the object rather than just re-projecting observed pixels

Zero-1-to-3 and SyncDreamer

It uses a 3D-aware neural network to understand the scene and then generates images from new viewpoints, filling in unseen regions

A rule-based, algorithmic method (often with randomness) that automatically creates complex scenes or objects without manual design

Minecraft, which builds terrains and biomes using algorithms rather than hand-crafted maps

It can produce infinite variety of content efficiently without storing each detail

It can be harder to control for specific artistic details and may sometimes look unnatural without tuning

A machine learning technique that generates highly detailed 3D models from a set of 2D images taken from different viewpoints.

A pretrained 2D text-to-image diffusion model (like Imagen) with Neural Radiance Fields (NeRF) to create 3D representations.

A direct mapping from input (like noise or text) to the output representation.

Natural-looking structures with branching or self-similar patterns, like plants or terrain.

Point Clouds

One or more 2D images of an object.

New images of the same object from different viewpoints.

A mathematical function that assigns to each point in space the shortest distance to the boundary of a given shape, with a sign indicating whether the point lies inside or outside the shape.

Negative values.

Positive values.

Zero.

Optimization methods typically use guidance from pre-trained 2D models to iteratively refine a 3D representation, while feedforward methods learn a direct mapping.

3D objects as compositions of distinct parts, generated jointly from a single input image.

Using three orthogonal 2D feature planes (e.g., XY, YZ, XZ).

Point-E is a feedforward model generating point clouds directly, while DreamFusion uses optimization to refine an implicit 3D representation guided by a 2D model.

The RGB color and volume density (sigma).

Point Clouds, Voxels, Meshes (also Implicit Representations, Images).

3D grids similar to 2D pixels.

Surfaces defined by vertices, edges, and faces (polygons).

Signed Distance Functions (SDFs) or Neural Radiance Fields (NeRFs).

ShapeNet (or ModelNet, Objaverse / Objaverse-XL).

Directly generating the 3D representation (e.g., Point Cloud, Voxel) using models like GANs, VAEs, or Diffusion Models.

Iteratively optimizing a 3D representation (often implicit, like NeRF) to match a target, guided by models like CLIP or 2D Diffusion.

Using algorithms and rules (like Fractals, L-Systems, Noise Functions) to create 3D content.

Reconstructing 3D scenes from video.