# Writing Sample: Creating a 3D model from a 2D picture

{% hint style="info" %}
This page is **not** a tutorial for the tools used to create a 3D model from pictures.  It is an introduction to the process and workflow used in Photogrammetry for artists who would typically create 3D models from scratch.&#x20;
{% endhint %}

### Taking Photos&#x20;

Start with 12 continuous shots taken by rotating around a fixed object in good natural lighting. Using a high quality camera is recommended.&#x20;

In this example, we're going to build a 3D model of a frog.We'll build the model by using the continuous shots of the frog sculpture shown below:

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FDTkUFbngDJsN03dwK0gK%2Fimage.png?alt=media&#x26;token=e8831b91-d2cf-487f-b2e4-67b192cbb2a0" alt=""><figcaption><p>Shot 1</p></figcaption></figure>

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FfLbgI1CefBWZ5ec1XQvT%2Fimage.png?alt=media&#x26;token=ba27f231-cd8e-40ea-a24e-aa991ff4b360" alt=""><figcaption><p>Shot 2</p></figcaption></figure>

### Importing photos&#x20;

Download [Regard 3D](http://www.regard3d.org/) and import the continuous shots into the tool.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2Fa7JIOpi5ljM21qbf6MQ6%2Fimage.png?alt=media&#x26;token=36e5dd52-e8a6-461b-8a51-e435649c4b19" alt=""><figcaption><p>Importing photos</p></figcaption></figure>

### Computing matches &#x20;

An algorithm in your chosen tool (Regard 3D in this case) will detect the key points from each image and try to match it to other points in subsequent photos to create a continuous point cloud. \
\
For example, if the first photo has a view of the frog's leg from the left, and the second photo has a view of the frog's leg from the right, then the algorithm can reasonably start mapping what a 3D view of the leg might look like.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FDLvmZwIYP0V5MwMLvy6q%2Fimage.png?alt=media&#x26;token=aa1ea70d-b006-4b4f-a6d6-ba03d09fb8d2" alt=""><figcaption><p>Computing matches</p></figcaption></figure>

### Generating a point cloud&#x20;

Once the matches have been computed, you'll need to use *triangulation* to generate a point cloud. \
Triangulation is the process of determining the location of a point by forming triangles to it from a known point. \
\
Regard 3D has now represented the entire scene as points or pixels. However, the points are scattered, and not good enough to generate a continuous and smooth 3D model.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2F8mgbOSlXR57LT7M2OPu6%2Fimage.png?alt=media&#x26;token=f994dabf-a2d8-4785-9676-c97ada40d20f" alt=""><figcaption><p>Generating a point cloud</p></figcaption></figure>

### Densifying a point cloud &#x20;

The densification process brings the sparse points in our point cloud closer together — making them dense. This step is necessary to close the hollow spaces in our scene.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FPqAIv0VzRkM0jUjP7XWN%2Fimage.png?alt=media&#x26;token=e7234988-ede4-4fe9-97bc-d1d96b95b39f" alt=""><figcaption><p>Densifying a point cloud</p></figcaption></figure>

We will now export the dense point cloud to another software called [Meshlab](http://www.meshlab.net/). We need to do this to:

* clear the unnecessary points from the scene — after all we only want to create a 3D model of the frog, not the entire scene.&#x20;
* generate a mesh, and add a surface to the model. \
  \
  Regard 3D allows you to export the entire scene as a single file which we will import into Meshlab.&#x20;

### Generating a mesh&#x20;

Generate a mesh around the dense point cloud which you just imported into Meshlab. Mesh generation connects the different points in the dense point cloud to create a grid-like structure.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FcoqBcJ2mcBSUNKssGUv2%2Fimage.png?alt=media&#x26;token=7adb6237-21d9-4d68-8255-85b1e5be5c73" alt=""><figcaption><p>Generating a mesh</p></figcaption></figure>

### Trimming&#x20;

Remove the unnecessary points from the scene. Select and remove all the areas that are not going to be part of your final 3D model.\
In this example, we want to generate a model of the frog only

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2F2KV2NbOQoLy5vIzjckKR%2Fimage.png?alt=media&#x26;token=c68b4e77-91e4-46d2-8036-0e207599b4ea" alt=""><figcaption><p>Trimming </p></figcaption></figure>

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2Fbcyb2T4rf7IWxyLGTnIb%2Fimage.png?alt=media&#x26;token=c582cad2-8f9b-4694-8893-ec6243a4bf60" alt=""><figcaption><p>Final mesh for 3D model generation</p></figcaption></figure>

{% hint style="warning" %}
You might notice that certain areas of the final mesh are not smooth. This is due to the dark areas on our original photos. During compute matching, Regard 3D was unable to build a continuous point cloud as certain areas of the photos were not bright enough.&#x20;
{% endhint %}

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FGUnpdBXmepwJCrei02I2%2Fimage.png?alt=media&#x26;token=d76cefe4-c7c5-496f-892b-8003e818d68f" alt=""><figcaption><p>Poorly-lit areas in the original photo may not generate point cloud</p></figcaption></figure>

Now that we have our final mesh, we can add a surface to it. A point cloud and mesh contains hollows which we need to cover using a surface. \
\
In this example we used [Poisson surface reconstruction](http://sites.fas.harvard.edu/~cs277/papers/poissonrecon.pdf)[ ](http://www.open3d.org/docs/release/tutorial/geometry/surface_reconstruction.html)algorithm available in Meshlab.

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FYoEiVpyH4LGOBff8YZkk%2Fimage.png?alt=media&#x26;token=6ae740c2-1dcd-4fd4-a5ad-b596ac149785" alt=""><figcaption><p>Poisson mesh generated in Meshlab</p></figcaption></figure>

As you can see, some unusual shapes were generated around our model. We'll need to trim the extra surface using the trimming tools again.&#x20;

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2F2FBejtzhQCdaF7VGAO4o%2Fimage.png?alt=media&#x26;token=80f381d1-0a11-400d-be3b-c29b0097d7e9" alt=""><figcaption><p>Retrimming unwanted points </p></figcaption></figure>

Once you've trimmed the extra parts, you can export the final model as a Polygon format(**.ply)** file. The model is now ready to be shared :handshake: !

### Visualizing and sharing the model&#x20;

[Sketchfab](https://sketchfab.com/) is a service which allows you to upload, visualize and share your 3D models. \
Sign up for a new account and upload the .**ply** file from the previous step. \
Your model can be viewed and shared using a Sketchfab link. See ours [here](https://skfb.ly/6SMyL) :white\_check\_mark: \
\
We've just created our first ever 3D model from pictures. Celebrate :tada: !

<figure><img src="https://332770572-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M4yUEdnfNT7F9R9vHow%2Fuploads%2FKwKErsDyxcOEpoXbk1Ay%2Fimage.png?alt=media&#x26;token=5624441d-9b4f-4580-9152-678c890311cb" alt=""><figcaption><p>Majestic resting frog</p></figcaption></figure>
