Synthetic Dataset using Blender + Python: Part 2

6 min readDec 27, 2020

Rendering the dataset

Now that we are comfortable with the Blender starters (from PART 1), we can start using Python to automate some of its aspects and generate a synthetic dataset.

Pre-requisites:

I will be using Ubuntu 20.01 and Blender 2.91.

Objective:

To generate a dataset using Blender and Python, with the characteristics:

Assume our object is centered at the origin (0,0,0)
Capture the object from a particular distance (R), in a circular path, a total of 10 images
The script should also output camera locations and orientation (in Z axis) along with the frames

Such dataset may help us find out the camera/robot’s location given a test image; not so simple as it sounds ;)

Let’s start with setting up our environment.

Open the console and launch Blender.

$ blender

Start with default project: The default project (create one if you haven’t…) gives you a cube centered at the origin with a Camera and a Light. As discussed in the last part, the object can be replaced with your object of Interest by simply importing its 3D model. For better visualization, I have duplicated the default cube (CTRL+C and CTRL+V) and colored them.
Setup the Camera: We plan to take snaps of the object of Interest (OoI) from various points of the trajectory programmed/desired by us. So, we start with an initial setup for our objects: camera, light and all cube(s). It signifies the initial position of our camera, before it can start capturing anything. The Object Properties for Camera look like:

Automate Camera motion using Python script: Let’s first try to understand what are we trying to accomplish here. Here, I am trying to move the camera in a circular trajectory but only till a quadrant; camera starts at the X axis and ends up at the Y-axis.

For a particular radius, this trajectory is traversed by shifting the camera in small steps. The smaller the steps (or step-size), the better (and more) the data. The same step is repeated for different distances or better called as radii.

We are familiar with with the fact that the X and Y coordinate in Cartesian coordinate can be replaced with rCos(theta) and rSin(theta) in spherical coordinate system. So, we can first find theta, and for a particular radius (r), we can find x and y. The Setup the Camera section shows the initial camera orientation. The X and Y coordinates have been fixed by us initially. We find the angle made by the camera at this time. Let’s call it init_angle. This is very close to the X axis, as obvious. Now, we need to limit our motion to a maximum of 90 degrees (or a single quadrant). Let’s call it target_angle. Now, while going from init_angle to target_angle, the number of steps to be taken are specified by num_steps_revolution (just because the camera is revoluting about the origin or the first cube). For simplicity, we choose only a single radius for trajectory.

Let’s not change the lights and get to the code.

Import required dependencies:

import bpy
import os
import numpy as np
from math import *
from mathutils import *

Now, we define the locations and names of the objects that we will be needing: the target a.k.a the OoI (object of interest):

#set your own target here
target = bpy.data.objects['Cube'] #Do not forget to check the object name
t_loc_x = target.location.x
t_loc_y = target.location.y

The target object is the one around which we want the camera to face. In our case, its the cube centered at the origin.

cam = bpy.data.objects['Camera']
cam_loc_x = cam.location.x
cam_loc_y = cam.location.y

Now, define the angles and radius.

R = (target.location.xy-cam.location.xy).length # Radius
num_steps_revolution = 36 #how many revolution steps in each circle/revolution
#ugly fix to get the initial angle right
init_angle  = (1-2*bool((cam_loc_y-t_loc_y)<0))*acos((cam_loc_x-t_loc_x)/dist)-2*pi*bool((cam_loc_y-t_loc_y)<0)
target_angle = (pi/2 - init_angle) # How much more to go...

Start looping the camera

for x in range(num_steps_revolution):
    # Use alpha to locate new camera position
    alpha = init_angle + (x+1)*target_angle/num_steps    # Move camera
    cam.rotation_euler[2] = pi/2+alpha # *Rotate* it to point to the object
    cam.location.x = t_loc_x+cos(alpha)*R
    cam.location.y = t_loc_y+sin(alpha)*R    # Save Path for renders
    file = os.path.join('/home/renders', x) # saves filename as the step number
    bpy.context.scene.render.filepath = file
    bpy.ops.render.render( write_still=True ) # render

The entire code (with slightly different variable names) can be found here

Running the entire code in a console (as shown in Part 1), should render and save 36 images in the path specified. A sample would be:

$ blender -b ~/Videos/blender/panel-synthetic-dataset.blend -P ~/Videos/blender/test_synthetic.py

To visualise if the camera trajectory will look like, I modified the initial script as follows:

I replaced the render part with camera generation. Thanks to this wonderful BlenderExchange Site As we increase the angle, progressing from initial angle init_angle to target angle target_angle, at each step, instead of rendering, I ask Blender to place a new camera at the newly calculated position. The result is as follows:

The blend file can be used for reference here: 10cams.blend

Except for the new cameras all facing towards negative Z axis, as it doesn’t affect our purpose, everything looks Good :)

A Step further:

This seemed very simplistic but great to understand how to get the job started. I used an upgraded version of the code, to give me even more data: I added rotation to revolution. Till now, our camera shifted to a new position in the same circular trajectory and took a snapshot and moved ahead. But now, we ask it to take even more snaps at the same exact spot, by rotating about itself. Further, I ask it to not only follow a single radius, but a range of radii; we need to specify the radii (r1, r2,…) for getting closer or farther from the object. This modified script can also be found here:

This script performs similar camera motions with rotation+revolutions and saves the camera data (location, orientation) as the file name. The process took a minimum of 5 hours on my non-GPU system and generated more thatn two thousand images