Run external C code with Sverchok SNLite node in Blender

f:id:asahidari:20210804013833p:plain

Overview

This entry describes how to run external C/C++ code in Blender, using Sverchok's Script Node Lite (SNLite). You can also use this approach with only Blender python without Sverchok. But if you use the add-on, you may write fewer codes and put the outputs into other nodes.

Blender requires add-ons not to include binary files like dll. So this approach cannot used to make add-ons. But this approach may extend the capability of Blender's scripting. When you do tasks that need large amount of calculation and high performance, such as implementing Machine learning, Fractals and Reaction Diffusion, this technique will be some help.

Required Environments

In this approach, these tools or modules will be needed.
- Blender and Sverchok add-on
- Python's ctypes or numpy/ctypeslib module (Blender's python already has them.)
- Precompiled C/C++ dynamic libraries(If you do not need to modify them.)
- C/C++ Compiler or IDE (If you need to build your custom C/C++ library.)

You can also use Cython to run C code. Both Ctypes and Cython have pros and cons. But if you have pre-compiled library, you do not have to compile C/C++ library. So in this blog, I will use ctypes (and numpy/ctypeslib).

Current Sverchok's approach (July, 2021)

Sverchok developers already recognized the possibility to use external C/C++ code using SNLite.
Interesting external dependencies to play with (discussion) · Issue #2152 · nortikin/sverchok · GitHub

But as mentioned above, Blender's add-on should not include binaries. They probably has been exploring the way to do high performance tasks without using C/C++ libraries and Cython. Currently, using Python's numba library,JIT(Just in time) compiler , is possible candidate. I hope this approach work well.
Can we use Numba?... · Issue #2646 · nortikin/sverchok · GitHub
add numba dependency by zeffii · Pull Request #4209 · nortikin/sverchok · GitHub

A simple example to use a C library

Script and code below is included in my GitHub repository.
GitHub - asahidari/run_c_code_with_snlite_b3d: An example of running C code in Blender Sverchok add-on.

This example simply multiply x/y/z values of all vertices, and generate results for all drawing steps. Using 'setup' function can provide this behavior, because 'setup' function can behaves like Processing 'setup'.

I learned this pattern by reading the issue below in Sverchok project in GitHub.
Reaction Diffusion script. · Issue #1734 · nortikin/sverchok · GitHub

C code:

#include <stdlib.h>

// declare callback function type
typedef void _callback_func(int step, int dim1, int dim2, double* ppArray, void* pObj);

// process method to multiply vertex-coordinates with scale value in each step
int process(int num_steps, double scale, int dim1, int dim2, double** verts, _callback_func cb_func, void* pObj) {

    if (verts == NULL) {
        return -1;
    }

    // Allocate buffer
    double* pBuff = (double*)malloc(sizeof(double) * dim1 * dim2);

    //  set initial value to the buffer
    for (int i = 0; i < dim1; i++) {
        for (int j = 0; j < dim2; j++) {
            pBuff[i*dim2 + j] = verts[i][j];
        }
    }

    for (int s = 0; s < num_steps; s++) {

        // multiply element values in each step
        for (int i = 0; i < dim1; i++) {
            for (int j = 0; j < dim2; j++) {
                pBuff[i*dim2 + j] *= scale;
            }
        }

        // call the callback function to pass the parameters
        if (cb_func != NULL) {
            cb_func(s, dim1, dim2, pBuff, pObj);
        }
    }

    // Release buffer
    free(pBuff);

    return 0;
}

Python code:

"""
in steps s d=10 n=2
in verts_in v
in scale s d=2.0 n=2
in framenum s d=0 n=2
out verts_out v
"""

def setup():
    
    import numpy as np
    import numpy.ctypeslib as npct
    import os
    import ctypes as ct

    def callback_func(step, dim1, dim2, data, selfp):

        # convert array type from c to numpy
        verts_arr = npct.as_array(ct.POINTER(ct.c_double).from_address(ct.addressof(data)), shape=(dim1, dim2))
        arr_stored = verts_arr.tolist()

        # call class method using self pointer
        instance = ct.cast(selfp, ct.py_object).value
        instance.store_frame(step, arr_stored)

    # class declaration
    # Derived from ctype.Structure to pass self pointer to c function 
    class CMultiply(ct.Structure):

        def __init__(self, steps=10, scale=2.0, verts=None):
            
            self.frame_storage = {}
            if verts == None:
                return;

            # declare callback function c type
            c_arr_type = np.ctypeslib.ndpointer(dtype=ct.c_double, flags='C_CONTIGUOUS')
            callback_func_type = ct.CFUNCTYPE(None, ct.c_int, ct.c_int, ct.c_int, c_arr_type, ct.c_void_p)
            
            # load library
            libscale_verts = npct.load_library('libscale_verts', os.path.dirname('/Path/to/the/library/directory/'))
            
            # declare argtypes and restype
            libscale_verts.process.argtypes = [
               ct.c_int, # step count
               ct.c_double, # scale value
               ct.c_int, # array dimension1
               ct.c_int, # array dimension2
               npct.ndpointer(dtype=np.uintp, ndim=1, flags='C'), # verts array
               callback_func_type, # callback func
               ct.py_object # self
               ]
            libscale_verts.process.restype = ct.c_int

            # convert array type
            verts_arr = np.array(verts)
            verts_arr_ptr = (verts_arr.__array_interface__['data'][0] + np.arange(verts_arr.shape[0])*verts_arr.strides[0]).astype(np.uintp)

            # call c function
            dim1, dim2 = verts_arr.shape[0], verts_arr.shape[1]
            res = libscale_verts.process(steps, scale, dim1, dim2, verts_arr_ptr, callback_func_type(callback_func), ct.py_object(self))

        # get frame
        def get_frame(self, number):
            return self.frame_storage.get(number) if number in self.frame_storage else []

        # store frame
        def store_frame(self, framestep, data):
            self.frame_storage[framestep] = data

    # instantiate main class
    cm = CMultiply(steps, scale, (verts_in[0] if verts_in is not None else None))

# set results per frame
verts_array = cm.get_frame(framenum)
verts_out.append(verts_array)

How to use

To use these scripts,

Compile c code to make a dynamic library.

  :: Windows
  cl.exe /D_USRDLL /D_WINDLL scale_verts.c /MT /link /DLL /OUT:libscale_verts.dll

  # macOS
  gcc -dynamiclib -o ./libscale_verts.dylib ./scale_verts.c

  # Linux
  gcc -c -fPIC scale_verts.c -o scale_verts.o
  gcc scale_verts.o -shared -o libscale_verts.so

Click '+New' button of Blender's text editor and click '+New' to create a text buffer.
Copy and paste this Python code into the text buffer.

Write the library path to the "load_library" argument in the python code.

          # load library
          libscale_verts = npct.load_library('libscale_verts', os.path.dirname('/Path/to/the/library/directory/'))

Open Sverchok node editor and create Script Node Lite (SNLite) node.
Write the text buffer name (default is 'Text') to the SNLite node text box and click the right button in the node.
Input some mesh vertices to the SNLite node and connect the output to a ViewerDraw node.
Change 'framenum' value of the SNLite node to draw the results of each step.

I recommend to launch Blender from command line to see debug code in your terminal. Link page below is how to launch Blender from command line.
Launching from the Command Line — Blender Manual

f:id:asahidari:20210806235901g:plain

What this project is used for?

This project is intended to be used as a template for projects using other C/C++ codes in Blender. This script includes the ways to load C library, to pass array between Python and C, and to use a python class method as a callback function. When you want to run other external C/C++ library in Blender or other Python environments, this project files may be some help.

I made a Reaction Diffusion simulation codes from this project. Top image of this page (Reaction Diffusion with a cube) is generated using the project below.
GitHub - asahidari/ReactionDiffusion_SNLite_b3d: Reaction Diffusion simulation with Sverchok SNLite node and C codes in Blender.