Architecture & Design Verification: 2. Python-UVM Integration

In the previous post, we introduced the concept of Python-driven UVM testing. This post dives deep into the DPI-C bridge that makes it possible—the C layer that connects Python's rich ecosystem to SystemVerilog's verification power.

DPI-C Fundamentals

DPI-C (Direct Programming Interface for C) is SystemVerilog's mechanism for calling C functions and vice versa. It's the bridge that enables foreign language integration.

Import vs Export

Direction	Keyword	Usage
SV calls C	`import "DPI-C"`	Call C functions from SystemVerilog
C calls SV	`export "DPI-C"`	Call SystemVerilog functions from C

// Import: SV can call these C functions
import "DPI-C" function int dpi_init_python(string module_name);
import "DPI-C" function int dpi_get_transaction(
  output int addr, output int data, output byte is_write
);
import "DPI-C" function void dpi_send_read_data(int data);

// Export: C can call this SV function
export "DPI-C" function void sv_log_message(string msg);

Data Type Mapping

DPI-C defines how SystemVerilog types map to C types:

SystemVerilog	C Type	Notes
`byte`	`char`	8-bit signed
`shortint`	`short`	16-bit signed
`int`	`int`	32-bit signed
`longint`	`long long`	64-bit signed
`string`	`const char*`	Null-terminated
`bit [N:0]`	`svBitVecVal*`	Packed array
`logic [N:0]`	`svLogicVecVal*`	4-state packed array

Performance Tip: Use int and byte for simple data exchange—they map directly to C primitives without conversion overhead.

Bridge Architecture

Our DPI bridge uses a modular, plugin-based architecture:

flowchart TB
    subgraph SV["SystemVerilog"]
        S1["UVM Sequence"]
        S2["DPI Import Declarations"]
    end
    
    subgraph Bridge["C Bridge Layer"]
        subgraph Core["Core Layer"]
            C1["dpi_core.c\nPython Lifecycle"]
            C2["dpi_registry.c\nPlugin Management"]
            C3["dpi_types.h\nCommon Types"]
        end
        subgraph Plugins["Plugin Layer"]
            P1["apb_plugin.c\nAPB Protocol"]
            P2["generic_plugin.c\nString Serialization"]
        end
    end
    
    subgraph Python["Python"]
        PY1["apb_driver.py"]
        PY2["Test Scripts"]
    end
    
    S1 --> S2
    S2 -->|"DPI Calls"| P1
    P1 --> C1
    C1 -->|"Python C API"| PY1
    PY1 --> PY2
    
    style Core fill:#fef3c7,stroke:#f59e0b
    style Plugins fill:#dbeafe,stroke:#3b82f6
    style Python fill:#d1fae5,stroke:#10b981

Directory Structure

dpi_bridge/
├── dpi_bridge.c           # Main entry point
├── core/
│   ├── dpi_types.h        # Return codes, types, logging macros
│   ├── dpi_core.h/c       # Python interpreter management
│   └── dpi_registry.h/c   # Plugin lifecycle management
└── plugins/
    ├── plugin_interface.h # Plugin contract definition
    ├── apb/
    │   └── apb_plugin.h/c # APB-specific DPI functions
    └── generic/
        └── generic_plugin.h/c # String-based serialization

Core Layer: Python Lifecycle Management

Type Definitions (dpi_types.h)

// dpi_types.h - Common types and return codes

#ifndef DPI_TYPES_H
#define DPI_TYPES_H

#include <stdint.h>
#include <stdio.h>

// Simulation time type
typedef int64_t dpi_time_t;

// Return codes
#define DPI_SUCCESS 0
#define DPI_ERROR   1

// Plugin status
typedef enum {
  PLUGIN_UNINITIALIZED,
  PLUGIN_INITIALIZED,
  PLUGIN_ACTIVE,
  PLUGIN_ERROR
} plugin_status_t;

// Logging macros
#define DPI_LOG_INFO(fmt, ...) \
  printf("[DPI-INFO] " fmt "\n", ##__VA_ARGS__)

#define DPI_LOG_ERROR(fmt, ...) \
  fprintf(stderr, "[DPI-ERROR] " fmt "\n", ##__VA_ARGS__)

#define DPI_LOG_DEBUG(fmt, ...) \
  printf("[DPI-DEBUG] " fmt "\n", ##__VA_ARGS__)

#endif

Python Initialization (dpi_core.c)

// dpi_core.c - Python interpreter management

#include <Python.h>
#include "dpi_types.h"
#include "dpi_core.h"

static int python_initialized = 0;

int dpi_core_init_python(void) {
  // Guard against multiple initialization
  if (python_initialized) {
    DPI_LOG_DEBUG("Python already initialized");
    return DPI_SUCCESS;
  }

  // Initialize Python interpreter
  Py_Initialize();

  if (!Py_IsInitialized()) {
    DPI_LOG_ERROR("Failed to initialize Python");
    return DPI_ERROR;
  }

  // Add custom paths to sys.path
  PyObject* sys_path = PySys_GetObject("path");
  PyList_Append(sys_path, PyUnicode_FromString("./sim/tests"));
  PyList_Append(sys_path, PyUnicode_FromString("./dpi_bridge/plugins"));

  python_initialized = 1;
  DPI_LOG_INFO("Python interpreter initialized");
  return DPI_SUCCESS;
}

void dpi_core_finalize_python(void) {
  if (python_initialized) {
    Py_Finalize();
    python_initialized = 0;
    DPI_LOG_INFO("Python interpreter finalized");
  }
}

Module Loading

// Load a Python module by name
PyObject* dpi_core_load_module(const char* module_name) {
  if (!python_initialized) {
    DPI_LOG_ERROR("Python not initialized");
    return NULL;
  }

  PyObject* module = PyImport_ImportModule(module_name);

  if (module == NULL) {
    DPI_LOG_ERROR("Failed to load module: %s", module_name);
    PyErr_Print();  // Print Python traceback
    return NULL;
  }

  DPI_LOG_INFO("Loaded module: %s", module_name);
  return module;
}

// Get a callable function from a module
PyObject* dpi_core_get_function(PyObject* module, const char* func_name) {
  PyObject* func = PyObject_GetAttrString(module, func_name);

  if (func == NULL || !PyCallable_Check(func)) {
    DPI_LOG_ERROR("Function not found or not callable: %s", func_name);
    Py_XDECREF(func);
    return NULL;
  }

  return func;
}

// Call a Python function with arguments
PyObject* dpi_core_call_function(PyObject* func, PyObject* args) {
  PyObject* result = PyObject_CallObject(func, args);

  if (result == NULL) {
    DPI_LOG_ERROR("Function call failed");
    PyErr_Print();  // Print traceback for debugging
  }

  return result;
}

APB Plugin: Protocol-Specific DPI Functions

The APB plugin exposes three DPI functions to SystemVerilog:

sequenceDiagram
    participant SV as SystemVerilog
    participant C as C Bridge
    participant Py as Python
    
    SV->>C: apb_init("apb_basic_test")
    C->>Py: load_test(test_name)
    Py-->>C: sequence object
    C-->>SV: DPI_SUCCESS
    
    loop For each transaction
        SV->>C: dpi_get_transaction(&addr, &data, &is_write)
        C->>Py: get_transaction(sim_time)
        Py-->>C: (is_write, addr, data)
        C-->>SV: 1 (valid) or 0 (done)
        
        alt Read transaction
            SV->>SV: Drive to DUT, get response
            SV->>C: dpi_send_read_data(read_data)
            C->>Py: send_read_data(sim_time, data)
        end
    end

APB Plugin Implementation

// apb_plugin.c - APB protocol DPI functions

#include <Python.h>
#include "svdpi.h"
#include "dpi_types.h"
#include "dpi_core.h"

static PyObject* py_module = NULL;
static PyObject* py_get_transaction = NULL;
static PyObject* py_send_read_data = NULL;

// Initialize APB plugin and load test module
int apb_init(const char* test_name) {
  // Initialize Python if needed
  if (dpi_core_init_python() != DPI_SUCCESS) {
    return DPI_ERROR;
  }

  // Load the APB driver module
  py_module = dpi_core_load_module("apb_driver");
  if (py_module == NULL) {
    return DPI_ERROR;
  }

  // Call load_test() to initialize the sequence
  PyObject* load_func = dpi_core_get_function(py_module, "load_test");
  PyObject* args = Py_BuildValue("(s)", test_name);
  PyObject* result = dpi_core_call_function(load_func, args);
  Py_DECREF(args);
  Py_XDECREF(result);

  // Cache function references for performance
  py_get_transaction = dpi_core_get_function(py_module, "get_transaction");
  py_send_read_data = dpi_core_get_function(py_module, "send_read_data");

  if (!py_get_transaction || !py_send_read_data) {
    return DPI_ERROR;
  }

  DPI_LOG_INFO("APB plugin initialized with test: %s", test_name);
  return DPI_SUCCESS;
}

// Get next transaction from Python
int dpi_get_transaction(int* addr, int* data, char* is_write) {
  // Pass simulation time to Python
  PyObject* args = Py_BuildValue("(L)", (long long)sv_time());
  PyObject* result = dpi_core_call_function(py_get_transaction, args);
  Py_DECREF(args);

  if (result == NULL || result == Py_None) {
    Py_XDECREF(result);
    return 0;  // No more transactions
  }

  // Unpack tuple: (is_write, addr, data)
  int write_flag, addr_val, data_val;
  if (!PyArg_ParseTuple(result, "iii", &write_flag, &addr_val, &data_val)) {
    DPI_LOG_ERROR("Failed to parse transaction tuple");
    Py_DECREF(result);
    return 0;
  }

  *is_write = (char)write_flag;
  *addr = addr_val;
  *data = data_val;

  Py_DECREF(result);
  return 1;  // Valid transaction
}

// Send read data back to Python
void dpi_send_read_data(int data) {
  PyObject* args = Py_BuildValue("(Li)", (long long)sv_time(), data);
  PyObject* result = dpi_core_call_function(py_send_read_data, args);
  Py_DECREF(args);
  Py_XDECREF(result);
}

// Cleanup
void apb_cleanup(void) {
  Py_XDECREF(py_get_transaction);
  Py_XDECREF(py_send_read_data);
  Py_XDECREF(py_module);
  DPI_LOG_INFO("APB plugin cleaned up");
}

Python Driver Layer

# apb_driver.py - DPI-callable interface

import os
import importlib

current_sequence = None

def load_test(test_name):
    """Load and initialize a test module."""
    global current_sequence

    try:
        # Dynamically import test module
        module = importlib.import_module(test_name)
        # Call factory method to create sequence
        current_sequence = module.create_sequence()
        print(f"[PY-INFO] Loaded test: {test_name}")
        return True
    except ImportError as e:
        print(f"[PY-ERROR] Failed to import {test_name}: {e}")
        return False

def get_transaction(sim_time):
    """Return next transaction tuple or None if done."""
    global current_sequence

    if current_sequence is None:
        return None

    tx = current_sequence.get_next()
    if tx is None:
        return None

    # Return tuple: (is_write, addr, data)
    return (1 if tx.is_write else 0, tx.addr, tx.data)

def send_read_data(sim_time, data):
    """Receive read data from simulation."""
    global current_sequence

    if current_sequence:
        current_sequence.receive_read_data(sim_time, data)

APB Sequence Base Class

# apb_base.py - Base classes for APB sequences

class APBTransaction:
    """Single APB transaction."""
    def __init__(self, addr, data, is_write):
        self.addr = addr
        self.data = data
        self.is_write = is_write

    def __repr__(self):
        op = "WR" if self.is_write else "RD"
        return f"{op} addr=0x{self.addr:08x} data=0x{self.data:08x}"


class APBSequence:
    """Base class for APB test sequences."""
    def __init__(self):
        self.transactions = []
        self.index = 0
        self.read_responses = []

    def write(self, addr, data):
        """Add write transaction. Returns self for chaining."""
        self.transactions.append(APBTransaction(addr, data, True))
        return self

    def read(self, addr):
        """Add read transaction. Returns self for chaining."""
        self.transactions.append(APBTransaction(addr, 0, False))
        return self

    def get_next(self):
        """Get next transaction or None if done."""
        if self.index >= len(self.transactions):
            return None
        tx = self.transactions[self.index]
        self.index += 1
        return tx

    def receive_read_data(self, sim_time, data):
        """Store read response for verification."""
        self.read_responses.append((sim_time, data))
        print(f"[PY-INFO] @{sim_time}: Read data = 0x{data:08x}")

SystemVerilog Integration

// apb_python_seq.sv - UVM sequence using DPI bridge

class apb_python_seq extends uvm_sequence #(apb_seq_item);

  `uvm_object_utils(apb_python_seq)

  // DPI-C imports
  import "DPI-C" function int apb_init(string test_name);
  import "DPI-C" function int dpi_get_transaction(
    output int addr,
    output int data,
    output byte is_write
  );
  import "DPI-C" function void dpi_send_read_data(int data);
  import "DPI-C" function void apb_cleanup();

  function new(string name = "apb_python_seq");
    super.new(name);
  endfunction

  task body();
    apb_seq_item req;
    int addr, data;
    byte is_write;
    string test_name;

    // Get test name from plusarg or environment
    if (!$value$plusargs("APB_TEST=%s", test_name)) begin
      test_name = "apb_basic_test";
    end

    // Initialize Python and load test
    if (apb_init(test_name) != 0) begin
      `uvm_fatal("DPI", $sformatf("Failed to init APB plugin with %s", test_name))
    end

    // Fetch and drive all transactions
    while (dpi_get_transaction(addr, data, is_write)) begin
      req = apb_seq_item::type_id::create("req");
      start_item(req);
      req.addr  = addr;
      req.data  = data;
      req.write = is_write;
      finish_item(req);

      `uvm_info("DPI_SEQ", $sformatf("%s addr=0x%0h data=0x%0h",
        is_write ? "WRITE" : "READ", addr, data), UVM_MEDIUM)

      // Send read data back to Python
      if (!is_write) begin
        dpi_send_read_data(req.data);
      end
    end

    // Cleanup
    apb_cleanup();
    `uvm_info("DPI_SEQ", "Python sequence complete", UVM_LOW)
  endtask

endclass

Building the DPI Library

#!/bin/bash
# build_dpi.sh - Compile DPI bridge shared library

# Get Python include/lib paths
PY_INCLUDES=$(python3-config --includes)
PY_LDFLAGS=$(python3-config --ldflags --embed)

# Xilinx xsim DPI header location
XSIM_INCLUDE="-I${XILINX_VIVADO}/data/xsim/include"

# Source files
SRCS="dpi_bridge/dpi_bridge.c \
      dpi_bridge/core/dpi_core.c \
      dpi_bridge/core/dpi_registry.c \
      dpi_bridge/plugins/apb/apb_plugin.c"

# Compile shared library
gcc -shared -fPIC -o libdpi_bridge.so \
    ${SRCS} \
    ${PY_INCLUDES} \
    ${XSIM_INCLUDE} \
    ${PY_LDFLAGS} \
    -Idpi_bridge/core \
    -Idpi_bridge/plugins

# Create symlink for Xilinx tools (expects .so without lib prefix)
ln -sf libdpi_bridge.so dpi_bridge.so

echo "DPI bridge built: dpi_bridge.so"

Memory Management: Reference Counting

Python uses reference counting for memory management. The DPI bridge must follow these rules:

Function	Effect	When to Use
`Py_INCREF(obj)`	Increment reference count	When storing object for later use
`Py_DECREF(obj)`	Decrement reference count	When done with borrowed object
`Py_XDECREF(obj)`	Safe decrement (handles NULL)	Cleanup when object might be NULL

// WRONG: Memory leak - result never released
PyObject* result = PyObject_CallObject(func, args);
// ... use result ...
// Missing: Py_DECREF(result);

// CORRECT: Proper cleanup
PyObject* result = PyObject_CallObject(func, args);
if (result != NULL) {
  // ... use result ...
  Py_DECREF(result);  // Release when done
}

// CORRECT: Safe cleanup for possibly-NULL objects
void cleanup(void) {
  Py_XDECREF(py_module);       // Safe even if NULL
  Py_XDECREF(py_get_transaction);
  Py_XDECREF(py_send_read_data);
}

Common Pitfalls

1. Forgetting to Initialize Python

// WRONG: Calling Python API before initialization
PyObject* module = PyImport_ImportModule("test");  // CRASH!

// CORRECT: Always initialize first
Py_Initialize();
PyObject* module = PyImport_ImportModule("test");

2. GIL (Global Interpreter Lock) Issues

// For multi-threaded simulations, acquire GIL before Python calls
PyGILState_STATE gstate = PyGILState_Ensure();

// ... Python API calls ...

PyGILState_Release(gstate);

3. Module Path Issues

// WRONG: Module not found because path not set
PyObject* module = PyImport_ImportModule("my_test");  // ImportError!

// CORRECT: Add path before importing
PyObject* sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString("./sim/tests"));
PyObject* module = PyImport_ImportModule("my_test");  // Works!

4. Library Linking Errors

# WRONG: Missing Python library
gcc -shared -o dpi.so dpi.c  # Undefined symbols!

# CORRECT: Include Python flags
gcc -shared -o dpi.so dpi.c $(python3-config --ldflags --embed)

Interview Questions

Q1: What is DPI-C and how does it work?

Answer: DPI-C (Direct Programming Interface for C) allows SystemVerilog to call C functions using import "DPI-C" and C to call SystemVerilog using export "DPI-C". The simulator compiles the C code into a shared library (.so/.dll) that's loaded at runtime. Data types are mapped between SV and C according to the LRM specification.

Q2: How do you handle memory management when calling Python from C?

Answer: Python uses reference counting. When storing a PyObject for later use, call Py_INCREF(). When done, call Py_DECREF() to release it. Use Py_XDECREF() for safe cleanup of possibly-NULL pointers. Failing to decrement causes memory leaks; decrementing too much causes crashes.

Q3: What is the GIL and when does it matter for DPI?

Answer: The Global Interpreter Lock (GIL) ensures only one thread executes Python bytecode at a time. For multi-threaded simulations, acquire the GIL with PyGILState_Ensure() before any Python API calls and release it with PyGILState_Release() afterward. Single-threaded simulations don't need explicit GIL handling.

Q4: How do you debug DPI integration issues?

Answer: Use PyErr_Print() to display Python tracebacks when function calls fail. Add logging macros at the C layer. Check library paths with ldd on Linux. Verify Python paths are correctly set in sys.path. Use $display on the SV side to trace DPI call sequences.

Key Takeaways

DPI-C enables SV↔C function calls via import/export declarations
Data types map directly: int↔int, string↔char*, etc.
Python C API manages interpreter lifecycle: Py_Initialize(), PyImport_ImportModule()
Reference counting is critical: Py_INCREF/DECREF for memory management
Plugin architecture separates core logic from protocol-specific code
Error handling: Always check return values, use PyErr_Print() for debugging
Build correctly: Include Python flags in gcc command

Next post: Writing Python Tests - Advanced patterns, debugging, and coverage-driven generation.

Full implementation: github.com/mayurkubavat/SV-Python

2. Python-UVM Integration - Building the DPI-C Bridge