Ctrl+K

Visualize DVS Input

Visualize DVS Input#

Speck integrates a Dynamic Vision Sensor(DVS) on the chip itself. Users can read, visualize and save the events that generated by the embedded DVS after making some modification on the hardware configuration.

This notebook demonstrates how to visualize the DVS Sensor events on the speck2e devkit. Unlike what demonstrated in the “quick start with nmnist”, instead of relying on DynapcnnNetwork and DynapcnnVisualizer to built the “hardware configuration”, “samna graph” and the GUI window, we show a way that build those necessary objects from scratch.

import sinabs.backend.dynapcnn.io as sio

1. Open devkit#

# list all the device we plugged to our PC
device_map = sio.get_device_map()
print(device_map)

{'speck2edevkit:0': device::DeviceInfo(serial_number=, usb_bus_number=0, usb_device_address=5, logic_version=0, device_type_name=Speck2eDevKit)}

# when open devkit, we just need to pass the device name to the `open_device` function of samna.device
devkit = sio.open_device("speck2edevkit:0")

2. Build samna graph#

Samna graph defines how the data flow between the devkit and host machine. In this case, we can send the raw DVS input events to a VizEventStreamer node which can help us to visualize them.

samna_graph

# Build a graph in samna to show dvs
import samna

samna_graph = samna.graph.EventFilterGraph()

_, _, streamer = samna_graph.sequential([
    devkit.get_model_source_node(),  # Specify the source of events to this graph as the devkit
    "Speck2eDvsToVizConverter",   # Convert the events to visualizer events
    "VizEventStreamer"  # Stream events to a visualizer via a streamer node
])

3. Lauch visualizer window#

The samna visualizer setup runs over the network. Any data or configuration therefore happens over a user-specified tcp port.

# Specify the tcp port of the visualizer
visualizer_port = "tcp://0.0.0.0:40000"

# Launch visualizer
gui_process = sio.launch_visualizer(receiver_endpoint=visualizer_port, disjoint_process=True)

3.1 Setup the visualizer#

In order to be able to configure the visualizer, we will need to create another graph node visualizer_config (BasicSourceNode) to connect to the streamer.

# Visualizer configuration branch of the graph.
visualizer_config, _ = samna_graph.sequential([
    samna.BasicSourceNode_ui_event(),  # For generating UI commands
    streamer
])

Let us now connect the streamer to the visualizer window that we launched above. We do this by specifying the destination port of the streamer.

# Connect to the visualizer
streamer.set_streamer_destination(visualizer_port)
if streamer.wait_for_receiver_count() == 0:
    raise Exception(f'Connecting to visualizer on {visualizer_port} fails.')

Finally, we can now configure the exact plots that will want to have in the vizualizer window. In this example, we simply want to launch one ActivityPlot. So we will specify how it needs to be configured using the ActivityPlotConfiguration and pass it to the visualizer.

# Specify which plot is to be shown in the visualizer
plot1 = samna.ui.ActivityPlotConfiguration(image_width=128, image_height=128, title="DVS Layer", layout=[0, 0, 1, 1])
visualizer_config.write([
    samna.ui.VisualizerConfiguration(plots=[plot1])
])

4. Launch the devkit and start visualization#

4.1 start samna graph#

samna_graph.start()

True

4.2 Enable the DVS array on speck#

The configuration of Speck2e devkit has an attribute named dvs_layer which is an instance of samna.speck2e.configuration.DvsLayerConfig.

There are two attribute for a DvsLayerConfig we can set to be True if we wanted to monitor the input spikes from the dvs sensor:

  1. monitor_enable: which let us monitor the pre-processing block of the devkit. The output events from the pre-processing block are Spike with the .layer attribute equals to 13. If user doesn’t apply any filtering, cropping, mirroring etc. in the dvs_layer, collected Spikes should be the same as the raw DvsEvent.

  2. raw_monitor_enable: which let us monitor the raw events from the dvs, the event type of the raw events is DvsEvent.

For this example, we will monitor the raw events produced by the DVS sensor.

devkit_config = samna.speck2e.configuration.SpeckConfiguration()
# enable monitoring the inputs from the DVS sensor 
devkit_config.dvs_layer.raw_monitor_enable = True
# Apply this configuration
devkit.get_model().apply_configuration(devkit_config)

5. Stop the devkit and visualization#

The samna graph is the central engine that routes and handles events off the chip. So stopping the graph is how you can terminate the visualization process. Note that this doesn’t terminate the chip/device from generating events. It just stops those events from being processed downstream.

# Stop the graph
samna_graph.stop()

If the visualizer is run in a sub-prcess you can terminate that window. Otherwise, you can simply close the window manually!

# If we used a sub-process to launch the visualizer, use that to terminate the visualizer. 
if gui_process:
    gui_process.terminate()
    gui_process.join()
Contents