mirror of https://github.com/F-Stack/f-stack.git
290 lines
11 KiB
ReStructuredText
290 lines
11 KiB
ReStructuredText
.. SPDX-License-Identifier: BSD-3-Clause
|
|
Copyright (c) 2023 Marvell.
|
|
|
|
Event DMA Adapter Library
|
|
=========================
|
|
|
|
DPDK :doc:`eventdev library <eventdev>` provides event driven programming model
|
|
with features to schedule events.
|
|
:doc:`DMA device library <dmadev>` provides an interface to DMA poll mode drivers
|
|
that support DMA operations.
|
|
Event DMA adapter is intended to bridge between the event device and the DMA device.
|
|
|
|
Packet flow from DMA device to the event device can be accomplished
|
|
using software and hardware based transfer mechanisms.
|
|
The adapter queries an eventdev PMD to determine which mechanism to be used.
|
|
The adapter uses an EAL service core function for software-based packet transfer
|
|
and uses the eventdev PMD functions to configure hardware-based packet transfer
|
|
between DMA device and the event device.
|
|
DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV``
|
|
to indicate the source of event.
|
|
|
|
Application can choose to submit a DMA operation directly to a DMA device
|
|
or send it to a DMA adapter via eventdev
|
|
based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability.
|
|
The first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode
|
|
and the second as the event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode.
|
|
Choice of mode can be specified while creating the adapter.
|
|
In the former mode, it is the application's responsibility to enable ingress packet ordering.
|
|
In the latter mode, it is the adapter's responsibility to enable ingress packet ordering.
|
|
|
|
|
|
Adapter Modes
|
|
-------------
|
|
|
|
RTE_EVENT_DMA_ADAPTER_OP_NEW mode
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode,
|
|
application submits DMA operations directly to an DMA device.
|
|
The adapter then dequeues DMA completions from the DMA device
|
|
and enqueues them as events to the event device.
|
|
This mode does not ensure ingress ordering
|
|
as the application directly enqueues to the dmadev without going through DMA/atomic stage.
|
|
In this mode, events dequeued from the adapter are treated as new events.
|
|
The application has to specify event information (response information)
|
|
which is needed to enqueue an event after the DMA operation is completed.
|
|
|
|
.. _figure_event_dma_adapter_op_new:
|
|
|
|
.. figure:: img/event_dma_adapter_op_new.*
|
|
|
|
Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode
|
|
|
|
|
|
RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode,
|
|
if the event PMD and DMA PMD supports internal event port
|
|
(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``),
|
|
the application should use ``rte_event_dma_adapter_enqueue()`` API
|
|
to enqueue DMA operations as events to DMA adapter.
|
|
If not, application retrieves DMA adapter's event port
|
|
using ``rte_event_dma_adapter_event_port_get()`` API,
|
|
links its event queue to this port
|
|
and starts enqueuing DMA operations as events to eventdev
|
|
using ``rte_event_enqueue_burst()``.
|
|
The adapter then dequeues the events
|
|
and submits the DMA operations to the dmadev.
|
|
After the DMA operation is complete,
|
|
the adapter enqueues events to the event device.
|
|
|
|
Applications can use this mode when ingress packet ordering is needed.
|
|
In this mode, events dequeued from the adapter will be treated as forwarded events.
|
|
Application has to specify event information (response information)
|
|
needed to enqueue the event after the DMA operation has completed.
|
|
|
|
.. _figure_event_dma_adapter_op_forward:
|
|
|
|
.. figure:: img/event_dma_adapter_op_forward.*
|
|
|
|
Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode
|
|
|
|
|
|
API Overview
|
|
------------
|
|
|
|
This section has a brief introduction to the event DMA adapter APIs.
|
|
The application is expected to create an adapter
|
|
which is associated with a single eventdev,
|
|
then add dmadev and vchan to the adapter instance.
|
|
|
|
|
|
Create an adapter instance
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
An adapter instance is created using ``rte_event_dma_adapter_create()``.
|
|
This function is called with event device
|
|
to be associated with the adapter and port configuration
|
|
for the adapter to setup an event port (if the adapter needs to use a service function).
|
|
|
|
Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW``
|
|
or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode.
|
|
|
|
.. code-block:: c
|
|
|
|
enum rte_event_dma_adapter_mode mode;
|
|
struct rte_event_dev_info dev_info;
|
|
struct rte_event_port_conf conf;
|
|
uint8_t evdev_id;
|
|
uint8_t dma_id;
|
|
int ret;
|
|
|
|
ret = rte_event_dev_info_get(dma_id, &dev_info);
|
|
|
|
conf.new_event_threshold = dev_info.max_num_events;
|
|
conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
|
|
conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
|
|
mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
|
|
ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);
|
|
|
|
|
|
``rte_event_dma_adapter_create_ext()`` function can be used by the application
|
|
to have a finer control on eventdev port allocation and setup.
|
|
The ``rte_event_dma_adapter_create_ext()`` function is passed a callback function.
|
|
The callback function is invoked if the adapter creates a service function
|
|
and uses an event port for it.
|
|
The callback is expected to fill the ``struct rte_event_dma_adapter_conf`` passed to it.
|
|
|
|
In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode,
|
|
if the event PMD and DMA PMD supports internal event port
|
|
(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``),
|
|
events with DMA operations should be enqueued to the DMA adapter
|
|
using ``rte_event_dma_adapter_enqueue()`` API.
|
|
If not, the event port created by the adapter can be retrieved
|
|
using ``rte_event_dma_adapter_event_port_get()`` API.
|
|
An application can use this event port to link with an event queue,
|
|
on which it enqueues events towards the DMA adapter using ``rte_event_enqueue_burst()``.
|
|
|
|
.. code-block:: c
|
|
|
|
uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
|
|
struct rte_event ev;
|
|
uint32_t cap;
|
|
int ret;
|
|
|
|
/* Fill in event info and update event_ptr with rte_dma_op */
|
|
memset(&ev, 0, sizeof(ev));
|
|
.
|
|
.
|
|
ev.event_ptr = op;
|
|
|
|
ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
|
|
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
|
|
ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
|
|
} else {
|
|
ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
|
|
ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
|
|
ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
|
|
ev.queue_id = app_qid;
|
|
ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
|
|
}
|
|
|
|
|
|
Event device configuration for service based adapter
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
|
|
``rte_event_dev_config::nb_event_ports`` is automatically incremented,
|
|
and event device is reconfigured with additional event port during service initialization.
|
|
This event device reconfigure logic also
|
|
increments the ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
|
|
if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.
|
|
|
|
Applications using this mode of adapter creation need not configure the event device
|
|
with ``rte_event_dev_config::nb_event_ports`` and
|
|
``rte_event_dev_config::nb_single_link_event_port_queues`` parameters
|
|
required for DMA adapter when the adapter is created using the above-mentioned API.
|
|
|
|
|
|
Querying adapter capabilities
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The ``rte_event_dma_adapter_caps_get()`` function allows the application
|
|
to query the adapter capabilities for an eventdev and dmadev combination.
|
|
This API provides whether dmadev and eventdev are connected using internal HW port or not.
|
|
|
|
.. code-block:: c
|
|
|
|
rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);
|
|
|
|
|
|
Adding vchan to the adapter instance
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
dmadev device ID and vchan are configured using dmadev APIs.
|
|
For more information, see :doc:`dmadev <dmadev>`.
|
|
|
|
.. code-block:: c
|
|
|
|
struct rte_dma_vchan_conf vchan_conf;
|
|
struct rte_dma_conf dev_conf;
|
|
uint8_t dev_id = 0;
|
|
uint16_t vchan = 0;
|
|
|
|
rte_dma_configure(dev_id, &dev_conf);
|
|
rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);
|
|
|
|
These dmadev ID and vchan are added to the instance
|
|
using the ``rte_event_dma_adapter_vchan_add()`` API.
|
|
The same is removed using ``rte_event_dma_adapter_vchan_del()`` API.
|
|
If hardware supports ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability,
|
|
event information must be passed to the add API.
|
|
|
|
.. code-block:: c
|
|
|
|
uint32_t cap;
|
|
int ret;
|
|
|
|
ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
|
|
if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
|
|
struct rte_event event;
|
|
|
|
rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
|
|
} else
|
|
rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);
|
|
|
|
|
|
Configuring service function
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
If the adapter uses a service function, the application is required
|
|
to assign a service core to the service function as show below.
|
|
|
|
.. code-block:: c
|
|
|
|
uint32_t service_id;
|
|
|
|
if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
|
|
rte_service_map_lcore_set(service_id, CORE_ID);
|
|
|
|
|
|
Set event response information
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` / ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode,
|
|
the application specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op``
|
|
and the event information (response information)
|
|
needed to enqueue an event after the DMA operation has completed.
|
|
The response information is specified in ``struct rte_event``
|
|
and appended to the ``struct rte_event_dma_adapter_op``.
|
|
|
|
|
|
Start the adapter instance
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The application calls ``rte_event_dma_adapter_start()`` to start the adapter.
|
|
This function calls the start callbacks of the eventdev PMDs
|
|
for hardware-based eventdev-dmadev connections
|
|
and ``rte_service_run_state_set()`` to enable the service function if one exists.
|
|
|
|
.. code-block:: c
|
|
|
|
rte_event_dma_adapter_start(id);
|
|
|
|
.. note::
|
|
|
|
The eventdev to which the event_dma_adapter is connected should be started
|
|
before calling ``rte_event_dma_adapter_start()``.
|
|
|
|
|
|
Get adapter statistics
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The ``rte_event_dma_adapter_stats_get()`` function reports counters
|
|
defined in ``struct rte_event_dma_adapter_stats``.
|
|
The received packet and enqueued event counts are a sum of the counts
|
|
from the eventdev PMD callbacks if the callback is supported,
|
|
and the counts maintained by the service function, if one exists.
|
|
|
|
|
|
Set/Get adapter runtime configuration parameters
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The runtime configuration parameters of adapter can be set/get using
|
|
``rte_event_dma_adapter_runtime_params_set()`` and
|
|
``rte_event_dma_adapter_runtime_params_get()`` respectively.
|
|
The parameters that can be set/get are defined in
|
|
``struct rte_event_dma_adapter_runtime_params``.
|