Discussion:
[very-RFC 0/8] TSN driver for the kernel
(too old to reply)
Henrik Austad
2016-06-11 23:01:49 UTC
Permalink
Hi all
(series based on v4.7-rc2, now with the correct netdev)

This is a *very* early RFC for a TSN-driver in the kernel. It has been
floating around in my repo for a while and I would appreciate some
feedback on the overall design to avoid doing some major blunders.

TSN: Time Sensitive Networking, formely known as AVB (Audio/Video
Bridging).

There are at least one AVB-driver (the AV-part of TSN) in the kernel
already, however this driver aims to solve a wider scope as TSN can do
much more than just audio. A very basic ALSA-driver is added to the end
that allows you to play music between 2 machines using aplay in one end
and arecord | aplay on the other (some fiddling required) We have plans
for doing the same for v4l2 eventually (but there are other fishes to
fry first). The same goes for a TSN_SOCK type approach as well.

TSN is all about providing infrastructure. Allthough there are a few
very interesting uses for TSN (reliable, deterministic network for audio
and video), once you have that reliable link, you can do a lot more.

Some notes on the design:

The driver is directed via ConfigFS as we need userspace to handle
stream-reservation (MSRP), discovery and enumeration (IEEE 1722.1) and
whatever other management is needed. Once we have all the required
attributes, we can create link using mkdir, and use write() to set the
attributes. Once ready, specify the 'shim' (basically a thin wrapper
between TSN and another subsystem) and we start pushing out frames.

The network part: it ties directly into the rx-handler for receive and
writes skb's using netdev_start_xmit(). This could probably be
improved. 2 new fields in netdev_ops have been introduced, and the Intel
igb-driver has been updated (as this is available as a PCI-e card). The
igb-driver works-ish


What remains
- tie to (g)PTP properly, currently using ktime_get() for presentation
time
- get time from shim into TSN and vice versa
- let shim create/manage buffer

Henrik Austad (8):
TSN: add documentation
TSN: Add the standard formerly known as AVB to the kernel
Adding TSN-driver to Intel I210 controller
Add TSN header for the driver
Add TSN machinery to drive the traffic from a shim over the network
Add TSN event-tracing
AVB ALSA - Add ALSA shim for TSN
MAINTAINERS: add TSN/AVB-entries

Documentation/TSN/tsn.txt | 147 +++++
MAINTAINERS | 14 +
drivers/media/Kconfig | 15 +
drivers/media/Makefile | 3 +-
drivers/media/avb/Makefile | 5 +
drivers/media/avb/avb_alsa.c | 742 +++++++++++++++++++++++
drivers/media/avb/tsn_iec61883.h | 124 ++++
drivers/net/ethernet/intel/Kconfig | 18 +
drivers/net/ethernet/intel/igb/Makefile | 2 +-
drivers/net/ethernet/intel/igb/igb.h | 19 +
drivers/net/ethernet/intel/igb/igb_main.c | 10 +-
drivers/net/ethernet/intel/igb/igb_tsn.c | 396 ++++++++++++
include/linux/netdevice.h | 32 +
include/linux/tsn.h | 806 ++++++++++++++++++++++++
include/trace/events/tsn.h | 349 +++++++++++
net/Kconfig | 1 +
net/Makefile | 1 +
net/tsn/Kconfig | 32 +
net/tsn/Makefile | 6 +
net/tsn/tsn_configfs.c | 623 +++++++++++++++++++
net/tsn/tsn_core.c | 975 ++++++++++++++++++++++++++++++
net/tsn/tsn_header.c | 203 +++++++
net/tsn/tsn_internal.h | 383 ++++++++++++
net/tsn/tsn_net.c | 403 ++++++++++++
24 files changed, 5306 insertions(+), 3 deletions(-)
create mode 100644 Documentation/TSN/tsn.txt
create mode 100644 drivers/media/avb/Makefile
create mode 100644 drivers/media/avb/avb_alsa.c
create mode 100644 drivers/media/avb/tsn_iec61883.h
create mode 100644 drivers/net/ethernet/intel/igb/igb_tsn.c
create mode 100644 include/linux/tsn.h
create mode 100644 include/trace/events/tsn.h
create mode 100644 net/tsn/Kconfig
create mode 100644 net/tsn/Makefile
create mode 100644 net/tsn/tsn_configfs.c
create mode 100644 net/tsn/tsn_core.c
create mode 100644 net/tsn/tsn_header.c
create mode 100644 net/tsn/tsn_internal.h
create mode 100644 net/tsn/tsn_net.c

--
2.7.4
Henrik Austad
2016-06-11 23:01:50 UTC
Permalink
From: Henrik Austad <***@cisco.com>

Describe the overall design behind the TSN standard, the TSN-driver,
requirements to userspace and new functionality introduced.

Cc: "David S. Miller" <***@davemloft.net>
Signed-off-by: Henrik Austad <***@cisco.com>
---
Documentation/TSN/tsn.txt | 147 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 147 insertions(+)
create mode 100644 Documentation/TSN/tsn.txt

Index: linux/Documentation/TSN/tsn.txt
===================================================================
--- /dev/null
+++ linux/Documentation/TSN/tsn.txt
@@ -0,0 +1,188 @@
+ Time Sensitive Networking (TSN)
+ -------------------------------
+
+[work in progress]
+
+1. Motivation
+=============
+
+TSN is a set of open standards, formerly known as 'AVB' (Audio/Video
+Bridging). It was renamed to TSN to better reflect that it can do much
+more than just media transport.
+
+TSN is a way to create reliable streams across a network without loss of
+frames due to congestion in the network. By using gPTP (a specialized
+IEEE-1588v2 PTP profile), the time can be synchronized with sub-us
+granularity across all the connected devices in the AVB domain.
+
+2. Intro to AVB/TSN
+===================
+
+The original standards were written with Audio/Video in mind, so the
+initial standards refer to this as 'AVB'. In later standards, this has
+changed to TSN, and AVB now refers to a service you can add on top of
+TSN. Hopefully it will not be too confusing.
+
+In this document, we refer to the infrastructure part as TSN and AVB to
+the ALSA/V4L2 shim which can be added on top of TSN to provide a
+media-service.
+
+TSN operates with 'streams', and one stream can contain pretty much
+whatever you like. Currently, only media has been defined properly
+though, which is why you only have media-subtypes for the
+avtp_subtype-field.
+
+For a media-setup, one stream can contain multiple channels, all going
+to the same destination. A destination can be a single Listener
+(singlecast) or a group of Listeners (multicast).
+
+2.1 Endpoints
+
+A TSN 'endpoint' is where a stream either originates or ends -what
+others would call sources (Talkers) and sinks (Listeners). Looking back
+at pre-TSN when this was called AVB, these names make a bit more sense.
+
+Common for both types, they need to be PTPv2 capable, i.e. you need to
+timestamp gPTP frames upon ingress/egress to improve the accuracy of
+PTP.
+
+2.1.1 Talkers
+
+Hardware requirements:
+- Multiple Tx-queues
+- Credit based shaper on at least one of the queues for pacing the
+ frames onto the network
+- VLAN capable
+
+2.1.2 Listener
+
+A Listener does not have the same requirements as a Talker as it cannot
+control the pace of the incoming frames anyway. It is beneficial if the
+NIC understands VLANs and has a few Rx-queues so that you can steer all
+TSN-frames to a dedicated queue.
+
+2.2 Bridges
+
+What TSN calls switches that are TSN-capable. They must be able to
+prioritize TSN-streams, have the credit-based shaper available for that
+class, support SRP, support gPTP and so on.
+
+2.3 Relevant standards
+
+* IEEE 802.1BA-2011 Audio Video Bridging (AVB) Systems
+
+* IEEE 802.1Q-2011 sec 34 and 35
+
+ What is referred to as:
+ IEEE 802.1Qav (Forwarding and Queueing for Time-sensitive Streams)
+ IEEE 802.1Qat (Stream Registration protocol)
+
+* IEEE 802.1AS gPTP
+
+ A PTPv2 profile (from IEEE 1588) tailored for this domain. Notable
+ changes include the requirement that all nodes in the network must be
+ gPTP capable (i.e. no traversing non-PTP entities), and it allows
+ traffic over a wider range of medium that what "pure" PTPv2 allows.
+
+* IEEE 1722 AVTP Layer 2 Transport Protocol for Time-Sensitive
+ Applications in Bridged Local Area Networks
+
+* IEEE 1722.1 Device Discovery, Connection Management and Control for 1722
+
+ What allows AVB (TSN) devices to handle discovery, enumeration and
+ control, basically let you connect 2 devices from a 3rd
+
+ In this (in the scope of the Linux kernel TSN driver) must be done
+ purely from userspace as we do not want the kernel to suddenly attach
+ to a remote system without the user's knowledge. This is further
+ reflected in how the attributes for the link is managed via ConfigFS.
+
+
+3. Overview and/or design of the TSN-driver
+===========================================
+
+The driver handles the shifting of data for TSN-streams. Anything else
+is left for userspace to handle. This includes stream reservation (using
+some sort of MSRP client), negotiating multicast addresses, finding the
+value of the different attributes and connect application(s) to the
+exposed devices (currently we only have an ALSA-device).
+
+ /--------------------\
+ | |
+ | Media application |
+ | |
+ \--------------------/
+ | |
+ +----------+ +----+
+ | |
+ | |
+ +------------+ |
+ | ALSA | |
+ +------------+ |
+ | |
+ | |
+ +------------+ +--------------+
+ | avb_alsa | | tsn_configfs |
+ | (tsn-shim) | +--------------+
+ +------------+ |
+ | |
+ | |
+ +------+ |
+ | |
+ | |
+ +------------+ |
+ | tsn_core |<--------+
+ +------------+
+ |
+ |
+ +------------+
+ | tsn_net |
+ +------------+
+ |
+ |
+ +------------+
+ | network |
+ | subsystem |
+ +------------+
+ |
+ |
+ ...
+
+
+3.1 Terms and concepts
+
+TSN uses the concept of streams and shims.
+
+- A shim is a thin wrapper that binds TSN to another subsystem (or
+ directly to userspace). avb_alsa is an example of such a shim.
+
+- A stream is the only data TSN cares about. What the data inside the
+ stream represents, is left for the associated shim to handle. TSN will
+ verify the headers up to the protocol specific header and then pass it
+ along to the shim.
+
+Note: currently, only the data-unit part is implemented, the control
+part, in which 1722.1 (discovery and enumeration) is part, is not
+handled.
+
+3.2 Userspace requirements
+
+(msrp-client, "tsnctl"-tool
+
+4. Creating a new link from userspace
+=====================================
+
+[coming]
+
+
+5. Creating a new shim
+======================
+
+shim_ops
+[coming]
+
+
+6. Other resources:
+===================
+
+https://en.wikipedia.org/wiki/Audio_Video_Bridging
Henrik Austad
2016-06-11 23:01:59 UTC
Permalink
From: Henrik Austad <***@cisco.com>

This exposes a *very* rudimentary and simplistic ALSA driver that hooks
into TSN to create a device for userspace.

It currently only supports 44.1/48kHz sampling, 2ch, S16_LE

Userspace is supposed to reserve bandwidth, find StreamID etc.

To use as a Talker:

mkdir /config/tsn/test/eth0/talker
cd /config/tsn/test/eth0/talker
echo 65535 > buffer_size
echo 08:00:27:08:9f:c3 > remote_mac
echo 42 > stream_id
echo alsa > enabled

aplay -Ddefault:CARD=avb -c2 -r48000 -fS16_LE /opt/rickroll.wav

The same applies to Listener and arecord.

Cc: Mauro Carvalho Chehab <***@osg.samsung.com>
Cc: Takashi Iwai <***@suse.de>
Cc: Mark Brown <***@kernel.org>
Signed-off-by: Henrik Austad <***@cisco.com>
---
drivers/media/Kconfig | 15 +
drivers/media/Makefile | 3 +-
drivers/media/avb/Makefile | 5 +
drivers/media/avb/avb_alsa.c | 742 +++++++++++++++++++++++++++++++++++++++
drivers/media/avb/tsn_iec61883.h | 124 +++++++
5 files changed, 888 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/avb/Makefile
create mode 100644 drivers/media/avb/avb_alsa.c
create mode 100644 drivers/media/avb/tsn_iec61883.h

diff --git a/drivers/media/Kconfig b/drivers/media/Kconfig
index a8518fb..14ad1d9 100644
--- a/drivers/media/Kconfig
+++ b/drivers/media/Kconfig
@@ -217,3 +217,18 @@ source "drivers/media/tuners/Kconfig"
source "drivers/media/dvb-frontends/Kconfig"

endif # MEDIA_SUPPORT
+
+config MEDIA_AVB_ALSA
+ tristate "ALSA part of AVB over TSN"
+ depends on TSN
+ help
+
+ Enable the ALSA device that hoooks into TSN and allows the
+ computer to send ethernet frames over the network carrying
+ audio-data to selected hosts.
+
+ This must be configured by userspace as MSRP and IEEE 1722.1
+ (discovery and enumeration) is not implemented within the
+ kernel.
+
+ If unsure, say N
\ No newline at end of file
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index e608bbc..a1ca09e 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -20,6 +20,7 @@ endif

obj-$(CONFIG_VIDEO_DEV) += v4l2-core/
obj-$(CONFIG_DVB_CORE) += dvb-core/
+obj-$(CONFIG_AVB) += avb/

# There are both core and drivers at RC subtree - merge before drivers
obj-y += rc/
@@ -30,4 +31,4 @@ obj-y += rc/

obj-y += common/ platform/ pci/ usb/ mmc/ firewire/
obj-$(CONFIG_VIDEO_DEV) += radio/
-
+obj-$(CONFIG_MEDIA_AVB_ALSA) += avb/
diff --git a/drivers/media/avb/Makefile b/drivers/media/avb/Makefile
new file mode 100644
index 0000000..5d6302c
--- /dev/null
+++ b/drivers/media/avb/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for the ALSA shim in AVB/TSN
+#
+
+obj-$(CONFIG_MEDIA_AVB_ALSA) += avb_alsa.o
diff --git a/drivers/media/avb/avb_alsa.c b/drivers/media/avb/avb_alsa.c
new file mode 100644
index 0000000..9aff7d3
--- /dev/null
+++ b/drivers/media/avb/avb_alsa.c
@@ -0,0 +1,742 @@
+/* Copyright 2016 Cisco Systems, Inc. and/or its affiliates. All rights
+ * reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#include <linux/platform_device.h>
+#include <sound/pcm_params.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <sound/core.h>
+#include <sound/pcm.h>
+
+#include <linux/tsn.h>
+#include "tsn_iec61883.h"
+
+struct avb_chip {
+ struct snd_card *card;
+ struct tsn_link *link;
+ struct snd_pcm *pcm;
+ struct snd_pcm_substream *substream;
+
+ /* Need a reference to this when we unregister the platform
+ * driver.
+ */
+ struct platform_device *device;
+
+ /* on first copy, we set a few values, use this to make sure we
+ * only do this once.
+ */
+ u8 first_copy;
+
+ u8 sample_size;
+ u8 channels;
+
+ /* current idx in 10ms set of frames
+ * class A: 80
+ * class B: 40
+ *
+ * This is mostly relevant for 44.1kHz samplefreq
+ */
+ u8 num_10ms_series;
+
+ u32 sample_freq;
+};
+
+/* currently, only playback is implemented in TSN layer
+ *
+
+ * FIXMEs: (should be set according to the active TSN link)
+ * - format
+ * - rates
+ * - channels
+ */
+static struct snd_pcm_hardware snd_avb_hw = {
+ .info = SNDRV_PCM_INFO_INTERLEAVED,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ .rates = SNDRV_PCM_RATE_44100 | SNDRV_PCM_RATE_48000,
+ .rate_min = 44100,
+ .rate_max = 48000,
+ .channels_min = 2,
+ .channels_max = 2,
+ .period_bytes_min = 4096,
+ .period_bytes_max = 32768,
+ .buffer_bytes_max = 32768,
+ .periods_min = 1,
+ .periods_max = 1024,
+ .fifo_size = 0,
+};
+
+static size_t snd_avb_copy_size(struct tsn_link *link);
+
+
+static int _set_chip_values(struct avb_chip *avb_chip,
+ struct snd_pcm_runtime *runtime)
+{
+ if (!avb_chip->first_copy)
+ return 0;
+
+
+ /*
+ * first copy, we now know that runtime has all the correct
+ * values set, we can grab channels and rate. Sample_size
+ * (runtime->format) is currently hard-coded to S16_LE.
+ */
+ avb_chip->channels = runtime->channels;
+ avb_chip->sample_freq = runtime->rate;
+ avb_chip->sample_size = 16;
+
+ if (snd_avb_copy_size(avb_chip->link) > avb_chip->link->max_payload_size) {
+ pr_err("%s: Resulting payload-size is larger (%zd) than available (%u)\n",
+ __func__, snd_avb_copy_size(avb_chip->link),
+ avb_chip->link->max_payload_size);
+ return -EINVAL;
+ }
+ avb_chip->first_copy = 0;
+ return 0;
+}
+
+static int _snd_avb_open(struct avb_chip *avb_chip,
+ struct snd_pcm_runtime *runtime)
+{
+ /*
+ * We do not know what some of these values are until we see the
+ * first copy. We set to sane defaults where we don't have exact
+ * content.
+ */
+ avb_chip->channels = 0;
+ avb_chip->sample_size = 0;
+ avb_chip->sample_freq = 0;
+ avb_chip->num_10ms_series = 0;
+ avb_chip->first_copy = 1;
+
+ runtime->hw = snd_avb_hw;
+ runtime->buffer_size = avb_chip->link->buffer_size;
+ return 0;
+}
+
+/*
+ * bytes_to_frames()
+ * frames_to_bytes()
+ *
+ * frames_to_bytes(runtime, runtrime->period_size);
+ *
+ * Interrupt callbacks:
+ * The field traonsfer_ack_begin and transfer_ack_end are called at the
+ * beginning and at the end of snd_pcm_period_elapsed(), respectively.
+ */
+static int snd_avb_playback_open(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ int ret = 0;
+
+ /*
+ * we've opened the PCM before probe returned properly and
+ * stored link in the struct.
+ */
+ if (!avb_chip || !avb_chip->link) {
+ pr_err("%s: Chip-data or link not available, cannot continue\n",
+ __func__);
+ return -EINVAL;
+ }
+ if (!avb_chip->link->estype_talker) {
+ pr_info("Link (%llu) not registered as Talker, cannot do playback\n",
+ avb_chip->link->stream_id);
+ return -EINVAL;
+ }
+
+ ret = _snd_avb_open(avb_chip, runtime);
+ if (ret < 0) {
+ pr_err("%s: Could not open playback-device (requested %d ch, %zd buffer)",
+ __func__, avb_chip->channels,
+ avb_chip->link->buffer_size);
+ return ret;
+ }
+ pr_info("%s: %d channel PCM stream opened successfully, buffersize: %zd\n",
+ __func__, avb_chip->channels, avb_chip->link->buffer_size);
+ return 0;
+}
+
+static int snd_avb_playback_close(struct snd_pcm_substream *substream)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+
+ tsn_lb_disable(avb_chip->link);
+
+ pr_info("%s: something happened\n", __func__);
+ return 0;
+}
+
+/*
+ * snd_avb snd_avb.0: BUG: ,
+ * pos = 12288,
+ * buffer size = 8192,
+ * period size = 2048
+ *
+ * Playback is when we *send* data to a remote speaker
+ */
+static int snd_avb_playback_copy(struct snd_pcm_substream *substream,
+ int channel,
+ snd_pcm_uframes_t pos,
+ void *src,
+ snd_pcm_uframes_t count)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ size_t bytes;
+ int ret;
+
+ /*
+ * From alsadoc:
+ *
+ * You need to check the channel argument, and if it's -1, copy
+ * the whole channels. Otherwise, you have to copy only the
+ * specified channel. Please check isa/gus/gus_pcm.c as an
+ * example.
+ */
+ if (channel != -1) {
+ pr_err("%s: partial copy not supportet\n", __func__);
+ return -EINVAL;
+ }
+
+ ret = _set_chip_values(avb_chip, runtime);
+ if (ret != 0)
+ return ret;
+
+ bytes = frames_to_bytes(runtime, count);
+ ret = tsn_buffer_write(avb_chip->link, src, bytes);
+ if (ret != bytes) {
+ pr_err("%s: Incorrect copy (%zd, %d) corruption possible\n",
+ __func__, bytes, ret);
+ return -EIO;
+ }
+ return 0;
+}
+
+static int snd_avb_capture_open(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ int ret = 0;
+
+ if (!avb_chip || !avb_chip->link) {
+ pr_err("%s: Chip-data or link not available, cannot continue\n",
+ __func__);
+ return -EINVAL;
+ }
+ if (avb_chip->link->estype_talker) {
+ pr_info("Link (%llu) registered as Talker, cannot capture\n",
+ avb_chip->link->stream_id);
+ return -EINVAL;
+ }
+ ret = _snd_avb_open(avb_chip, runtime);
+ if (ret < 0) {
+ pr_err("%s: Could not open capture-device (requested %d ch, %zd buffer)",
+ __func__, avb_chip->channels,
+ avb_chip->link->buffer_size);
+ return ret;
+ }
+ tsn_lb_enable(avb_chip->link);
+ pr_info("%s: %d channel PCM stream opened successfully, buffersize: %zd\n",
+ __func__, avb_chip->channels, avb_chip->link->buffer_size);
+ return 0;
+}
+
+static int snd_avb_capture_close(struct snd_pcm_substream *substream)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+
+ if (!avb_chip || !avb_chip->link)
+ return -EINVAL;
+ pr_err("%s: closing stream\n", __func__);
+
+ tsn_lb_disable(avb_chip->link);
+
+ return 0;
+}
+
+static int snd_avb_capture_copy(struct snd_pcm_substream *substream,
+ int channel,
+ snd_pcm_uframes_t pos,
+ void *src,
+ snd_pcm_uframes_t count)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ size_t bytes;
+ int ret;
+
+ bytes = frames_to_bytes(runtime, count);
+ ret = tsn_buffer_read(avb_chip->link, src, bytes);
+ if (ret != bytes) {
+ pr_err("%s: incorrect copy (%zd, %d), corrupt capture possible\n",
+ __func__, bytes, ret);
+ tsn_lb_disable(avb_chip->link);
+ return -EIO;
+ }
+ return 0;
+}
+
+static int snd_avb_silence(struct snd_pcm_substream *substream,
+ int channel, snd_pcm_uframes_t pos,
+ snd_pcm_uframes_t count)
+{
+ /* FIXME, should do more than nothing */
+ return 0;
+}
+
+/*
+ * Called when the client defines buffer_size, period_size, format etc
+ * for the pcm substream.
+ *
+ * This is where link->buffer is allocated and link->buffer_size is
+ * defined.
+ *
+ * We are called in the beginning of snd_pcm_hw_params in
+ * sound/core/pcm_native.c, we cannot override runtime-values as they
+ * are updated from hw_params.
+ */
+static int snd_avb_pcm_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *hw_params)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ unsigned int bsize = params_buffer_bytes(hw_params);
+ int ret = 0;
+
+ /* We need this reference for the refill callback so that we can
+ * call snd_pcm_period_elapsed();
+ */
+ avb_chip->substream = substream;
+ ret = tsn_set_buffer_size(avb_chip->link, bsize);
+ if (ret < 0) {
+ pr_err("%s: could not set buffer_size (alsa requested too large? (%d)\n",
+ __func__, ret);
+ goto out;
+ }
+
+ avb_chip->num_10ms_series = 0;
+ pr_info("%s: successfully set hw-params\n", __func__);
+out:
+ return ret;
+}
+
+static int snd_avb_pcm_hw_free(struct snd_pcm_substream *substream)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+
+ if (!avb_chip || !avb_chip->link)
+ return -EINVAL;
+ tsn_clear_buffer_size(avb_chip->link);
+ pr_info("%s: something happened\n", __func__);
+ avb_chip->substream = NULL;
+ return 0;
+}
+
+static int snd_avb_pcm_prepare(struct snd_pcm_substream *substream)
+{
+ /* verify that samplerate, freq and size is what we have set in
+ * the link.
+ */
+
+ return 0;
+}
+
+/*
+ * When the PCM stream is started, stopped, paused etc.
+ *
+ * Atomic function (some lock is being held by PCM layer)
+ */
+static int snd_avb_pcm_trigger(struct snd_pcm_substream *substream,
+ int cmd)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+
+ switch (cmd) {
+ case SNDRV_PCM_TRIGGER_START:
+ case SNDRV_PCM_TRIGGER_RESUME:
+ /* pr_err("%s: starting for some reason\n", __func__); */
+ tsn_lb_enable(avb_chip->link);
+ break;
+ case SNDRV_PCM_TRIGGER_STOP:
+ case SNDRV_PCM_TRIGGER_SUSPEND:
+ /* memset buffer to 0 */
+ /* pr_err("%s: stopping for some reason\n", __func__); */
+ tsn_lb_disable(avb_chip->link);
+ break;
+ default:
+ pr_info("%s: cmd: %d (return -EINVAL)\n", __func__, cmd);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/*
+ * current hw-position in the buffer, in frames from 0 to buffer_size -1
+ *
+ * Need to know where the hw-pointer is and how this corresponds to the
+ * underlying TSN-buffer setup
+ *
+ * Atomic function (some lock is being held by PCM layer)
+ *
+ */
+static snd_pcm_uframes_t snd_avb_pcm_pointer(struct snd_pcm_substream *substream)
+{
+ struct avb_chip *avb_chip = snd_pcm_substream_chip(substream);
+ struct tsn_link *link = avb_chip->link;
+ snd_pcm_uframes_t pointer;
+
+ if (link->estype_talker)
+ pointer = bytes_to_frames(substream->runtime,
+ link->tail - link->buffer);
+ else
+ pointer = bytes_to_frames(substream->runtime,
+ link->head - link->buffer);
+ return pointer;
+}
+
+static struct snd_pcm_ops snd_avb_playback_ops = {
+ .open = snd_avb_playback_open,
+ .close = snd_avb_playback_close,
+ .copy = snd_avb_playback_copy,
+ .silence = snd_avb_silence,
+ .ioctl = snd_pcm_lib_ioctl,
+ .hw_params = snd_avb_pcm_hw_params,
+ .hw_free = snd_avb_pcm_hw_free,
+ .prepare = snd_avb_pcm_prepare,
+ .trigger = snd_avb_pcm_trigger,
+ .pointer = snd_avb_pcm_pointer,
+};
+
+static struct snd_pcm_ops snd_avb_capture_ops = {
+ .open = snd_avb_capture_open,
+ .close = snd_avb_capture_close,
+ .copy = snd_avb_capture_copy,
+ .silence = snd_avb_silence,
+ .ioctl = snd_pcm_lib_ioctl,
+ .hw_params = snd_avb_pcm_hw_params,
+ .hw_free = snd_avb_pcm_hw_free,
+ .prepare = snd_avb_pcm_prepare,
+ .trigger = snd_avb_pcm_trigger,
+ .pointer = snd_avb_pcm_pointer,
+};
+
+/*
+ * Callback for tsn_core for moving data into the buffer.
+ *
+ * This should be a wrapper (replace it with) the refill-functionality ALSA use.
+ */
+static size_t snd_avb_refill(struct tsn_link *link)
+{
+ struct avb_chip *avb_chip = link->media_chip;
+
+ if (avb_chip && avb_chip->substream) {
+ snd_pcm_period_elapsed(avb_chip->substream);
+ return 0;
+ }
+ return -EINVAL;
+}
+
+static size_t snd_avb_drain(struct tsn_link *link)
+{
+ struct avb_chip *avb_chip = link->media_chip;
+
+ if (avb_chip && avb_chip->substream) {
+ snd_pcm_period_elapsed(avb_chip->substream);
+ return 0;
+ }
+ return -EINVAL;
+}
+
+static size_t snd_avb_hdr_size(struct tsn_link *link)
+{
+ /* return the size of the iec61883-6 audio header */
+ return _iec61883_hdr_len();
+}
+
+static size_t snd_avb_copy_size(struct tsn_link *link)
+{
+ struct avb_chip *chip = link->media_chip;
+ /* use values in avb_chip, not link */
+ size_t framesize = (chip->sample_size >> 3) * chip->channels;
+ size_t numframes = 0;
+
+ if (!chip->sample_freq)
+ return link->max_payload_size;
+
+ /* size of each frame (samples per frame, sample-size && class)
+ * sample_size: 16 -> 2
+ * spframe : 12 (class b)
+ * channels: 2
+ *
+ * framesize: 2*12*2 -> 48
+ */
+
+ switch (chip->sample_freq) {
+ case 44100:
+ /*
+ * Class B: 40 frames, first 12 bytes, next 39 should be 11
+ */
+ if (!link->class_a) {
+ numframes = (chip->num_10ms_series ? 11 : 12);
+ chip->num_10ms_series++;
+ if (chip->num_10ms_series > 39)
+ chip->num_10ms_series = 0;
+ } else {
+ /* Class A slightly more involved
+ * Need 41 6 bytes and 39 5 bytes
+ *
+ * If 0th is set to 6, remaining odd idx should
+ * be 6, even (except 0th) to be 6
+ */
+ numframes = 5;
+ if (!chip->num_10ms_series ||
+ (chip->num_10ms_series % 0x2))
+ numframes++;
+ chip->num_10ms_series++;
+ if (chip->num_10ms_series > 79)
+ chip->num_10ms_series = 0;
+ }
+ break;
+ case 48000:
+ numframes = (link->class_a ? 6 : 12);
+ break;
+ default:
+ pr_err("Unsupported sample_freq (%d), disabling link\n",
+ chip->sample_freq);
+ tsn_lb_disable(link);
+ return -EINVAL;
+ }
+ return numframes * framesize;
+}
+
+static void snd_avb_assemble_iidc(struct tsn_link *link,
+ struct avtpdu_header *header, size_t bytes)
+{
+ _iec61883_hdr_assemble(header, bytes);
+}
+
+static int snd_avb_validate_iidc(struct tsn_link *link,
+ struct avtpdu_header *header)
+{
+ return _iec61883_hdr_verify(header);
+}
+
+static void *snd_avb_get_payload_data(struct tsn_link *link,
+ struct avtpdu_header *header)
+{
+ return _iec61883_payload(header);
+}
+
+static int snd_avb_new_pcm(struct avb_chip *avb_chip, int device)
+{
+ struct snd_pcm *pcm;
+ int err;
+
+ err = snd_pcm_new(avb_chip->card, "AVB PCM", device, 1, 1, &pcm);
+ if (err < 0)
+ return err;
+ pcm->private_data = avb_chip;
+ strcpy(pcm->name, "AVB PCM");
+ avb_chip->pcm = pcm;
+
+ /* only playback at the moment, once we implement capture, we
+ * need to grab the Talker/Listener from TSN link
+ */
+ snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK, &snd_avb_playback_ops);
+ snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE, &snd_avb_capture_ops);
+
+ return 0;
+
+}
+static int snd_avb_probe(struct platform_device *devptr)
+{
+ int err;
+ struct snd_card *card;
+ struct avb_chip *avb_chip;
+
+ pr_info("%s: starting\n", __func__);
+
+ /*
+ * older kernel use snd_card_create. This is handled by
+ * tsn_compat.h in an attempt to make it easier to backport to
+ * older kernels.
+ */
+ err = snd_card_new(&devptr->dev, 1, "avb", THIS_MODULE,
+ sizeof(struct avb_chip), &card);
+ if (err < 0) {
+ pr_err("%s: trouble creating new card -> %d\n",
+ __func__, err);
+ return err;
+ }
+ avb_chip = card->private_data;
+ avb_chip->card = card;
+
+
+ /* create PCM device*/
+ err = snd_avb_new_pcm(avb_chip, 0);
+ if (err < 0) {
+ pr_err("%s: could not create new PCM device\n", __func__);
+ goto err_out;
+ }
+
+ /* register card */
+ pr_info("%s: ready to register card\n", __func__);
+ strcpy(card->driver, "Avb");
+ strcpy(card->shortname, "Avb");
+ sprintf(card->longname, "Avb %i", devptr->id + 1);
+ err = snd_card_register(card);
+ if (err < 0) {
+ pr_err("%s: Could not register card -> %d\n",
+ __func__, err);
+ snd_card_free(card);
+ return err;
+ }
+
+ if (err == 0) {
+ platform_set_drvdata(devptr, card);
+ pr_info("%s: Successfully initialized %s\n",
+ __func__, card->shortname);
+ return 0;
+ }
+err_out:
+ snd_card_free(card);
+ return err;
+}
+
+/*
+ * We are here as a result from being removed via
+ * tsn_link->shim_ops->media_close, which is snd_avb_close()
+ */
+static int snd_avb_remove(struct platform_device *devptr)
+{
+ struct snd_card *card = platform_get_drvdata(devptr);
+ struct avb_chip *avb_chip = card->private_data;
+
+ /* Make sure link holds no ref to this now dead card */
+ if (avb_chip && avb_chip->link) {
+ avb_chip->link->media_chip = NULL;
+ avb_chip->link = NULL;
+ }
+
+ /* call into link->ops->media_close() ? */
+ snd_card_free(card);
+ return 0;
+}
+
+static struct platform_driver snd_avb_driver = {
+ .probe = snd_avb_probe,
+ .remove = snd_avb_remove,
+ .driver = {
+ .name = "snd_avb",
+ .pm = NULL, /* don't care about Power Management */
+ },
+};
+
+static int snd_avb_close(struct tsn_link *link)
+{
+ struct avb_chip *avb_chip = link->media_chip;
+
+ if (!link->media_chip)
+ return 0;
+
+ pr_info("%s: Removing device\n", __func__);
+
+ platform_device_unregister(avb_chip->device);
+ /* platform unregister will call into snd_avb_remove */
+ platform_driver_unregister(&snd_avb_driver);
+
+ /* update link to remove pointer to now invalid memory */
+ link->media_chip = NULL;
+ return 0;
+}
+
+static int snd_avb_new(struct tsn_link *link)
+{
+ struct avb_chip *avb_chip;
+ struct snd_card *card;
+ struct platform_device *device;
+ int err;
+
+ err = platform_driver_register(&snd_avb_driver);
+ if (err < 0) {
+ pr_info("%s: trouble registering driver %d, unreg. partial driver and abort.\n",
+ __func__, err);
+ return err;
+ }
+
+ /*
+ * We only register a single card for now, look to
+ * /sys/devices/platform/snd_avb.0 for content.
+ *
+ * Probe will be triggered if name is same as .name in platform_driver
+ */
+ device = platform_device_register_simple("snd_avb", 0, NULL, 0);
+ if (IS_ERR(device)) {
+ pr_info("%s: ERROR registering simple platform-device\n",
+ __func__);
+ platform_driver_unregister(&snd_avb_driver);
+ return -ENODEV;
+ }
+
+ /* store data in driver so we can access it in .probe */
+ card = platform_get_drvdata(device);
+ if (card == NULL) {
+ pr_info("%s: Did not get anything from platform_get_drvdata()\n",
+ __func__);
+ platform_device_unregister(device);
+ return -ENODEV;
+ }
+ avb_chip = card->private_data;
+ avb_chip->device = device;
+ avb_chip->link = link;
+
+ link->media_chip = avb_chip;
+
+ return 0;
+}
+
+static struct tsn_shim_ops shim_ops = {
+ .shim_name = "alsa",
+ .probe = snd_avb_new,
+ .buffer_refill = snd_avb_refill,
+ .buffer_drain = snd_avb_drain,
+ .media_close = snd_avb_close,
+ .hdr_size = snd_avb_hdr_size,
+ .copy_size = snd_avb_copy_size,
+ .assemble_header = snd_avb_assemble_iidc,
+ .validate_header = snd_avb_validate_iidc,
+ .get_payload_data = snd_avb_get_payload_data,
+};
+
+static int __init avb_alsa_init(void)
+{
+ if (tsn_shim_register_ops(&shim_ops)) {
+ pr_err("Could not register ALSA-shim with TSN\n");
+ return -EINVAL;
+ }
+ pr_info("AVB ALSA added OK\n");
+ return 0;
+}
+
+static void __exit avb_alsa_exit(void)
+{
+ tsn_shim_deregister_ops(&shim_ops);
+}
+
+module_init(avb_alsa_init);
+module_exit(avb_alsa_exit);
+MODULE_AUTHOR("Henrik Austad");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("TSN ALSA shim driver");
diff --git a/drivers/media/avb/tsn_iec61883.h b/drivers/media/avb/tsn_iec61883.h
new file mode 100644
index 0000000..bf26138
--- /dev/null
+++ b/drivers/media/avb/tsn_iec61883.h
@@ -0,0 +1,124 @@
+#ifndef TSN_IEC61883_H
+#define TSN_IEC61883_H
+#include <linux/tsn.h>
+
+/*
+ * psh:
+ * tag:2
+ * channel:6
+ * tcode:4
+ * sy:4
+ * See IEEE 1722.1 :: 6.2 for details
+ */
+struct iec61883_tag {
+ u8 tag:2;
+ u8 channel:6;
+ u8 tcode:4;
+ u8 sy:4;
+} __packed;
+
+struct iec61883_audio_header {
+ u8 sid:6;
+ u8 cip_1:2;
+
+ u8 dbs:8;
+
+ u8 rsv:2; /* reserved */
+ u8 sph:1;
+ u8 qpc:3;
+ u8 fn:2;
+
+ u8 dbc;
+
+ u8 fmt:6;
+ u8 cip_2:2;
+ u8 fdf;
+ u16 syt;
+ u8 payload[0];
+} __packed;
+
+static inline size_t _iec61883_hdr_len(void)
+{
+ return sizeof(struct iec61883_audio_header);
+}
+
+static inline int _iec61883_hdr_verify(struct avtpdu_header *hdr)
+{
+ struct iec61883_audio_header *dh;
+ struct iec61883_tag *psh;
+
+ if (hdr->subtype != AVTP_61883_IIDC)
+ return -EINVAL;
+ dh = (struct iec61883_audio_header *)&hdr->data;
+ psh = (struct iec61883_tag *)&hdr->psh;
+
+ /* Verify 61883 header */
+ if (psh->tag != 1 || psh->channel != 31 ||
+ psh->tcode != 0xA || psh->sy != 0)
+ return -EINVAL;
+
+ /* check flags that should be static from frame to frame */
+ if (dh->cip_1 != 0 || dh->sid != 0x3f || dh->qpc != 0 || dh->fn != 0 ||
+ dh->sph != 0 || dh->cip_2 != 2)
+ return -EINVAL;
+
+ if (dh->dbs != ntohs(hdr->sd_len)*2 || dh->dbc != hdr->seqnr)
+ return -EINVAL;
+
+ return 0;
+}
+
+static inline void _iec61883_hdr_assemble(struct avtpdu_header *hdr,
+ size_t bytes)
+{
+ struct iec61883_tag *psh;
+ struct iec61883_audio_header *dh;
+
+ if (bytes > 0x7f)
+ pr_warn("%s: hdr->dbs will overflow, malformed frame will be the result\n",
+ __func__);
+
+
+ hdr->subtype = AVTP_61883_IIDC;
+
+ /* IIDC 61883 header */
+ psh = (struct iec61883_tag *)&hdr->psh;
+ psh->tag = 1;
+ psh->channel = 31; /* 0x1f */
+ psh->tcode = 0xA;
+ psh->sy = 0;
+
+ dh = (struct iec61883_audio_header *)&hdr->data;
+ dh->cip_1 = 0;
+ dh->sid = 63; /* 0x3f */
+ dh->dbs = (u8)(bytes*2); /* number of quadlets of data in AVTPDU */
+ dh->qpc = 0;
+ dh->fn = 0;
+ dh->sph = 0;
+ dh->dbc = hdr->seqnr;
+ dh->cip_2 = 2;
+
+ /*
+ * FMT (Format ID): same as specified in iec 61883-1:2003
+ *
+ * For IEC 61883-6, it shall be 0x10 (16) to define Audio and
+ * Music data
+ */
+ dh->fmt = 0x10;
+
+ /* FIXME: find value
+ * Could be sampling-freq, but 8 bits give 0 - 65kHz sampling.
+ */
+ dh->fdf = 0;
+
+ dh->syt = 0xFFFF;
+}
+
+static inline void *_iec61883_payload(struct avtpdu_header *hdr)
+{
+ struct iec61883_audio_header *dh = (struct iec61883_audio_header *)&hdr->data;
+ /* TODO: add some basic checks before returning payload ? */
+ return &dh->payload;
+}
+
+#endif /* TSN_IEC61883_H */
--
2.7.4
Richard Cochran
2016-06-15 11:49:22 UTC
Permalink
Now that I understand better...
Post by Henrik Austad
Userspace is supposed to reserve bandwidth, find StreamID etc.
mkdir /config/tsn/test/eth0/talker
cd /config/tsn/test/eth0/talker
echo 65535 > buffer_size
echo 08:00:27:08:9f:c3 > remote_mac
echo 42 > stream_id
echo alsa > enabled
This is exactly why configfs is the wrong interface. If you implement
the AVB device in alsa-lib user space, then you can handle the
reservations, configuration, UDP sockets, etc, in a way transparent to
the aplay program.

Heck, if done properly, your layer could discover the AVB nodes in the
network and present each one as a separate device...

Thanks,
Richard
Henrik Austad
2016-06-15 12:13:19 UTC
Permalink
Post by Richard Cochran
Now that I understand better...
Post by Henrik Austad
Userspace is supposed to reserve bandwidth, find StreamID etc.
mkdir /config/tsn/test/eth0/talker
cd /config/tsn/test/eth0/talker
echo 65535 > buffer_size
echo 08:00:27:08:9f:c3 > remote_mac
echo 42 > stream_id
echo alsa > enabled
This is exactly why configfs is the wrong interface. If you implement
the AVB device in alsa-lib user space, then you can handle the
reservations, configuration, UDP sockets, etc, in a way transparent to
the aplay program.
And how would v4l2 benefit from this being in alsalib? Should we require
both V4L and ALSA to implement the same, or should we place it in a common
place for all.

And what about those systems that want to use TSN but is not a
media-device, they should be given a raw-socket to send traffic over,
should they also implement something in a library?

So no, here I think configfs is an apt choice.
Post by Richard Cochran
Heck, if done properly, your layer could discover the AVB nodes in the
network and present each one as a separate device...
No, you definately do not want the kernel to automagically add devices
whenever something pops up on the network, for this you need userspace to
be in control. 1722.1 should not be handled in-kernel.
--
Henrik Austad
Richard Cochran
2016-06-15 12:43:52 UTC
Permalink
Post by Henrik Austad
And how would v4l2 benefit from this being in alsalib? Should we require
both V4L and ALSA to implement the same, or should we place it in a common
place for all.
I don't require V4L to implement anything. You were the one wanting
AVB "devices". Go ahead and do that, but in user space please. We
don't want to have kernel code that sends arbitrary Layer2 and UDP
media packets.

The example you present of using aplay over the network is a cute
idea, but after all it is a trivial application. I have nothing
against creating virtual ALSA devices (if done properly), but that
model won't work for more realistic AVB networks.
Post by Henrik Austad
And what about those systems that want to use TSN but is not a
media-device, they should be given a raw-socket to send traffic over,
should they also implement something in a library?
A raw socket is the way to do it, yes.

Since TSN is about bandwidth reservation and time triggered Ethernet,
decoupled from the contents of the streams, each new TSN application
will have to see what works best. If common ground is found, then a
library makes sense to do.

At this point, it is way too early to guess how that will look. But
media packet formats clearly belong in user space.
Post by Henrik Austad
So no, here I think configfs is an apt choice.
Post by Richard Cochran
Heck, if done properly, your layer could discover the AVB nodes in the
network and present each one as a separate device...
No, you definately do not want the kernel to automagically add devices
whenever something pops up on the network, for this you need userspace to
be in control. 1722.1 should not be handled in-kernel.
The layer should be in user space. Alsa-lib *is* user space.

Thanks,
Richard
Henrik Austad
2016-06-11 23:02:13 UTC
Permalink
This adds support for loading the igb.ko module with tsn
capabilities. This requires a 2-step approach. First enabling TSN in
config, then load the module with use_tsn=1.

Once enabled and loaded, the controller will be placed in "Qav-mode"
which is when the credit-based shaper is available, 3 of the queues are
removed from regular traffic, max payload is set to 1522 octets (no
jumboframes allowed).

It dumps the registers of interest before and after, so this clutters
kern.log a bit. In time this will be reduced / tied to the debug-param
for the module.

Note: currently this driver is *not* stable, it is still a work in
progress.

Cc: Jeff Kirsher <***@intel.com>
Cc: intel-wired-***@lists.osuosl.org
Cc: "David S. Miller" <***@davemloft.net>
Signed-off-by: Henrik Austad <***@cisco.com>
---
drivers/net/ethernet/intel/Kconfig | 18 ++
drivers/net/ethernet/intel/igb/Makefile | 2 +-
drivers/net/ethernet/intel/igb/igb.h | 19 ++
drivers/net/ethernet/intel/igb/igb_main.c | 10 +-
drivers/net/ethernet/intel/igb/igb_tsn.c | 396 ++++++++++++++++++++++++++++++
5 files changed, 443 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/intel/igb/igb_tsn.c

diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 714bd10..8e620a9 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -99,6 +99,24 @@ config IGB
To compile this driver as a module, choose M here. The module
will be called igb.

+config IGB_TSN
+ tristate "TSN Support for Intel(R) 82575/82576 i210 Network Controller"
+ depends on IGB && TSN
+ ---help---
+ This driver supports TSN (AVB) on Intel I210 network controllers.
+
+ When enabled, it will allow the module to be loaded with
+ "use_tsn" which will initialize the controller to A/V-mode
+ instead of legacy-mode. This will take 3 of the tx-queues and
+ place them in 802.1Q QoS mode and enable the credit-based
+ shaper for 2 of the queues.
+
+ If built with this option, but not loaded with use_tsn, the
+ only difference is a slightly larger module, no extra
+ code paths are called.
+
+ If unsure, say No
+
config IGB_HWMON
bool "Intel(R) PCI-Express Gigabit adapters HWMON support"
default y
diff --git a/drivers/net/ethernet/intel/igb/Makefile b/drivers/net/ethernet/intel/igb/Makefile
index 5bcb2de..1a9b776 100644
--- a/drivers/net/ethernet/intel/igb/Makefile
+++ b/drivers/net/ethernet/intel/igb/Makefile
@@ -33,4 +33,4 @@ obj-$(CONFIG_IGB) += igb.o

igb-objs := igb_main.o igb_ethtool.o e1000_82575.o \
e1000_mac.o e1000_nvm.o e1000_phy.o e1000_mbx.o \
- e1000_i210.o igb_ptp.o igb_hwmon.o
+ e1000_i210.o igb_ptp.o igb_hwmon.o igb_tsn.o
diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index b9609af..708f705 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -356,6 +356,7 @@ struct hwmon_buff {
#define IGB_RETA_SIZE 128

/* board specific private data structure */
+
struct igb_adapter {
unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];

@@ -472,6 +473,13 @@ struct igb_adapter {
int copper_tries;
struct e1000_info ei;
u16 eee_advert;
+
+#if IS_ENABLED(CONFIG_IGB_TSN)
+ /* Reserved BW for class A and B */
+ u16 sra_idleslope_res;
+ u16 srb_idleslope_res;
+ u8 tsn_ready:1;
+#endif /* IGB_TSN */
};

#define IGB_FLAG_HAS_MSI BIT(0)
@@ -552,6 +560,17 @@ void igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, unsigned char *va,
struct sk_buff *skb);
int igb_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr);
int igb_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr);
+/* This should be the only place where we add ifdeffery
+ * to include tsn-stuff or not. Everything else is located in igb_tsn.c
+ */
+#if IS_ENABLED(CONFIG_IGB_TSN)
+void igb_tsn_init(struct igb_adapter *adapter);
+int igb_tsn_capable(struct net_device *netdev);
+int igb_tsn_link_configure(struct net_device *netdev, enum sr_class sr_class,
+ u16 framesize, u16 vid);
+#else
+static inline void igb_tsn_init(struct igb_adapter *adapter) { }
+#endif /* CONFIG_IGB_TSN */
void igb_set_flag_queue_pairs(struct igb_adapter *, const u32);
#ifdef CONFIG_IGB_HWMON
void igb_sysfs_exit(struct igb_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index ef3d642..4d8789f 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2142,6 +2142,10 @@ static const struct net_device_ops igb_netdev_ops = {
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = igb_netpoll,
#endif
+#if IS_ENABLED(CONFIG_IGB_TSN)
+ .ndo_tsn_capable = igb_tsn_capable,
+ .ndo_tsn_link_configure = igb_tsn_link_configure,
+#endif /* CONFIG_IGB_TSN */
.ndo_fix_features = igb_fix_features,
.ndo_set_features = igb_set_features,
.ndo_fdb_add = igb_ndo_fdb_add,
@@ -2665,6 +2669,8 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* do hw tstamp init after resetting */
igb_ptp_init(adapter);

+ igb_tsn_init(adapter);
+
dev_info(&pdev->dev, "Intel(R) Gigabit Ethernet Network Connection\n");
/* print bus type/speed/width info, not applicable to i354 */
if (hw->mac.type != e1000_i354) {
@@ -5323,8 +5329,10 @@ static netdev_tx_t igb_xmit_frame(struct sk_buff *skb,
/* The minimum packet size with TCTL.PSP set is 17 so pad the skb
* in order to meet this minimum size requirement.
*/
- if (skb_put_padto(skb, 17))
+ if (skb_put_padto(skb, 17)) {
+ pr_err("%s: skb_put_padto FAILED. skb->len < 17\n", __func__);
return NETDEV_TX_OK;
+ }

return igb_xmit_frame_ring(skb, igb_tx_queue_mapping(adapter, skb));
}
diff --git a/drivers/net/ethernet/intel/igb/igb_tsn.c b/drivers/net/ethernet/intel/igb/igb_tsn.c
new file mode 100644
index 0000000..641f4f2
--- /dev/null
+++ b/drivers/net/ethernet/intel/igb/igb_tsn.c
@@ -0,0 +1,396 @@
+/*
+ * Copyright(c) 2015-2016 Henrik Austad <***@cisco.com>
+ * Cisco Systems, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+/* FIXME: This should probably be handled by some Makefile-magic */
+
+#if IS_ENABLED(CONFIG_IGB_TSN)
+#include "igb.h"
+#include <linux/module.h>
+
+/* NOTE: keep the defines not present in e1000_regs.h to avoid
+ * cluttering too many files. Once we are pretty stable, these will move
+ * into it's proper home. Until then, make merge a bit easier by
+ * avoiding it
+ */
+
+/* Qav regs */
+#define E1000_IRPBS 0x02404 /* Rx Packet Buffer Size - RW */
+#define E1000_ITPBS 0x03404 /* Tx buffer size assignment */
+#define E1000_TQAVCTRL 0x03570 /* Tx Qav Control */
+#define E1000_DTXMXPKTSZ 0x0355C /* DMA TX Maximum Packet Size */
+
+/* Qav defines. */
+#define E1000_TQAVCH_ZERO_CREDIT 0x80000000
+#define E1000_LINK_RATE 0x7735
+
+/* queue mode, 0=strict, 1=SR mode */
+#define E1000_TQAVCC_QUEUEMODE 0x80000000
+/* Transmit mode, 0=legacy, 1=QAV */
+#define E1000_TQAVCTRL_TXMODE 0x00000001
+/* report DMA time of tx packets */
+#define E1000_TQAVCTRL_1588_STAT_EN 0x00000004
+/* data fetch arbitration */
+#define E1000_TQAVCTRL_DATA_FETCH_ARB 0x00000010
+/* data tx arbitration */
+#define E1000_TQAVCTRL_DATA_TRAN_ARB 0x00000100
+/* data launch time valid */
+#define E1000_TQAVCTRL_DATA_TRAN_TIM 0x00000200
+/* stall SP to guarantee SR */
+#define E1000_TQAVCTRL_SP_WAIT_SR 0x00000400
+
+/* ... and associated shift value */
+#define E1000_TQAVCTRL_FETCH_TM_SHIFT (16)
+
+/* QAV Tx mode control registers where _n can be 0 or 1. */
+#define E1000_TQAVCC(_idx) (0x03004 + 0x40 * (_idx))
+
+/* Tx Qav High Credit - See 7.2.7.6 for calculations
+ * intel 8.12.18
+ */
+#define E1000_TQAVHC(_idx) (0x0300C + 0x40 * (_idx))
+
+/* Queues priority masks where _n and _p can be 0-3. */
+
+#define MAX_FRAME_SIZE 1522
+#define MIN_FRAME_SIZE 64
+
+static int use_tsn = -1;
+static int debug_tsn = -1;
+module_param(use_tsn, int, 0);
+module_param(debug_tsn, int, 0);
+MODULE_PARM_DESC(use_tsn, "use_tsn (0=off, 1=enabled)");
+MODULE_PARM_DESC(debug_tsn, "debug_tsn (0=off, 1=enabled)");
+
+/* For a full list of the registers dumped here, see sec 8.1.3 in the
+ * i210 controller datasheet.
+ */
+static inline void _tsn_dump_regs(struct igb_adapter *adapter)
+{
+ u32 val = 0;
+ struct device *dev;
+ struct e1000_hw *hw = &adapter->hw;
+
+ /* do not dump regs if we're not debugging driver */
+ if (debug_tsn != 1)
+ return;
+
+ dev = &adapter->pdev->dev;
+ dev_info(dev, "num_tx_queues=%d, num_rx_queues=%d\n",
+ adapter->num_tx_queues, adapter->num_rx_queues);
+
+ /* 0x0008 - E1000_STATUS Device status register */
+ val = rd32(E1000_STATUS);
+ dev_info(&adapter->pdev->dev, "\n");
+ dev_info(dev, "Status: FullDuplex=%s, LinkUp=%s, speed=%0x01x\n",
+ val & 0x1 ? "FD" : "HD",
+ val & 0x2 ? "LU" : "LD",
+ val & 0xc0 >> 6);
+
+ /* E1000_VET vlan ether type */
+ val = rd32(E1000_VET);
+ dev_info(dev, "VLAN ether type: VET.VET=0x%04x, VET.VET_EXT=0x%04x\n",
+ val & 0xffff, (val >> 16) & 0xffff);
+
+ /* E1000_RXPBS (RXPBSIZE) Rx Packet Buffer Size */
+ val = rd32(E1000_RXPBS);
+ dev_info(dev, "Rx Packet buffer: RXPBSIZE=%dkB, Bmc2ospbsize=%dkB, cfg_ts_en=%s\n",
+ val & 0x1f,
+ (val >> 6) & 0x1f,
+ (val & (1 << 31)) ? "cfg_ts_en" : "cfg_ts_dis");
+
+ /* Transmit stuff */
+ /* E1000_TXPBS (TXPBSIZE) Tx Packet Buffer Size - RW */
+ val = rd32(E1000_TXPBS);
+ dev_info(dev, "Tx Packet buffer: Txpb0size=%dkB, Txpb1size=%dkB, Txpb2size=%dkB, Txpb3size=%dkB, os2Bmcpbsize=%dkB\n",
+ val & 0x3f, (val >> 6) & 0x3f, (val >> 12) & 0x3f,
+ (val >> 18) & 0x3f, (val >> 24) & 0x3f);
+
+ /* E1000_TCTL (TCTL) Tx control - RW*/
+ val = rd32(E1000_TCTL);
+ dev_info(dev, "Tx control reg: TxEnable=%s, CT=0x%X\n",
+ val & 2 ? "EN" : "DIS", (val >> 3) & 0x3F);
+
+ /* TQAVHC : Transmit Qav High credits 0x300C + 0x40*n - RW */
+ val = rd32(E1000_TQAVHC(0));
+ dev_info(dev, "E1000_TQAVHC0: %0x08x\n", val);
+ val = rd32(E1000_TQAVHC(1));
+ dev_info(dev, "E1000_TQAVHC1: %0x08x\n", val);
+
+ /* TQAVCC[0-1]: Transmit Qav 0x3004 + 0x40*n - RW */
+ val = rd32(E1000_TQAVCC(0));
+ dev_info(dev, "E1000_TQAVCC0: idleSlope=%02x, QueueMode=%s\n",
+ val % 0xff,
+ val > 31 ? "Stream reservation" : "Strict priority");
+ val = rd32(E1000_TQAVCC(1));
+ dev_info(dev, "E1000_TQAVCC1: idleSlope=%02x, QueueMode=%s\n",
+ val % 0xff,
+ val > 31 ? "Stream reservation" : "Strict priority");
+
+ /* TQAVCTRL : Transmit Qav control - RW */
+ val = rd32(E1000_TQAVCTRL);
+ dev_info(dev, "E1000_TQAVCTRL: TransmitMode=%s,1588_STAT_EN=%s,DataFetchARB=%s,DataTranARB=%s,DataTranTIM=%s,SP_WAIT_SR=%s,FetchTimDelta=%dns (0x%04x)\n",
+ (val & 0x0001) ? "Qav" : "Legacy",
+ (val & 0x0004) ? "En" : "Dis",
+ (val & 0x0010) ? "Most Empty" : "Round Robin",
+ (val & 0x0100) ? "Credit Shaper" : "Strict priority",
+ (val & 0x0200) ? "Valid" : "N/A",
+ (val & 0x0400) ? "Wait" : "nowait",
+ (val >> 16) * 32, (val >> 16));
+}
+
+/* Place the NIC in Qav-mode.
+ *
+ * This will result in a _single_ queue for normal BE traffic, the rest
+ * will be grabbed by the Qav-machinery and kept for strict priority
+ * transmission.
+ *
+ * I210 Datasheet Sec 7.2.7.7 gives a lot of information.
+ */
+void igb_tsn_init(struct igb_adapter *adapter)
+{
+ struct e1000_hw *hw = &adapter->hw;
+ u32 val;
+
+ if (use_tsn != 1) {
+ adapter->tsn_ready = 0;
+ dev_info(&adapter->pdev->dev, "%s got use_tsn > 0 (%d)\n",
+ __func__, use_tsn);
+ return;
+ }
+
+ if (debug_tsn < 0 || debug_tsn > 1)
+ debug_tsn = 0;
+
+ if (!adapter->pdev) {
+ adapter->tsn_ready = 0;
+ return;
+ }
+
+ switch (adapter->pdev->device) {
+ case 0x1533: /* E1000_DEV_ID_I210_COPPER */
+ case 0x1536: /* E1000_DEV_ID_I210_FIBER */
+ case 0x1537: /* E1000_DEV_ID_I210_SERDES: */
+ case 0x1538: /* E1000_DEV_ID_I210_SGMII: */
+ case 0x157b: /* E1000_DEV_ID_I210_COPPER_FLASHLESS: */
+ case 0x157c: /* E1000_DEV_ID_I210_SERDES_FLASHLESS: */
+ break;
+ default:
+ /* not a known IGB-TSN capable device */
+ adapter->tsn_ready = 0;
+ return;
+ }
+ _tsn_dump_regs(adapter);
+
+ /* Set Tx packet buffer size assignment, see 7.2.7.7 in i210
+ * PB0: 8kB
+ * PB1: 8kB
+ * PB2: 4kB
+ * PB3: 4kB
+ * os2bmcsize: 2kB
+ * sumTx: 26kB
+ *
+ * Rxpbsize: 0x20 (32kB)
+ * bmc2ossize: 0x02
+ * sumRx: 34kB
+ *
+ * See 8.3.1 && 8.3.2
+ */
+ val = (0x02 << 24 | 0x04 << 18 | 0x04 << 12 | 0x08 << 6 | 0x08);
+ wr32(E1000_ITPBS, val);
+ wr32(E1000_IRPBS, (0x02 << 6 | 0x20));
+
+ /* DMA Tx maximum packet size, the largest frame DMA should transport
+ * do not allow frames larger than 1522 + preample. Reg expects
+ * size in 64B increments. 802.1BA 6.3
+ * Round up to 1536 to handle 64B increments
+ *
+ * Initial value: 0x98 (152 => 9728 bytes)
+ */
+ wr32(E1000_DTXMXPKTSZ, 1536 >> 6);
+
+ /* Place card in Qav-mode, use tx-queue 0,1 for Qav
+ * (Credit-based shaper), 2,3 for standard priority (and
+ * best-effort) traffic.
+ *
+ * i210 8.12.19 and 8.12.21
+ *
+ * - Fetch: most empty and time based (not round-robin)
+ * - Transmit: Credit based shaper for SR queues
+ * - Data launch time valid (in Qav mode)
+ * - Wait for SR queues to ensure that launch time is always valid.
+ * - Set ~10us wait-time-delta, 32ns granularity
+ *
+ * Do *not* enable Tx for shaper (E1000_TQAVCTRL_DATA_TRAN_ARB)
+ * yet as we do not have data to Tx
+ */
+ val = E1000_TQAVCTRL_TXMODE |
+ E1000_TQAVCTRL_DATA_FETCH_ARB |
+ E1000_TQAVCTRL_DATA_TRAN_TIM |
+ E1000_TQAVCTRL_SP_WAIT_SR |
+ 320 << E1000_TQAVCTRL_FETCH_TM_SHIFT;
+
+ wr32(E1000_TQAVCTRL, val);
+
+ /* For now, only set CreditBased shaper for A and B, not set
+ * idleSlope as we have not yet gotten any streams.
+ * 8.12.19
+ */
+ wr32(E1000_TQAVCC(0), E1000_TQAVCC_QUEUEMODE);
+ wr32(E1000_TQAVCC(1), E1000_TQAVCC_QUEUEMODE);
+
+ wr32(E1000_TQAVHC(0), E1000_TQAVCH_ZERO_CREDIT);
+ wr32(E1000_TQAVHC(1), E1000_TQAVCH_ZERO_CREDIT);
+
+ /* reset Tx Descriptor tail and head for the queues */
+ wr32(E1000_TDT(0), 0);
+ wr32(E1000_TDT(1), 0);
+ wr32(E1000_TDH(0), 0);
+ wr32(E1000_TDH(1), 0);
+
+ _tsn_dump_regs(adapter);
+ dev_info(&adapter->pdev->dev, "\n");
+
+ adapter->sra_idleslope_res = 0;
+ adapter->srb_idleslope_res = 0;
+ adapter->tsn_ready = 1;
+
+ dev_info(&adapter->pdev->dev, "%s: setup done\n", __func__);
+}
+
+int igb_tsn_capable(struct net_device *netdev)
+{
+ struct igb_adapter *adapter;
+
+ if (!netdev)
+ return -EINVAL;
+ adapter = netdev_priv(netdev);
+ if (use_tsn == 1)
+ return adapter->tsn_ready == 1;
+ return 0;
+}
+
+/* igb_tsn_link_configure - configure NIC to handle a new stream
+ *
+ * @netdev: pointer to NIC device
+ * @class: the class for the stream used to find the correct queue.
+ * @framesize: size of each frame, *including* headers (not preamble)
+ * @vid: VLAN ID
+ *
+ * NOTE: the sr_class only instructs the driver which queue to use, not
+ * what priority the network expects for a given class. This is
+ * something userspace must find out and then let the tsn-driver set in
+ * the frame before xmit.
+ *
+ * FIXME: remove bw-req from a stream that goes away.
+ */
+int igb_tsn_link_configure(struct net_device *netdev, enum sr_class class,
+ u16 framesize, u16 vid)
+{
+ /* FIXME: push into adapter-storage */
+ static int class_a_size;
+ static int class_b_size;
+ int err;
+ u32 idle_slope_a = 0;
+ u32 idle_slope_b = 0;
+ u32 new_is = 0;
+ u32 hicred_a = 0;
+ u32 hicred_b = 0;
+ u32 tqavctrl;
+
+ struct igb_adapter *adapter;
+ struct e1000_hw *hw;
+
+ if (!netdev)
+ return -EINVAL;
+ adapter = netdev_priv(netdev);
+ hw = &adapter->hw;
+
+ if (!igb_tsn_capable(netdev)) {
+ pr_err("%s: NIC not capable\n", __func__);
+ return -EINVAL;
+ }
+
+ if (framesize > MAX_FRAME_SIZE || framesize < MIN_FRAME_SIZE) {
+ pr_err("%s: framesize (%u) must be [%d,%d]\n", __func__,
+ framesize, MIN_FRAME_SIZE, MAX_FRAME_SIZE);
+ return -EINVAL;
+ }
+
+ /* TODO: is this the correct place/way? Is it required? */
+ rtnl_lock();
+ pr_info("%s: adding VLAN %u to HW filter on device %s\n",
+ __func__, vid, netdev->name);
+ err = vlan_vid_add(netdev, htons(ETH_P_8021Q), vid);
+ if (err != 0)
+ pr_err("%s: error adding vlan %u, res=%d\n",
+ __func__, vid, err);
+ rtnl_unlock();
+
+ /* Grab current values of idle_slope */
+ idle_slope_a = rd32(E1000_TQAVHC(0)) & ~E1000_TQAVCH_ZERO_CREDIT;
+ idle_slope_b = rd32(E1000_TQAVHC(1)) & ~E1000_TQAVCH_ZERO_CREDIT;
+
+ /* Calculate new idle slope and add to appropriate idle_slope
+ * idle_slope = BW * linkrate * 2 (0r 0.2 for 100Mbit)
+ * BW: % of total bandwidth
+ */
+ new_is = framesize * E1000_LINK_RATE * 16 / 1000000;
+
+ switch (class) {
+ case SR_CLASS_A:
+ new_is *= 2; /* A is 8kHz, B is 4kHz */
+ idle_slope_a += new_is;
+ class_a_size = framesize;
+ break;
+ case SR_CLASS_B:
+ idle_slope_b += new_is;
+ class_b_size = framesize;
+ break;
+ default:
+ pr_err("%s: unhandled SR-class (%d)\n", __func__, class);
+ return -EINVAL;
+ }
+
+ /* HiCred: cred obtained while waiting for current frame &&
+ * higher-class frames to finish xmit.
+ *
+ * Covered in detail in 7.2.7.6 in i210 datasheet
+ * For class A: only worst-case framesize that just started;
+ * i.e. 1522 * idleSlope / linkrate;
+ * For class B: (worst-case framesize + burstSize(A))*idleSlope
+ *
+ * See 802.1Q Annex L, eq L.10 for hicred_a and L.41 for
+ * hicred_b
+ */
+ if (class == SR_CLASS_A) {
+ hicred_a = E1000_TQAVCH_ZERO_CREDIT + idle_slope_a * MAX_FRAME_SIZE / E1000_LINK_RATE;
+ wr32(E1000_TQAVCC(0), E1000_TQAVCC_QUEUEMODE | idle_slope_a);
+ wr32(E1000_TQAVHC(0), hicred_a);
+ } else {
+ hicred_b = E1000_TQAVCH_ZERO_CREDIT | idle_slope_b * (MAX_FRAME_SIZE + class_a_size) / (E1000_LINK_RATE - idle_slope_a);
+ wr32(E1000_TQAVCC(1), E1000_TQAVCC_QUEUEMODE | idle_slope_b);
+ wr32(E1000_TQAVHC(1), hicred_b);
+ }
+
+ /* Enable Tx for shaper now that we have data */
+ tqavctrl = rd32(E1000_TQAVCTRL);
+ if (!(tqavctrl & E1000_TQAVCTRL_DATA_TRAN_ARB)) {
+ tqavctrl |= E1000_TQAVCTRL_DATA_TRAN_ARB;
+ wr32(E1000_TQAVCTRL, tqavctrl);
+ }
+ _tsn_dump_regs(netdev_priv(netdev));
+ return 0;
+}
+
+#endif /* #if IS_ENABLED(CONFIG_IGB_TSN) */
--
2.7.4
Henrik Austad
2016-06-11 23:02:27 UTC
Permalink
From: Henrik Austad <***@cisco.com>

Not sure how relevant this is other than making a point about
maintaining it.

Signed-off-by: Henrik Austad <***@cisco.com>
---
MAINTAINERS | 14 ++++++++++++++
1 file changed, 14 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index ed42cb6..ef5d926 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11634,6 +11634,20 @@ T: git git://linuxtv.org/anttip/media_tree.git
S: Maintained
F: drivers/media/tuners/tua9001*

+TSN CORE DRIVER
+M: Henrik Austad <***@cisco.com>
+L: linux-***@vger.kernel.org
+S: Supported
+F: drivers/net/tsn/
+F: include/linux/tsn.h
+F: include/trace/events/tsn.h
+
+TSN_AVB_DRIVER
+M: Henrik Austad <***@cisco.com>
+L: alsa-***@alsa-project.org (moderated for non-subscribers)
+S: Supported
+F: drivers/media/avb/
+
TULIP NETWORK DRIVERS
L: ***@vger.kernel.org
L: linux-***@vger.kernel.org
--
2.7.4
Henrik Austad
2016-06-11 23:02:39 UTC
Permalink
TSN provides a mechanism to create reliable, jitter-free, low latency
guaranteed bandwidth links over a local network. It does this by
reserving a path through the network. Support for TSN must be found in
both the NIC as well as in the network itself.

This adds required hooks into netdev_ops so that the core TSN driver can
use this when configuring a new NIC or setting up a new link.

Cc: "David S. Miller" <***@davemloft.net>
Signed-off-by: Henrik Austad <***@austad.us>
---
include/linux/netdevice.h | 32 ++++++++++++++++++++++++++++++++
net/Kconfig | 1 +
net/tsn/Kconfig | 32 ++++++++++++++++++++++++++++++++
3 files changed, 65 insertions(+)
create mode 100644 net/tsn/Kconfig

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index f45929c..de025eb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -109,6 +109,13 @@ enum netdev_tx {
};
typedef enum netdev_tx netdev_tx_t;

+#if IS_ENABLED(CONFIG_TSN)
+enum sr_class {
+ SR_CLASS_A = 1,
+ SR_CLASS_B = 2,
+};
+#endif
+
/*
* Current order: NETDEV_TX_MASK > NET_XMIT_MASK >= 0 is significant;
* hard_start_xmit() return < NET_XMIT_MASK means skb was consumed.
@@ -902,6 +909,22 @@ struct tc_to_netdev {
*
* void (*ndo_poll_controller)(struct net_device *dev);
*
+ * TSN functions (if CONFIG_TSN)
+ *
+ * int (*ndo_tsn_capable)(struct net_device *dev);
+ * If a particular device is capable of sustaining TSN traffic
+ * provided current configuration
+ * int (*ndo_tsn_link_configure)(struct net_device *dev,
+ * enum sr_class class,
+ * u16 framesize,
+ * u16 vid);
+ * - When a new TSN link is either added or removed, this is called to
+ * update the bandwidth for the particular stream-class
+ * - The framesize is the size of the _entire_ frame, not just the
+ * payload since the full size is required to allocate bandwidth through
+ * the credit based shaper in the NIC
+ * - the vlan_id is the configured vlan for TSN in this session.
+ *
* SR-IOV management functions.
* int (*ndo_set_vf_mac)(struct net_device *dev, int vf, u8* mac);
* int (*ndo_set_vf_vlan)(struct net_device *dev, int vf, u16 vlan, u8 qos);
@@ -1148,6 +1171,15 @@ struct net_device_ops {
#ifdef CONFIG_NET_RX_BUSY_POLL
int (*ndo_busy_poll)(struct napi_struct *dev);
#endif
+
+#if IS_ENABLED(CONFIG_TSN)
+ int (*ndo_tsn_capable)(struct net_device *dev);
+ int (*ndo_tsn_link_configure)(struct net_device *dev,
+ enum sr_class class,
+ u16 framesize,
+ u16 vid);
+#endif /* CONFIG_TSN */
+
int (*ndo_set_vf_mac)(struct net_device *dev,
int queue, u8 *mac);
int (*ndo_set_vf_vlan)(struct net_device *dev,
diff --git a/net/Kconfig b/net/Kconfig
index ff40562..fa9f691 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -215,6 +215,7 @@ source "net/802/Kconfig"
source "net/bridge/Kconfig"
source "net/dsa/Kconfig"
source "net/8021q/Kconfig"
+source "net/tsn/Kconfig"
source "net/decnet/Kconfig"
source "net/llc/Kconfig"
source "net/ipx/Kconfig"
diff --git a/net/tsn/Kconfig b/net/tsn/Kconfig
new file mode 100644
index 0000000..1fc3c1d
--- /dev/null
+++ b/net/tsn/Kconfig
@@ -0,0 +1,32 @@
+#
+# Configuration for 802.1 Time Sensitive Networking (TSN)
+#
+
+config TSN
+ tristate "802.1 TSN Support"
+ depends on VLAN_8021Q && PTP_1588_CLOCK && CONFIGFS_FS
+ ---help---
+ Select this if you want to enable TSN on capable interfaces.
+
+ TSN allows you to set up deterministic links on your LAN (only
+ L2 is currently supported). Once loaded, the driver will probe
+ all available interfaces if they are capable of supporting TSN
+ links.
+
+ Once loaded, a directory in configfs called tsn/ will expose
+ the capable NICs and allow userspace to create
+ links. Userspace must provide us with a StreamID as well as
+ reserving bandwidth through the network and once this is done,
+ a new link can be created by issuing a mkdir() in configfs and
+ updating the attributes for the new link.
+
+ TSN itself does not produce nor consume data, it is dependent
+ upon 'shims' doing this, which can be virtually anything. ALSA
+ is a good candidate.
+
+ For more information, refer to the TSN-documentation in the
+ kernel documentation repository.
+
+ The resulting module will be called 'tsn'
+
+ If unsure, say N.
--
2.7.4
Henrik Austad
2016-06-11 23:03:14 UTC
Permalink
From: Henrik Austad <***@cisco.com>

In short summary:

* tsn_core.c is the main driver of tsn, all new links go through
here and all data to/form the shims are handled here
core also manages the shim-interface.

* tsn_configfs.c is the API to userspace. TSN is driven from userspace
and a link is created, configured, enabled, disabled and removed
purely from userspace. All attributes requried must be determined by
userspace, preferrably via IEEE 1722.1 (discovery and enumeration).

* tsn_header.c small part that handles the actual header of the frames
we send. Kept out of core for cleanliness.

* tsn_net.c handles operations towards the networking layer.

The current driver is under development. This means that from the moment it
is enabled with a shim, it will send traffic, either 0-traffic (frames of
reserved length but with payload 0) or actual traffic. This will change
once the driver stabilizes.

For more detail, see Documentation/networking/tsn/

Cc: "David S. Miller" <***@davemloft.net>
Signed-off-by: Henrik Austad <***@cisco.com>
---
net/Makefile | 1 +
net/tsn/Makefile | 6 +
net/tsn/tsn_configfs.c | 623 +++++++++++++++++++++++++++++++
net/tsn/tsn_core.c | 975 +++++++++++++++++++++++++++++++++++++++++++++++++
net/tsn/tsn_header.c | 203 ++++++++++
net/tsn/tsn_internal.h | 383 +++++++++++++++++++
net/tsn/tsn_net.c | 403 ++++++++++++++++++++
7 files changed, 2594 insertions(+)
create mode 100644 net/tsn/Makefile
create mode 100644 net/tsn/tsn_configfs.c
create mode 100644 net/tsn/tsn_core.c
create mode 100644 net/tsn/tsn_header.c
create mode 100644 net/tsn/tsn_internal.h
create mode 100644 net/tsn/tsn_net.c

diff --git a/net/Makefile b/net/Makefile
index bdd1455..c15482e 100644
--- a/net/Makefile
+++ b/net/Makefile
@@ -79,3 +79,4 @@ ifneq ($(CONFIG_NET_L3_MASTER_DEV),)
obj-y += l3mdev/
endif
obj-$(CONFIG_QRTR) += qrtr/
+obj-$(CONFIG_TSN) += tsn/
diff --git a/net/tsn/Makefile b/net/tsn/Makefile
new file mode 100644
index 0000000..0d87687
--- /dev/null
+++ b/net/tsn/Makefile
@@ -0,0 +1,6 @@
+#
+# Makefile for the Linux TSN subsystem
+#
+
+obj-$(CONFIG_TSN) += tsn.o
+tsn-objs :=tsn_core.o tsn_configfs.o tsn_net.o tsn_header.o
diff --git a/net/tsn/tsn_configfs.c b/net/tsn/tsn_configfs.c
new file mode 100644
index 0000000..f3d0986
--- /dev/null
+++ b/net/tsn/tsn_configfs.c
@@ -0,0 +1,623 @@
+/*
+ * ConfigFS interface to TSN
+ * Copyright (C) 2015- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/configfs.h>
+#include <linux/netdevice.h>
+#include <linux/rtmutex.h>
+#include <linux/tsn.h>
+#include "tsn_internal.h"
+
+static inline struct tsn_link *to_tsn_link(struct config_item *item)
+{
+ /* this line causes checkpatch to WARN. making checkpatch happy,
+ * makes code messy..
+ */
+ return item ? container_of(to_config_group(item), struct tsn_link, group) : NULL;
+}
+
+static inline struct tsn_nic *to_tsn_nic(struct config_group *group)
+{
+ return group ? container_of(group, struct tsn_nic, group) : NULL;
+}
+
+/* -----------------------------------------------
+ * Tier2 attributes
+ *
+ * The content of the links userspace can see/modify
+ * -----------------------------------------------
+*/
+static ssize_t _tsn_max_payload_size_show(struct config_item *item,
+ char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%u\n", (u32)link->max_payload_size);
+}
+
+static ssize_t _tsn_max_payload_size_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ u16 mpl_size = 0;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change Payload size on on enabled link\n");
+ return -EINVAL;
+ }
+ ret = kstrtou16(page, 0, &mpl_size);
+ if (ret)
+ return ret;
+
+ /* 802.1BA-2011 6.4 payload must be <1500 octets (excluding
+ * headers, tags etc) However, this is not directly mappable to
+ * how some hw handles things, so to be conservative, we
+ * restrict it down to [26..1485]
+ *
+ * This is also the _payload_ size, which does not include the
+ * AVTPDU header. This is an upper limit to how much raw data
+ * the shim can transport in each frame.
+ */
+ if (!tsnh_payload_size_valid(mpl_size, link->shim_header_size)) {
+ pr_err("%s: payload (%u) should be [26..1480] octets.\n",
+ __func__, (u32)mpl_size);
+ return -EINVAL;
+ }
+ link->max_payload_size = mpl_size;
+ return count;
+}
+
+static ssize_t _tsn_shim_header_size_show(struct config_item *item,
+ char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%u\n", (u32)link->shim_header_size);
+}
+
+static ssize_t _tsn_shim_header_size_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ u16 hdr_size = 0;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change shim-header size on on enabled link\n");
+ return -EINVAL;
+ }
+
+ ret = kstrtou16(page, 0, &hdr_size);
+ if (ret)
+ return ret;
+
+ if (!tsnh_payload_size_valid(link->max_payload_size, hdr_size))
+ return -EINVAL;
+
+ link->shim_header_size = hdr_size;
+ return count;
+}
+
+static ssize_t _tsn_stream_id_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%llu\n", link->stream_id);
+}
+
+static ssize_t _tsn_stream_id_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ u64 sid;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change StreamID on on enabled link\n");
+ return -EINVAL;
+ }
+ ret = kstrtou64(page, 0, &sid);
+ if (ret)
+ return ret;
+
+ if (sid == link->stream_id)
+ return count;
+
+ if (tsn_find_by_stream_id(sid)) {
+ pr_warn("Cannot set sid to %llu - exists\n", sid);
+ return -EEXIST;
+ }
+ if (sid != link->stream_id)
+ tsn_readd_link(link, sid);
+ return count;
+}
+
+static ssize_t _tsn_buffer_size_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%zu\n", link->buffer_size);
+}
+
+static ssize_t _tsn_buffer_size_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ u32 tmp;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change Buffer Size on on enabled link\n");
+ return -EINVAL;
+ }
+
+ ret = kstrtou32(page, 0, &tmp);
+ /* only allow buffers !0 and smaller than 8MB for now */
+ if (!ret && tmp) {
+ pr_info("%s: update buffer_size from %zu to %u\n",
+ __func__, link->buffer_size, tmp);
+ link->buffer_size = (size_t)tmp;
+ return count;
+ }
+ return -EINVAL;
+}
+
+static ssize_t _tsn_class_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%s\n", (link->class_a ? "A" : "B"));
+}
+
+static ssize_t _tsn_class_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ char class[2] = { 0 };
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change Class-type on on enabled link\n");
+ return -EINVAL;
+ }
+ if (strncpy(class, page, 1)) {
+ if (strcmp(class, "a") == 0 || strcmp(class, "A") == 0)
+ link->class_a = 1;
+ else if (strcmp(class, "b") == 0 || strcmp(class, "B") == 0)
+ link->class_a = 0;
+ return count;
+ }
+
+ pr_err("%s: Could not copy new class into buffer\n", __func__);
+ return -EINVAL;
+}
+
+static ssize_t _tsn_vlan_id_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%u\n", link->vlan_id);
+}
+
+static ssize_t _tsn_vlan_id_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ u16 vlan_id;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change VLAN-ID on on enabled link\n");
+ return -EINVAL;
+ }
+ ret = kstrtou16(page, 0, &vlan_id);
+ if (ret)
+ return ret;
+ if (vlan_id > 0xfff)
+ return -EINVAL;
+ link->vlan_id = vlan_id & 0xfff;
+ return count;
+}
+
+static ssize_t _tsn_pcp_a_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "0x%x\n", link->pcp_a);
+}
+
+static ssize_t _tsn_pcp_a_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ int ret = 0;
+ u8 pcp;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change PCP-A on enabled link.\n");
+ return -EINVAL;
+ }
+ ret = kstrtou8(page, 0, &pcp);
+ if (ret)
+ return ret;
+ if (pcp > 0x7)
+ return -EINVAL;
+ link->pcp_a = pcp & 0x7;
+ return count;
+}
+
+static ssize_t _tsn_pcp_b_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "0x%x\n", link->pcp_b);
+}
+
+static ssize_t _tsn_pcp_b_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ int ret = 0;
+ u8 pcp;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change PCP-B on enabled link.\n");
+ return -EINVAL;
+ }
+ ret = kstrtou8(page, 0, &pcp);
+ if (ret)
+ return ret;
+ if (pcp > 0x7)
+ return -EINVAL;
+ link->pcp_b = pcp & 0x7;
+ return count;
+}
+
+static ssize_t _tsn_end_station_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%s\n",
+ (link->estype_talker ? "Talker" : "Listener"));
+}
+
+static ssize_t _tsn_end_station_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ char estype[9] = {0};
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change End-station type on enabled link.\n");
+ return -EINVAL;
+ }
+ if (strncpy(estype, page, 8)) {
+ if (strncmp(estype, "Talker", 6) == 0 ||
+ strncmp(estype, "talker", 6) == 0) {
+ link->estype_talker = 1;
+ return count;
+ } else if (strncmp(estype, "Listener", 8) == 0 ||
+ strncmp(estype, "listener", 8) == 0) {
+ link->estype_talker = 0;
+ return count;
+ }
+ }
+ return -EINVAL;
+}
+
+static ssize_t _tsn_enabled_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%s\n", tsn_shim_get_active(link));
+}
+
+static ssize_t _tsn_enabled_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ char driver_type[SHIM_NAME_SIZE] = { 0 };
+ struct tsn_shim_ops *shim_ops;
+ size_t len;
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+
+ strncpy(driver_type, page, SHIM_NAME_SIZE - 1);
+ len = strlen(driver_type);
+ while (len-- > 0) {
+ if (driver_type[len] == '\n')
+ driver_type[len] = 0x00;
+ }
+ if (tsn_link_is_on(link)) {
+ if (strncmp(driver_type, "off", 3) == 0) {
+ tsn_teardown_link(link);
+ } else {
+ pr_err("Unknown value (%s), ignoring\n", driver_type);
+ return -EINVAL;
+ }
+ } else {
+ shim_ops = tsn_shim_find_by_name(driver_type);
+
+ if (!shim_ops) {
+ pr_info("%s: could not enable desired shim, %s is not available\n",
+ __func__, driver_type);
+ return -EINVAL;
+ }
+
+ ret = tsn_prepare_link(link, shim_ops);
+ if (ret != 0) {
+ pr_err("%s: Trouble perparing link, somethign went wrong - %d\n",
+ __func__, ret);
+ return ret;
+ }
+ }
+ return count;
+}
+
+static ssize_t _tsn_remote_mac_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%pM\n", link->remote_mac);
+}
+
+static ssize_t _tsn_remote_mac_store(struct config_item *item,
+ const char *page, size_t count)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ unsigned char mac[6] = {0};
+ int ret = 0;
+
+ if (!link)
+ return -EINVAL;
+ if (tsn_link_is_on(link)) {
+ pr_err("ERROR: Cannot change Remote MAC on enabled link.\n");
+ return -EINVAL;
+ }
+ ret = sscanf(page, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx",
+ &mac[0], &mac[1], &mac[2], &mac[3], &mac[4], &mac[5]);
+ if (ret > 0) {
+ pr_info("Got MAC, copying to storage\n");
+ memcpy(link->remote_mac, mac, 6);
+ return count;
+ }
+ return -EINVAL;
+}
+
+static ssize_t _tsn_local_mac_show(struct config_item *item, char *page)
+{
+ struct tsn_link *link = to_tsn_link(item);
+
+ if (!link)
+ return -EINVAL;
+ return sprintf(page, "%pMq\n", link->nic->dev->perm_addr);
+}
+
+CONFIGFS_ATTR(_tsn_, max_payload_size);
+CONFIGFS_ATTR(_tsn_, shim_header_size);
+CONFIGFS_ATTR(_tsn_, stream_id);
+CONFIGFS_ATTR(_tsn_, buffer_size);
+CONFIGFS_ATTR(_tsn_, class);
+CONFIGFS_ATTR(_tsn_, vlan_id);
+CONFIGFS_ATTR(_tsn_, pcp_a);
+CONFIGFS_ATTR(_tsn_, pcp_b);
+CONFIGFS_ATTR(_tsn_, end_station);
+CONFIGFS_ATTR(_tsn_, enabled);
+CONFIGFS_ATTR(_tsn_, remote_mac);
+CONFIGFS_ATTR_RO(_tsn_, local_mac);
+static struct configfs_attribute *tsn_tier2_attrs[] = {
+ &_tsn_attr_max_payload_size,
+ &_tsn_attr_shim_header_size,
+ &_tsn_attr_stream_id,
+ &_tsn_attr_buffer_size,
+ &_tsn_attr_class,
+ &_tsn_attr_vlan_id,
+ &_tsn_attr_pcp_a,
+ &_tsn_attr_pcp_b,
+ &_tsn_attr_end_station,
+ &_tsn_attr_enabled,
+ &_tsn_attr_remote_mac,
+ &_tsn_attr_local_mac,
+ NULL,
+};
+
+static struct config_item_type group_tsn_tier2_type = {
+ .ct_owner = THIS_MODULE,
+ .ct_attrs = tsn_tier2_attrs,
+ .ct_group_ops = NULL,
+};
+
+/* -----------------------------------------------
+ * Tier1
+ *
+ * The only interesting info at this level are the available links
+ * belonging to this nic. This will be the subdirectories. Apart from
+ * making/removing tier-2 folders, nothing else is required here.
+ */
+static struct config_group *group_tsn_1_make_group(struct config_group *group,
+ const char *name)
+{
+ struct tsn_nic *nic = to_tsn_nic(group);
+ struct tsn_link *link = tsn_create_and_add_link(nic);
+
+ if (!nic || !link)
+ return ERR_PTR(-ENOMEM);
+
+ config_group_init_type_name(&link->group, name, &group_tsn_tier2_type);
+
+ return &link->group;
+}
+
+static void group_tsn_1_drop_group(struct config_group *group,
+ struct config_item *item)
+{
+ struct tsn_link *link = to_tsn_link(item);
+ struct tsn_nic *nic = to_tsn_nic(group);
+
+ if (link) {
+ tsn_teardown_link(link);
+ tsn_remove_link(link);
+ }
+ pr_info("Dropping %s from NIC: %s\n", item->ci_name, nic->name);
+}
+
+static struct configfs_attribute *tsn_tier1_attrs[] = {
+ NULL,
+};
+
+static struct configfs_group_operations group_tsn_1_group_ops = {
+ .make_group = group_tsn_1_make_group,
+ .drop_item = group_tsn_1_drop_group,
+};
+
+static struct config_item_type group_tsn_tier1_type = {
+ .ct_group_ops = &group_tsn_1_group_ops,
+ .ct_attrs = tsn_tier1_attrs,
+ .ct_owner = THIS_MODULE,
+};
+
+/* -----------------------------------------------
+ * Tier0
+ *
+ * Top level. This will expose all the TSN-capable NICs as well as
+ * currently active StreamIDs and registered shims. 'Global' info goes
+ * here.
+ */
+static ssize_t _tsn_used_sids_show(struct config_item *item, char *page)
+{
+ return tsn_get_stream_ids(page, PAGE_SIZE);
+}
+
+static ssize_t _tsn_available_shims_show(struct config_item *item, char *page)
+{
+ return tsn_shim_export_probe_triggers(page);
+}
+
+static struct configfs_attribute tsn_used_sids = {
+ .ca_owner = THIS_MODULE,
+ .ca_name = "stream_ids",
+ .ca_mode = S_IRUGO,
+ .show = _tsn_used_sids_show,
+};
+
+static struct configfs_attribute available_shims = {
+ .ca_owner = THIS_MODULE,
+ .ca_name = "available_shims",
+ .ca_mode = S_IRUGO,
+ .show = _tsn_available_shims_show,
+};
+
+static struct configfs_attribute *group_tsn_attrs[] = {
+ &tsn_used_sids,
+ &available_shims,
+ NULL,
+};
+
+static struct config_item_type group_tsn_tier0_type = {
+ .ct_group_ops = NULL,
+ .ct_attrs = group_tsn_attrs,
+ .ct_owner = THIS_MODULE,
+};
+
+int tsn_configfs_init(struct tsn_list *tlist)
+{
+ int ret = 0;
+ struct tsn_nic *next;
+ struct configfs_subsystem *subsys;
+
+ if (!tlist || !tlist->num_avail)
+ return -EINVAL;
+
+ /* Tier-0 */
+ subsys = &tlist->tsn_subsys;
+ strncpy(subsys->su_group.cg_item.ci_namebuf, "tsn",
+ CONFIGFS_ITEM_NAME_LEN);
+ subsys->su_group.cg_item.ci_type = &group_tsn_tier0_type;
+
+ config_group_init(&subsys->su_group);
+ mutex_init(&subsys->su_mutex);
+
+ /* Tier-1
+ * (tsn-capable NICs), automatic subgroups
+ */
+ list_for_each_entry(next, &tlist->head, list) {
+ config_group_init_type_name(&next->group, next->name,
+ &group_tsn_tier1_type);
+ configfs_add_default_group(&next->group, &subsys->su_group);
+ }
+
+ /* This is the final step, once done, system is live, make sure
+ * init has completed properly
+ */
+ ret = configfs_register_subsystem(subsys);
+ if (ret) {
+ pr_err("Trouble registering TSN ConfigFS subsystem\n");
+ return ret;
+ }
+
+ pr_warn("configfs_init_module() OK\n");
+ return 0;
+}
+
+void tsn_configfs_exit(struct tsn_list *tlist)
+{
+ if (!tlist)
+ return;
+ configfs_unregister_subsystem(&tlist->tsn_subsys);
+ pr_warn("configfs_exit_module()\n");
+}
diff --git a/net/tsn/tsn_core.c b/net/tsn/tsn_core.c
new file mode 100644
index 0000000..51f1d13
--- /dev/null
+++ b/net/tsn/tsn_core.c
@@ -0,0 +1,975 @@
+/*
+ * TSN Core main part of TSN driver
+ *
+ * Copyright (C) 2015- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include <linux/rtmutex.h>
+#include <linux/hashtable.h>
+#include <linux/netdevice.h>
+#include <linux/net.h>
+#include <linux/dma-mapping.h>
+#include <net/sock.h>
+#include <net/net_namespace.h>
+#include <linux/hrtimer.h>
+#include <linux/configfs.h>
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/tsn.h>
+#include "tsn_internal.h"
+
+static struct tsn_list tlist;
+static int in_debug;
+static int on_cpu = -1;
+
+#define TLINK_HASH_BITS 8
+DEFINE_HASHTABLE(tlinks, TLINK_HASH_BITS);
+
+static LIST_HEAD(tsn_shim_ops);
+
+/* Called with link->lock held */
+static inline size_t _get_low_water(struct tsn_link *link)
+{
+ /* use max_payload_size and give a rough estimate of how many
+ * bytes that would be for low_water_ms
+ */
+ int low_water_ms = 20;
+ int numframes = low_water_ms * 8;
+
+ if (link->class_a)
+ numframes *= 2;
+ return link->max_payload_size * numframes;
+}
+
+/* Called with link->lock held */
+static inline size_t _get_high_water(struct tsn_link *link)
+{
+ size_t low_water = _get_low_water(link);
+
+ return max(link->used_buffer_size - low_water, low_water);
+}
+
+/**
+ * _tsn_set_buffer - register a memory region to use as the buffer
+ *
+ * This is used when we are operating in !external_buffer mode.
+ *
+ * TSN expects a ring-buffer and will update pointers to keep track of
+ * where we are. When the buffer is refilled, head and tail will be
+ * updated accordingly.
+ *
+ * @param link the link that should hold the buffer
+ * @param buffer the new buffer
+ * @param bufsize size of new buffer.
+ *
+ * @returns 0 on success, negative on error
+ *
+ * Must be called with tsn_lock() held.
+ */
+static int _tsn_set_buffer(struct tsn_link *link, void *buffer, size_t bufsize)
+{
+ if (link->buffer) {
+ pr_err("%s: Cannot add buffer, buffer already registred\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ trace_tsn_set_buffer(link, bufsize);
+ link->buffer = buffer;
+ link->head = link->buffer;
+ link->tail = link->buffer;
+ link->end = link->buffer + bufsize;
+ link->buffer_size = bufsize;
+ link->used_buffer_size = bufsize;
+ return 0;
+}
+
+/**
+ * _tsn_free_buffer - remove internal buffers
+ *
+ * This is the buffer where we store data before shipping it to TSN, or
+ * where incoming data is staged.
+ *
+ * @param link - the link that holds the buffer
+ *
+ * Must be called with tsn_lock() held.
+ */
+static void _tsn_free_buffer(struct tsn_link *link)
+{
+ if (!link)
+ return;
+ trace_tsn_free_buffer(link);
+ kfree(link->buffer);
+ link->buffer = NULL;
+ link->head = NULL;
+ link->tail = NULL;
+ link->end = NULL;
+}
+
+int tsn_set_buffer_size(struct tsn_link *link, size_t bsize)
+{
+ if (!link)
+ return -EINVAL;
+
+ if (bsize > link->buffer_size) {
+ pr_err("%s: requested buffer (%zd) larger than allocated memory (%zd)\n",
+ __func__, bsize, link->buffer_size);
+ return -ENOMEM;
+ }
+
+ tsn_lock(link);
+ link->used_buffer_size = bsize;
+ link->tail = link->buffer;
+ link->head = link->buffer;
+ link->end = link->buffer + link->used_buffer_size;
+ link->low_water_mark = _get_low_water(link);
+ link->high_water_mark = _get_high_water(link);
+ tsn_unlock(link);
+
+ pr_info("Set buffer_size, size: %zd, lowwater: %zd, highwater: %zd\n",
+ link->used_buffer_size, link->low_water_mark,
+ link->high_water_mark);
+ return 0;
+}
+EXPORT_SYMBOL(tsn_set_buffer_size);
+
+int tsn_clear_buffer_size(struct tsn_link *link)
+{
+ if (!link)
+ return -EINVAL;
+
+ tsn_lock(link);
+ link->tail = link->buffer;
+ link->head = link->buffer;
+ link->end = link->buffer + link->buffer_size;
+ memset(link->buffer, 0, link->used_buffer_size);
+ link->used_buffer_size = link->buffer_size;
+ link->low_water_mark = _get_low_water(link);
+ link->high_water_mark = _get_high_water(link);
+ tsn_unlock(link);
+ return 0;
+}
+EXPORT_SYMBOL(tsn_clear_buffer_size);
+
+void *tsn_set_external_buffer(struct tsn_link *link, void *buffer,
+ size_t buffer_size)
+{
+ void *old_buffer;
+
+ if (!link)
+ return NULL;
+ if (buffer_size < link->max_payload_size)
+ pr_warn("%s: buffer_size (%zu) < max_payload_size (%u)\n",
+ __func__, buffer_size, link->max_payload_size);
+
+ tsn_lock(link);
+ if (!link->external_buffer && link->buffer)
+ _tsn_free_buffer(link);
+
+ old_buffer = link->buffer;
+ link->external_buffer = 1;
+ link->buffer_size = buffer_size;
+ link->used_buffer_size = buffer_size;
+ link->buffer = buffer;
+ link->head = link->buffer;
+ link->tail = link->buffer;
+ link->end = link->buffer + link->used_buffer_size;
+ tsn_unlock(link);
+ return old_buffer;
+}
+EXPORT_SYMBOL(tsn_set_external_buffer);
+
+/* Caller must hold link->lock!
+ *
+ * Write data *into* buffer, either from net or from shim due to a
+ * closing underflow event.
+ */
+static void __tsn_buffer_write(struct tsn_link *link, void *src, size_t bytes)
+{
+ int rem = 0;
+
+ /* No Need To Wrap, if overflow we will overwrite without
+ * warning.
+ */
+ trace_tsn_buffer_write(link, bytes);
+ if (link->head + bytes < link->end) {
+ memcpy(link->head, src, bytes);
+ link->head += bytes;
+ } else {
+ rem = link->end - link->head;
+ memcpy(link->head, src, rem);
+ memcpy(link->buffer, (src + rem), bytes - rem);
+ link->head = link->buffer + (bytes - rem);
+ }
+}
+
+int tsn_buffer_write(struct tsn_link *link, void *src, size_t bytes)
+{
+ if (!link)
+ return -EINVAL;
+
+ /* We should not do anything if link has gone inactive */
+ if (!tsn_link_is_on(link))
+ return 0;
+
+ /* Copied a batch of data and if link is disabled, it is now
+ * safe to enable it. Otherwise we will continue to send
+ * null-frames to remote.
+ */
+ if (!tsn_lb(link))
+ tsn_lb_enable(link);
+
+ __tsn_buffer_write(link, src, bytes);
+
+ return bytes;
+}
+EXPORT_SYMBOL(tsn_buffer_write);
+
+/**
+ * tsn_buffer_write_net - take data from a skbuff and write it into buffer
+ *
+ * When we receive a frame, we grab data from the skbuff and add it to
+ * link->buffer.
+ *
+ * Note that this routine does NOT CARE about channels, samplesize etc,
+ * it is a _pure_ copy that handles ringbuffer wraps etc.
+ *
+ * This function have side-effects as it will update internal tsn_link
+ * values and trigger refill() should the buffer run low.
+ *
+ * NOTE: called from tsn_rx_handler() -> _tsnh_handle_du(), with
+ * tsn_lock held.
+ *
+ * @param link current link that holds the buffer
+ * @param buffer the buffer to copy from
+ * @param bytes number of bytes
+ * @returns Bytes copied into link->buffer, negative value upon error.
+ */
+int tsn_buffer_write_net(struct tsn_link *link, void *src, size_t bytes)
+{
+ size_t used;
+
+ if (!link)
+ return -EINVAL;
+
+ /* Driver has not been enabled yet, i.e. it is in state 'off' and we
+ * have no way of knowing the state of the buffers.
+ * Silently drop the data, pretend write went ok
+ */
+ trace_tsn_buffer_write_net(link, bytes);
+ if (!tsn_lb(link))
+ return bytes;
+
+ __tsn_buffer_write(link, src, bytes);
+
+ /* If we stored more data than high_water, we need to drain
+ *
+ * In ALSA, this will trigger a snd_pcm_period_elapsed() for the
+ * substream connected to this particular link.
+ */
+ used = _tsn_buffer_used(link);
+ if (used > link->high_water_mark) {
+ trace_tsn_buffer_drain(link, used);
+ link->ops->buffer_drain(link);
+ }
+
+ return bytes;
+}
+
+/* caller must hold link->lock!
+ *
+ * Read data *from* buffer, either to net or to shim due to a
+ * closing overflow event.
+ *
+ * Function will *not* care if you read past head and into unchartered
+ * territory, caller must ascertain validity of bytes.
+ */
+static void __tsn_buffer_read(struct tsn_link *link, void *dst, size_t bytes)
+{
+ int rem = 0;
+
+ trace_tsn_buffer_read(link, bytes);
+ if ((link->tail + bytes) < link->end) {
+ memcpy(dst, link->tail, bytes);
+ link->tail += bytes;
+ } else {
+ rem = link->end - link->tail;
+ memcpy(dst, link->tail, rem);
+ memcpy(dst + rem, link->buffer, bytes - rem);
+ link->tail = link->buffer + bytes - rem;
+ }
+}
+
+/**
+ * tsn_buffer_read_net - read data from link->buffer and give to network layer
+ *
+ * When we send a frame, we grab data from the buffer and add it to the
+ * sk_buff->data, this is primarily done by the Tx-subsystem in tsn_net
+ * and is typically done in small chunks
+ *
+ * @param link current link that holds the buffer
+ * @param buffer the buffer to copy into, must be at least of size bytes
+ * @param bytes number of bytes.
+ *
+ * Note that this routine does NOT CARE about channels, samplesize etc,
+ * it is a _pure_ copy that handles ringbuffer wraps etc.
+ *
+ * This function have side-effects as it will update internal tsn_link
+ * values and trigger refill() should the buffer run low.
+ *
+ * NOTE: expects to be called with locks held
+ *
+ * @return Bytes copied into link->buffer, negative value upon error.
+ */
+int tsn_buffer_read_net(struct tsn_link *link, void *buffer, size_t bytes)
+{
+ size_t used;
+
+ if (!link)
+ return -EINVAL;
+
+ /* link is currently inactive, e.g. we send frames, but without
+ * content
+ *
+ * This can be done before we ship data, or if we are muted
+ * (without expressively stating that over 1722.1
+ *
+ * We do not need to grab any locks here as we won't touch the
+ * link
+ */
+ if (!tsn_lb(link)) {
+ memset(buffer, 0, bytes);
+ goto out;
+ }
+
+ /* sanity check of bytes to read
+ * FIXME
+ */
+
+ __tsn_buffer_read(link, buffer, bytes);
+
+ /* Trigger refill from client app */
+ used = _tsn_buffer_used(link);
+ if (used < link->low_water_mark) {
+ trace_tsn_refill(link, used);
+ link->ops->buffer_refill(link);
+ }
+out:
+ return bytes;
+}
+
+int tsn_buffer_read(struct tsn_link *link, void *buffer, size_t bytes)
+{
+ if (!link)
+ return -EINVAL;
+
+ /* We should not do anything if link has gone inactive */
+ if (!tsn_link_is_on(link))
+ return 0;
+
+ tsn_lock(link);
+ __tsn_buffer_read(link, buffer, bytes);
+ tsn_unlock(link);
+ return bytes;
+}
+EXPORT_SYMBOL(tsn_buffer_read);
+
+static int _tsn_send_batch(struct tsn_link *link)
+{
+ int ret = 0;
+ int num_frames = (link->class_a ? 8 : 4);
+ u64 ts_base_ns = ktime_to_ns(ktime_get()) + (link->class_a ? 2000000 : 50000000);
+ u64 ts_delta_ns = (link->class_a ? 125000 : 250000);
+
+ trace_tsn_send_batch(link, num_frames, ts_base_ns, ts_delta_ns);
+ ret = tsn_net_send_set(link, num_frames, ts_base_ns, ts_delta_ns);
+ if (ret < 0)
+ pr_err("%s: could not send frame - %d\n", __func__, ret);
+
+ return ret;
+}
+
+static int _tsn_hrtimer_callback(struct tsn_link *link)
+{
+ int ret = _tsn_send_batch(link);
+
+ if (ret) {
+ pr_err("%s: Error sending frames (%d), disabling link.\n",
+ __func__, ret);
+ tsn_teardown_link(link);
+ return 0;
+ }
+ return 0;
+}
+
+static enum hrtimer_restart tsn_hrtimer_callback(struct hrtimer *hrt)
+{
+ struct tsn_list *list = container_of(hrt, struct tsn_list, tsn_timer);
+ struct tsn_link *link;
+ struct hlist_node *tmp;
+ int bkt = 0;
+
+ if (!tsn_core_running(list))
+ return HRTIMER_NORESTART;
+
+ hrtimer_forward_now(hrt, ns_to_ktime(list->period_ns));
+
+ hash_for_each_safe(tlinks, bkt, tmp, link, node) {
+ if (tsn_link_is_on(link) && link->estype_talker)
+ _tsn_hrtimer_callback(link);
+ }
+
+ return HRTIMER_RESTART;
+}
+
+static long tsn_hrtimer_init(void *arg)
+{
+ /* Run every 1ms, _tsn_send_batch will figure out how many
+ * frames to send for active frames
+ */
+ struct tsn_list *list = (struct tsn_list *)arg;
+
+ hrtimer_init(&list->tsn_timer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL | HRTIMER_MODE_PINNED);
+
+ list->tsn_timer.function = tsn_hrtimer_callback;
+ hrtimer_cancel(&list->tsn_timer);
+ atomic_set(&list->running, 1);
+
+ hrtimer_start(&list->tsn_timer, ns_to_ktime(list->period_ns),
+ HRTIMER_MODE_REL);
+ return 0;
+}
+
+static void tsn_hrtimer_exit(struct tsn_list *list)
+{
+ atomic_set(&list->running, 0);
+ hrtimer_cancel(&list->tsn_timer);
+}
+
+/**
+ * tsn_prepare_link - prepare link for role as Talker/Receiver
+ *
+ * Iow; this will start shipping data through the network-layer.
+ *
+ * @link: the actual link
+ *
+ * Current status: each link will get a periodic hrtimer that interrupts
+ * and ships data every 1ms. This will change once we have proper driver
+ * for hw (i.e. i210 driver).
+ */
+int tsn_prepare_link(struct tsn_link *link, struct tsn_shim_ops *shim_ops)
+{
+ int ret = 0;
+ void *buffer;
+ u16 framesize;
+ struct net_device *netdev;
+
+ /* TODO: use separate buckets (lists/rbtrees/whatever) for
+ * class_a and class_b talker streams. hrtimer-callback should
+ * not iterate over all.
+ */
+
+ if (!link || !shim_ops || !shim_ops->probe)
+ return -EINVAL;
+
+ pr_info("TSN: allocating buffer, %zd bytes\n", link->buffer_size);
+
+ tsn_lock(link);
+
+ /* configure will calculate idle_slope based on framesize
+ * (header + payload)
+ */
+ netdev = link->nic->dev;
+ if (netdev->netdev_ops->ndo_tsn_link_configure) {
+ framesize = link->max_payload_size +
+ link->shim_header_size + tsnh_len_all();
+ ret = netdev->netdev_ops->ndo_tsn_link_configure(netdev, link->class_a,
+ framesize, link->vlan_id & 0xfff);
+ if (ret < 0)
+ pr_err("Could not configure link - %d\n", ret);
+ }
+
+ link->ops = shim_ops;
+ tsn_unlock(link);
+ ret = link->ops->probe(link);
+ if (ret != 0) {
+ pr_err("%s: Could not probe shim (%d), cannot create link\n",
+ __func__, ret);
+ link->ops = NULL;
+ goto out;
+ }
+
+ tsn_lock(link);
+ if (!link->external_buffer) {
+ buffer = kmalloc(link->buffer_size, GFP_KERNEL);
+ if (!buffer) {
+ pr_err("%s: Could not allocate memory (%zu) for buffer\n",
+ __func__, link->buffer_size);
+ link->ops = NULL;
+ ret = -ENOMEM;
+ goto unlock_out;
+ }
+
+ ret = _tsn_set_buffer(link, buffer, link->buffer_size);
+ if (ret != 0) {
+ pr_err("%s: Could not set buffer for TSN, got %d\n",
+ __func__, ret);
+ goto unlock_out;
+ }
+ } else {
+ /* FIXME: not handled */
+ pr_info("TSN does not currently handle externally hosted buffers. This is on the TODO-list\n");
+ ret = -EINVAL;
+ goto unlock_out;
+ }
+
+ tsn_link_on(link);
+
+unlock_out:
+ tsn_unlock(link);
+out:
+ pr_info("%s: ret=%d\n", __func__, ret);
+ return ret;
+}
+
+int tsn_teardown_link(struct tsn_link *link)
+{
+ if (!link)
+ return -EINVAL;
+
+ tsn_lock(link);
+ tsn_lb_disable(link);
+ tsn_link_off(link);
+ tsn_unlock(link);
+
+ /* Need to call media_close() without (spin-)locks held.
+ */
+ if (link->ops)
+ link->ops->media_close(link);
+
+ tsn_lock(link);
+ link->ops = NULL;
+ _tsn_free_buffer(link);
+ tsn_unlock(link);
+ pr_info("%s: disabling all parts of link\n", __func__);
+ return 0;
+}
+
+int tsn_shim_register_ops(struct tsn_shim_ops *shim_ops)
+{
+ if (!shim_ops)
+ return -EINVAL;
+
+ if (!shim_ops->buffer_refill || !shim_ops->buffer_drain ||
+ !shim_ops->media_close || !shim_ops->copy_size ||
+ !shim_ops->validate_header || !shim_ops->assemble_header ||
+ !shim_ops->get_payload_data)
+ return -EINVAL;
+
+ INIT_LIST_HEAD(&shim_ops->head);
+ list_add_tail(&shim_ops->head, &tsn_shim_ops);
+ return 0;
+}
+EXPORT_SYMBOL(tsn_shim_register_ops);
+
+void tsn_shim_deregister_ops(struct tsn_shim_ops *shim_ops)
+{
+ struct tsn_link *link;
+ struct hlist_node *tmp;
+ int bkt;
+
+ hash_for_each_safe(tlinks, bkt, tmp, link, node) {
+ if (!link)
+ continue;
+ if (link->ops == shim_ops)
+ tsn_teardown_link(link);
+ }
+ list_del(&shim_ops->head);
+}
+EXPORT_SYMBOL(tsn_shim_deregister_ops);
+
+char *tsn_shim_get_active(struct tsn_link *link)
+{
+ if (!link || !link->ops)
+ return "off";
+ return link->ops->shim_name;
+}
+
+struct tsn_shim_ops *tsn_shim_find_by_name(const char *name)
+{
+ struct tsn_shim_ops *ops;
+
+ if (!name || list_empty(&tsn_shim_ops))
+ return NULL;
+
+ list_for_each_entry(ops, &tsn_shim_ops, head) {
+ if (strcmp(name, ops->shim_name) == 0)
+ return ops;
+ }
+ return NULL;
+}
+
+ssize_t tsn_shim_export_probe_triggers(char *page)
+{
+ struct tsn_shim_ops *ops;
+ ssize_t res = 0;
+
+ if (!page || list_empty(&tsn_shim_ops))
+ return 0;
+ list_for_each_entry(ops, &tsn_shim_ops, head) {
+ res += snprintf((page + res), PAGE_SIZE - res, "%s\n",
+ ops->shim_name);
+ }
+ return res;
+}
+
+struct tsn_link *tsn_create_and_add_link(struct tsn_nic *nic)
+{
+ u64 sid = 0;
+ struct tsn_link *link = kzalloc(sizeof(*link), GFP_KERNEL);
+
+ if (!link)
+ return NULL;
+ if (!nic) {
+ kfree(link);
+ return NULL;
+ }
+
+ spin_lock_init(&link->lock);
+ tsn_lock(link);
+ tsn_link_off(link);
+ tsn_lb_disable(link);
+ do {
+ sid = prandom_u32();
+ sid |= prandom_u32() << 31;
+ } while (tsn_find_by_stream_id(sid));
+ link->stream_id = sid;
+
+ /* There's a slim chance that we actually hit on the first frame
+ * of data, but if we do, remote seqnr is most likely 0. If this
+ * is not up to par,, fix in rx_handler
+ */
+ link->last_seqnr = 0xff;
+
+ /* class B audio 48kHz sampling, S16LE, 2ch and IEC61883-6 CIP
+ * header
+ */
+ link->max_payload_size = 48;
+ link->shim_header_size = 8;
+
+ /* Default VLAN ID is SR_PVID (2) unless otherwise supplied from
+ * MSRP, PCP is default 3 for class A, 2 for Class B (See IEEE
+ * 802.1Q-2011, table 6-6)
+ */
+ link->vlan_id = 0x2;
+ link->pcp_a = 3;
+ link->pcp_b = 2;
+ link->class_a = 0;
+
+ link->buffer_size = 16536;
+ /* default: talker since listener isn't implemented yet. */
+ link->estype_talker = 1;
+
+ link->nic = nic;
+ tsn_unlock(link);
+
+ /* Add the newly created link to the hashmap of all active links.
+ *
+ * test if sid is present in hashmap already (barf on that)
+ */
+
+ mutex_lock(&tlist.lock);
+ hash_add(tlinks, &link->node, link->stream_id);
+ mutex_unlock(&tlist.lock);
+ pr_info("%s: added link with stream_id: %llu\n",
+ __func__, link->stream_id);
+
+ return link;
+}
+
+ssize_t tsn_get_stream_ids(char *page, ssize_t len)
+{
+ struct tsn_link *link;
+ struct hlist_node *tmp;
+ char *buffer = page;
+ int bkt;
+
+ if (!page)
+ return 0;
+
+ if (hash_empty(tlinks))
+ return sprintf(buffer, "no links registered\n");
+
+ hash_for_each_safe(tlinks, bkt, tmp, link, node)
+ buffer += sprintf(buffer, "%llu\n", link->stream_id);
+
+ return (buffer - page);
+}
+
+struct tsn_link *tsn_find_by_stream_id(u64 sid)
+{
+ struct tsn_link *link;
+
+ if (hash_empty(tlinks))
+ return 0;
+
+ hash_for_each_possible(tlinks, link, node, sid) {
+ if (link->stream_id == sid)
+ return link;
+ }
+
+ return NULL;
+}
+
+void tsn_remove_link(struct tsn_link *link)
+{
+ if (!link)
+ return;
+ tsn_net_close(link);
+ mutex_lock(&tlist.lock);
+ hash_del(&link->node);
+ if (link->ops) {
+ link->ops->media_close(link);
+ link->ops = NULL;
+ }
+
+ mutex_unlock(&tlist.lock);
+}
+
+void tsn_readd_link(struct tsn_link *link, u64 newkey)
+{
+ if (!link)
+ return;
+ tsn_lock(link);
+ if (hash_hashed(&link->node)) {
+ pr_info("%s: updating link with stream_id %llu -> %llu\n",
+ __func__, link->stream_id, newkey);
+ tsn_remove_link(link);
+ }
+
+ link->stream_id = newkey;
+ tsn_unlock(link);
+
+ hash_add(tlinks, &link->node, link->stream_id);
+}
+
+static int _tsn_capable_nic(struct net_device *netdev, struct tsn_nic *nic)
+{
+ if (!nic || !netdev || !netdev->netdev_ops ||
+ !netdev->netdev_ops->ndo_tsn_capable)
+ return -EINVAL;
+
+ if (netdev->netdev_ops->ndo_tsn_capable(netdev) > 0)
+ nic->capable = 1;
+
+ return 0;
+}
+
+/* Identify all TSN-capable NICs in the system
+ */
+static int tsn_nic_probe(void)
+{
+ struct net *net;
+ struct net_device *netdev;
+ struct tsn_nic *nic;
+
+ net = &init_net;
+ rcu_read_lock();
+ for_each_netdev_rcu(net, netdev) {
+ pr_info("Found %s, alias %s on irq %d\n",
+ netdev->name,
+ netdev->ifalias,
+ netdev->irq);
+ pr_info("MAC: %pM", netdev->dev_addr);
+ if (netdev->tx_queue_len)
+ pr_info("Tx queue length: %lu\n", netdev->tx_queue_len);
+ nic = kzalloc(sizeof(*nic), GFP_KERNEL);
+ if (!nic) {
+ pr_err("Could not allocate memory for tsn_nic!\n");
+ return -ENOMEM;
+ }
+ nic->dev = netdev;
+ nic->txq = netdev->num_tx_queues;
+ nic->name = netdev->name;
+ nic->tsn_list = &tlist;
+ nic->dma_size = 1048576;
+
+ _tsn_capable_nic(netdev, nic);
+
+ /* if not capable and we are not in debug-mode, drop nic
+ * and continue
+ */
+ if (!nic->capable && !in_debug) {
+ pr_info("Invalid capabilities for NIC (%s), dropping from TSN list\n",
+ netdev->name);
+ kfree(nic);
+ continue;
+ }
+
+ INIT_LIST_HEAD(&nic->list);
+ mutex_lock(&tlist.lock);
+ list_add_tail(&nic->list, &tlist.head);
+ tlist.num_avail++;
+ mutex_unlock(&tlist.lock);
+ }
+ rcu_read_unlock();
+
+ return 0;
+}
+
+static void tsn_free_nic_list(struct tsn_list *list)
+{
+ struct tsn_nic *tmp, *next;
+
+ mutex_lock(&list->lock);
+ list_for_each_entry_safe(tmp, next, &list->head, list) {
+ pr_info("Dropping %s from list\n", tmp->dev->name);
+ list_del(&tmp->list);
+ tmp->dev = NULL;
+ kfree(tmp);
+ }
+ mutex_unlock(&list->lock);
+}
+
+/* all active links are stored in hashmap 'tlinks'
+ */
+static void tsn_remove_all_links(void)
+{
+ int bkt;
+ struct tsn_link *link;
+ struct hlist_node *tmp;
+
+ hash_for_each_safe(tlinks, bkt, tmp, link, node) {
+ pr_info("%s removing a link\n", __func__);
+ if (!tsn_teardown_link(link))
+ tsn_remove_link(link);
+ }
+
+ pr_info("%s: all links have been removed\n", __func__);
+}
+
+static int __init tsn_init_module(void)
+{
+ int ret = 0;
+
+ INIT_LIST_HEAD(&tlist.head);
+ mutex_init(&tlist.lock);
+
+ atomic_set(&tlist.running, 0);
+ tlist.period_ns = 1000000;
+
+ /* Find all NICs, attach a rx-handler for sniffing out TSN
+ * traffic on *all* of them.
+ */
+ tlist.num_avail = 0;
+ ret = tsn_nic_probe();
+ if (ret < 0) {
+ pr_err("%s: somethign went awry whilst probing for NICs, aborting\n",
+ __func__);
+ goto out;
+ }
+
+ if (!tlist.num_avail) {
+ pr_err("%s: No capable NIC found. Perhaps load with in_debug=1 ?\n",
+ __func__);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* register Rx-callbacks for all (valid) NICs */
+ ret = tsn_net_add_rx(&tlist);
+ if (ret < 0) {
+ pr_err("%s: Could add Rx-handler, aborting\n", __func__);
+ goto error_rx_out;
+ }
+
+ /* init DMA regions etc */
+ ret = tsn_net_prepare_tx(&tlist);
+ if (ret < 0) {
+ pr_err("%s: could not prepare Tx, aborting\n", __func__);
+ goto error_tx_out;
+ }
+
+ /* init hashtable */
+ hash_init(tlinks);
+
+ /* init configfs */
+ ret = tsn_configfs_init(&tlist);
+ if (ret < 0) {
+ pr_err("%s: Could not initialize configfs properly (%d), aborting\n",
+ __func__, ret);
+ goto error_cfs_out;
+ }
+
+ /* Test to see if on_cpu is available */
+ if (on_cpu >= 0) {
+ pr_info("%s: pinning timer on CPU %d\n", __func__, on_cpu);
+ ret = work_on_cpu(on_cpu, tsn_hrtimer_init, &tlist);
+ if (ret != 0) {
+ pr_err("%s: could not init hrtimer properly on CPU %d, aborting\n",
+ __func__, on_cpu);
+ goto error_hrt_out;
+ }
+ } else {
+ ret = tsn_hrtimer_init(&tlist);
+ if (ret < 0) {
+ pr_err("%s: could not init hrtimer properly, aborting\n",
+ __func__);
+ goto error_hrt_out;
+ }
+ }
+ pr_info("TSN subsystem init OK\n");
+ return 0;
+
+error_hrt_out:
+ tsn_remove_all_links();
+ tsn_configfs_exit(&tlist);
+error_cfs_out:
+ tsn_net_disable_tx(&tlist);
+error_tx_out:
+ tsn_net_remove_rx(&tlist);
+error_rx_out:
+ tsn_free_nic_list(&tlist);
+out:
+ return ret;
+}
+
+static void __exit tsn_exit_module(void)
+{
+ pr_warn("removing module TSN\n");
+ tsn_hrtimer_exit(&tlist);
+
+ tsn_remove_all_links();
+ tsn_configfs_exit(&tlist);
+
+ /* Unregister Rx-handlers if set */
+ tsn_net_remove_rx(&tlist);
+
+ tsn_net_disable_tx(&tlist);
+
+ tsn_free_nic_list(&tlist);
+
+ pr_warn("TSN exit\n");
+}
+module_param(in_debug, int, S_IRUGO);
+module_param(on_cpu, int, S_IRUGO);
+module_init(tsn_init_module);
+module_exit(tsn_exit_module);
+MODULE_AUTHOR("Henrik Austad");
+MODULE_LICENSE("GPL");
diff --git a/net/tsn/tsn_header.c b/net/tsn/tsn_header.c
new file mode 100644
index 0000000..a0d31c5
--- /dev/null
+++ b/net/tsn/tsn_header.c
@@ -0,0 +1,203 @@
+/*
+ * Network header handling for TSN
+ *
+ * Copyright (C) 2015- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#include <linux/tsn.h>
+#include <trace/events/tsn.h>
+
+#include "tsn_internal.h"
+
+#define AVTP_GPTP_TIMEMASK 0xFFFFFFFF
+
+static u32 tsnh_avtp_timestamp(u64 ptime_ns)
+{
+ /* See 1722-2011, 5.4.8
+ *
+ * (AS_sec * 1e9 + AS_ns) % 2^32
+ *
+ * Just use ktime_get_ns() and grab lower 32 bits of it.
+ */
+ /* u64 ns = ktime_to_ns(ktime_get()); */
+ u32 gptp_ts = ptime_ns & AVTP_GPTP_TIMEMASK;
+ return gptp_ts;
+}
+
+int tsnh_ch_init(struct avtp_ch *header)
+{
+ if (!header)
+ return -EINVAL;
+ header = memset(header, 0, sizeof(*header));
+
+ /* This should be changed when setting control / data
+ * content. Set to experimental to allow for strange content
+ * should callee not do job properly
+ */
+ header->subtype = AVTP_EXPERIMENTAL;
+
+ header->version = 0;
+ return 0;
+}
+
+int _tsnh_validate_du_header(struct tsn_link *link, struct avtp_ch *ch,
+ struct sk_buff *skb)
+{
+ struct avtpdu_header *header = (struct avtpdu_header *)ch;
+ struct sockaddr_ll *sll;
+ u16 bytes;
+ u8 seqnr;
+
+ if (ch->cd)
+ return -EINVAL;
+
+ /* As a minimum, we should match the sender's MAC to the
+ * expected MAC before we pass the frame along.
+ *
+ * This does not give much in the way of security (a malicious
+ * user could probably fake this), but it should remove most
+ * accidents.
+ */
+ sll = (struct sockaddr_ll *)&skb->cb;
+ sll->sll_halen = dev_parse_header(skb, sll->sll_addr);
+ if (sll->sll_halen != 6) {
+ trace_printk("%s: received MAC address length mismatch. Expected 6 bytes, got %d\n",
+ __func__, sll->sll_halen);
+ return -EPROTO;
+ }
+
+ if (memcmp(link->remote_mac, &sll->sll_addr, 6)) {
+ trace_printk("%s: received MAC-address mismatch (expected %pM, got %pM), dropping frame\n",
+ __func__, link->remote_mac, &sll->sll_addr);
+ return -EPROTO;
+ }
+
+ /* Current iteration of TSNis 0b000 only */
+ if (ch->version)
+ return -EPROTO;
+
+ /* Invalid StreamID, should not have ended up here in the first
+ * place (since we do DU only), if invalid sid, how did we find
+ * the link?
+ */
+ if (!ch->sv)
+ return -EPROTO;
+
+ /* Check seqnr, if we have lost one frame, we _could_ insert an
+ * empty frame, but since we have frame-guarantee from 802.1Qav,
+ * we don't
+ */
+ seqnr = (link->last_seqnr + 1) & 0xff;
+ if (header->seqnr != seqnr) {
+ trace_printk("%llu: seqnr mismatch. Got %u, expected %u\n",
+ link->stream_id, header->seqnr, seqnr);
+ return -EPROTO;
+ }
+
+ bytes = ntohs(header->sd_len);
+ if (bytes == 0 || bytes > link->max_payload_size) {
+ trace_printk("%llu: payload size larger than expected (%u, expected %u)\n",
+ link->stream_id, bytes, link->max_payload_size);
+ return -EINVAL;
+ }
+
+ /* let shim validate header here as well */
+ if (link->ops->validate_header &&
+ link->ops->validate_header(link, header) != 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+int tsnh_assemble_du(struct tsn_link *link, struct avtpdu_header *header,
+ size_t bytes, u64 ts_pres_ns)
+{
+ int ret = 0;
+ void *data;
+
+ if (!header || !link)
+ return -EINVAL;
+
+ tsnh_ch_init((struct avtp_ch *)header);
+ header->cd = 0;
+ header->sv = 1;
+ header->mr = 0;
+ header->gv = 0;
+ header->tv = 1;
+ header->tu = 0;
+ header->avtp_timestamp = htonl(tsnh_avtp_timestamp(ts_pres_ns));
+ header->gateway_info = 0;
+ header->sd_len = htons(bytes);
+
+ tsn_lock(link);
+ if (!link->ops) {
+ pr_err("%s: No available ops, cannot assemble data-unit\n",
+ __func__);
+ ret = -EINVAL;
+ goto unlock_out;
+ }
+ /* get pointer to where data starts */
+ data = link->ops->get_payload_data(link, header);
+
+ if (bytes > link->used_buffer_size) {
+ pr_err("bytes > buffer_size (%zd > %zd)\n",
+ bytes, link->used_buffer_size);
+ ret = -EINVAL;
+ goto unlock_out;
+ }
+
+ header->stream_id = cpu_to_be64(link->stream_id);
+ header->seqnr = link->last_seqnr++;
+ link->ops->assemble_header(link, header, bytes);
+ tsn_unlock(link);
+
+ /* payload */
+ ret = tsn_buffer_read_net(link, data, bytes);
+ if (ret != bytes) {
+ pr_err("%s: Could not copy %zd bytes of data. Res: %d\n",
+ __func__, bytes, ret);
+ /* FIXME: header cleanup */
+ goto out;
+ }
+ ret = 0;
+out:
+ return ret;
+unlock_out:
+ tsn_unlock(link);
+ return ret;
+}
+
+int _tsnh_handle_du(struct tsn_link *link, struct avtp_ch *ch)
+{
+ struct avtpdu_header *header = (struct avtpdu_header *)ch;
+ void *data;
+ u16 bytes;
+ int ret;
+
+ bytes = ntohs(header->sd_len);
+
+ trace_tsn_du(link, bytes);
+ /* bump seqnr */
+ data = link->ops->get_payload_data(link, header);
+ if (!data)
+ return -EINVAL;
+
+ link->last_seqnr = header->seqnr;
+ ret = tsn_buffer_write_net(link, data, bytes);
+ if (ret != bytes) {
+ trace_printk("%s: Could not copy %u bytes of data. Res: %d\n",
+ __func__, bytes, ret);
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/net/tsn/tsn_internal.h b/net/tsn/tsn_internal.h
new file mode 100644
index 0000000..d0d2201
--- /dev/null
+++ b/net/tsn/tsn_internal.h
@@ -0,0 +1,383 @@
+/*
+ * Copyright (C) 2015- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef _TSN_INTERNAL_H_
+#define _TSN_INTERNAL_H_
+#include <linux/tsn.h>
+
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+
+/* TODO:
+ * - hide tsn-structs and provide handlers
+ * - decouple config/net from core
+ */
+
+struct avtpdu_header;
+struct tsn_link;
+struct tsn_shim_ops;
+
+#define IS_TSN_FRAME(x) (ntohs(x) == ETH_P_TSN)
+#define IS_PTP_FRAME(x) (ntohs(x) == ETH_P_1588)
+#define IS_1Q_FRAME(x) (ntohs(x) == ETH_P_8021Q)
+
+/**
+ * tsn_add_link - create and add a new link to the system
+ *
+ * Note: this will not enable the link, just allocate most of the data
+ * required for the link. One notable exception being the buffer as we
+ * can modify the buffersize before we start the link.
+ *
+ * @param nic : the nic the link is tied to
+ * @returns the new link
+ */
+struct tsn_link *tsn_create_and_add_link(struct tsn_nic *nic);
+
+/**
+ * tsn_get_stream_ids - write all current Stream IDs into the page.
+ *
+ * @param page the page to write into
+ * @param len size of page
+ * @returns the number of bytes written
+ */
+ssize_t tsn_get_stream_ids(char *page, ssize_t len);
+
+/**
+ * tsn_find_by_stream_id - given a sid, find the corresponding link
+ *
+ * @param sid stream_id
+ * @returns tsn_link struct or NULL if not found
+ */
+struct tsn_link *tsn_find_by_stream_id(u64 sid);
+
+/**
+ * tsn_readd_link - make sure a link is moved to the correct bucket when
+ * stream_id is updated
+ *
+ * @link the TSN link
+ * @old_key previous key for which it can be located in the hashmap
+ *
+ */
+void tsn_readd_link(struct tsn_link *link, u64 old_key);
+
+/**
+ * tsn_remove_link: cleanup and remove from internal storage
+ *
+ * @link: the link to be removed
+ */
+void tsn_remove_link(struct tsn_link *link);
+
+/**
+ * tsn_prepare_link - make link ready for usage
+ *
+ * Caller is happy with the different knobs, this will create the link and start
+ * pushing the data.
+ *
+ * Requirement:
+ * - callback registered
+ * - State set to either Talker or Listener
+ *
+ * @param active link
+ * @param the shim_ops to use for the new link
+ * @return 0 on success, negative on error
+ */
+int tsn_prepare_link(struct tsn_link *link, struct tsn_shim_ops *shim_ops);
+int tsn_teardown_link(struct tsn_link *link);
+
+/**
+ * tsn_set_external_buffer - force an update of the buffer
+ *
+ * This will cause tsn_core to use an external buffer. If external
+ * buffering is already in use, this has the effect of forcing an update
+ * of the buffer.
+ *
+ * This will cause tsn_core to swap buffers. The current buffer is
+ * returned and the new is used in place.
+ *
+ * Note: If the new buffer is NULL or buffer_size is less than
+ * max_payload_size, the result can be interesting (by calling this
+ * function, you claim to know what you are doing and should pass sane
+ * values).
+ *
+ * This can also be used if you need to resize the buffer in use.
+ *
+ * Core will continue to use the tsn_shim_swap when the new buffer is
+ * full.
+ *
+ * @param link current link owning the buffer
+ * @param buffer new buffer to use
+ * @param buffer_size size of new buffer
+ * @return old buffer
+ */
+void *tsn_set_external_buffer(struct tsn_link *link, void *buffer,
+ size_t buffer_size);
+
+/**
+ * tsn_buffer_write_net - write data *into* link->buffer from the network layer
+ *
+ * Used by tsn_net and will typicall accept very small pieces of data.
+ *
+ * @param link the link associated with the stream_id in the frame
+ * @param src pointer to data in buffer
+ * @param bytes number of bytes to copy
+ * @return number of bytes copied into the buffer
+ */
+int tsn_buffer_write_net(struct tsn_link *link, void *src, size_t bytes);
+
+/**
+ * tsn_buffer_read_net - read data from link->buffer and give to network layer
+ *
+ * When we send a frame, we grab data from the buffer and add it to the
+ * sk_buff->data, this is primarily done by the Tx-subsystem in tsn_net
+ * and is typically done in small chunks
+ *
+ * @param link current link that holds the buffer
+ * @param buffer the buffer to copy into, must be at least of size bytes
+ * @param bytes number of bytes.
+ *
+ * Note that this routine does NOT CARE about channels, samplesize etc,
+ * it is a _pure_ copy that handles ringbuffer wraps etc.
+ *
+ * This function have side-effects as it will update internal tsn_link
+ * values and trigger refill() should the buffer run low.
+ *
+ * @return Bytes copied into link->buffer, negative value upon error.
+ */
+int tsn_buffer_read_net(struct tsn_link *link, void *buffer, size_t bytes);
+
+/**
+ * tsn_core_running(): test if the link is running
+ *
+ * By running, we mean that it is configured and a proper shim has been
+ * loaded. It does *not* mean that we are currently pushing data in any
+ * direction, see tsn_net_buffer_disabled() for this
+ *
+ * @param struct tsn_link active link
+ * @returns 1 if core is running
+ */
+static inline int tsn_core_running(struct tsn_list *list)
+{
+ if (list)
+ return atomic_read(&list->running);
+ return 0;
+}
+
+/**
+ * _tsn_buffer_used - how much of the buffer is filled with valid data
+ *
+ * - assumes link->running in state running
+ * - will ignore change changed state
+ *
+ * We write to head, read from tail.
+ */
+static inline size_t _tsn_buffer_used(struct tsn_link *link)
+{
+ return (link->head - link->tail) % link->used_buffer_size;
+}
+
+static inline void tsn_lock(struct tsn_link *link)
+{
+ spin_lock(&link->lock);
+}
+
+static inline void tsn_unlock(struct tsn_link *link)
+{
+ spin_unlock(&link->lock);
+}
+
+/* -----------------------------
+ * ConfigFS handling
+ */
+int tsn_configfs_init(struct tsn_list *tlist);
+void tsn_configfs_exit(struct tsn_list *tlist);
+
+/* -----------------------------
+ * TSN Header
+ */
+
+static inline size_t tsnh_len(void)
+{
+ /* include 802.1Q tag */
+ return sizeof(struct avtpdu_header);
+}
+
+static inline u16 tsnh_len_all(void)
+{
+ return (u16)tsnh_len() + ETH_HLEN;
+}
+
+/**
+ * tsnh_payload_size_valid - if the entire payload is within size-limit
+ *
+ * Ensure that max_payload_size and shim_header_size is within acceptable limits
+ *
+ * We need both values to calculate the payload size when reserving
+ * bandwidth, but only payload-size when instructing the shim to copy
+ * out data for us.
+ *
+ * @param max_payload_size requested payload to send in each frame (upper limit)
+ * @return 0 on invalid, 1 on valid
+ */
+static inline int tsnh_payload_size_valid(u16 max_payload_size,
+ u16 shim_hdr_size)
+{
+ /* VLAN_ETH_ZLEN 64 */
+ /* VLAN_ETH_FRAME_LEN 1518 */
+ u32 framesize = max_payload_size + tsnh_len_all() + shim_hdr_size;
+
+ return framesize >= VLAN_ETH_ZLEN && framesize <= VLAN_ETH_FRAME_LEN;
+}
+
+/**
+ * _tsnh_validate_du_header - basic header validation
+ *
+ * This expects the parameters to be present and the link-lock to be
+ * held.
+ *
+ * @param header header to verify
+ * @param link owner of stream
+ * @param socket_buffer
+ * @return 0 on valid, negative on invalid/error
+ */
+int _tsnh_validate_du_header(struct tsn_link *link, struct avtp_ch *ch,
+ struct sk_buff *skb);
+
+/**
+ * tsnh_assemble_du - assemble header and copy data from buffer
+ *
+ * This function will initialize the header and pass final init to
+ * shim->assemble_header before copying data into the buffer.
+ *
+ * It assumes that 'bytes' is a sane value, i.e. that it is a valid
+ * multiple of number of channels, sample size etc.
+ *
+ * @param link Current TSN link, also holds the buffer
+ *
+ * @param header header to assemble for data
+ *
+ * @param bytes Number of bytes to send in this frame
+ *
+ * @param ts_pres_ns current for when the frame should be presented or
+ * considered valid by the receiving end. In
+ * nanoseconds since epoch, will be converted to gPTP
+ * compatible timestamp.
+ *
+ * @return 0 on success, negative on error
+ */
+int tsnh_assemble_du(struct tsn_link *link, struct avtpdu_header *header,
+ size_t bytes, u64 ts_pres_ns);
+
+/**
+ * _tsnh_handle_du - handle incoming data and store to media-buffer
+ *
+ * This assumes that the frame actually belongs to the link and that it
+ * has passed basic validation.
+ *
+ * It also expects the link lock to be held.
+ *
+ * @param link Link associated with stream_id
+ * @param header Header of incoming frame
+ * @return number of bytes copied to buffer or negative on error
+ */
+int _tsnh_handle_du(struct tsn_link *link, struct avtp_ch *ch);
+
+static inline struct avtp_ch *tsnh_ch_from_skb(struct sk_buff *skb)
+{
+ if (!skb)
+ return NULL;
+ if (!IS_TSN_FRAME(eth_hdr(skb)->h_proto))
+ return NULL;
+
+ return (struct avtp_ch *)skb->data;
+}
+
+/**
+ * tsn_net_add_rx - add Rx handler for all NICs listed
+ *
+ * @param list tsn_list to add Rx handler to
+ * @return 0 on success, negative on error
+ */
+int tsn_net_add_rx(struct tsn_list *list);
+
+/**
+ * tsn_net_remove_rx - remove Rx-handlers for all tsn_nics
+ *
+ * Go through all NICs and remove those Rx-handlers we have
+ * registred. If someone else has added an Rx-handler to the NIC, we do
+ * not touch it.
+ *
+ * @param list list of all tsn_nics (with links)
+ */
+void tsn_net_remove_rx(struct tsn_list *list);
+
+/**
+ * tsn_net_open_tx - prepare all capable links for Tx
+ *
+ * This will prepare all NICs for Tx, and those marked as 'capable'
+ * will be initialized with DMA regions. Note that this is not the final
+ * step for preparing for Tx, it is only when we have active links that
+ * we know how much bandwidth we need and then can set the appropriate
+ * idleSlope params etc.
+ *
+ * @tlist: list of all available card
+ * @return: negative on error, on success the number of prepared NICS
+ * are returned.
+ */
+int tsn_net_prepare_tx(struct tsn_list *tlist);
+
+/**
+ * tsn_net_disable_tx - disable Tx on card
+ *
+ * This frees DMA-memory from capable NICs
+ *
+ * @param tsn_list: link to all available NICs used by TSN
+ */
+void tsn_net_disable_tx(struct tsn_list *tlist);
+
+/**
+ * tsn_net_set_vlan - try to register the VLAN on the NIC
+ *
+ * Some NICs will handle VLAN themselves, try to register this vlan with
+ * the card to enable hw-support for Tx via this VLAN
+ *
+ * @param: tsn_link the active link
+ * @return: 0 on success, negative on error.
+ */
+int tsn_net_set_vlan(struct tsn_link *link);
+
+/**
+ * tsn_net_close - close down link properly
+ *
+ * @param struct tsn_link * active link to close down
+ */
+void tsn_net_close(struct tsn_link *link);
+
+/**
+ * tsn_net_send_set - send a set of frames
+ *
+ * We want to assemble a number of sk_buffs at a time and ship them off
+ * in a single go and then go back to sleep. Pacing should be done by
+ * hardware, or if we are in in_debug, we don't really care anyway
+ *
+ * @param link : current TSN-link
+ * @param num : the number of frames to create
+ * @param ts_base_ns : base timestamp for when the frames should be
+ * considered valid
+ * @param ts_delta_ns : time between each frame in the set
+ */
+int tsn_net_send_set(struct tsn_link *link, size_t num, u64 ts_base_ns,
+ u64 ts_delta_ns);
+
+#endif /* _TSN_INTERNAL_H_ */
diff --git a/net/tsn/tsn_net.c b/net/tsn/tsn_net.c
new file mode 100644
index 0000000..560e2fd
--- /dev/null
+++ b/net/tsn/tsn_net.c
@@ -0,0 +1,403 @@
+/*
+ * Network part of TSN
+ *
+ * Copyright (C) 2015- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/socket.h>
+#include <linux/skbuff.h>
+#include <linux/if_vlan.h>
+#include <linux/skbuff.h>
+#include <net/sock.h>
+
+#include <linux/tsn.h>
+#include <trace/events/tsn.h>
+#include "tsn_internal.h"
+
+/**
+ * tsn_rx_handler - consume all TSN-tagged frames and forward to tsn_link.
+ *
+ * This handler, if it regsters properly, will consume all TSN-tagged
+ * frames belonging to registered Stream IDs
+ *
+ * Unknown StreamIDs will be passed through without being touched.
+ *
+ * @param pskb sk_buff with incomign data
+ * @returns RX_HANDLER_CONSUMED for TSN frames to known StreamIDs,
+ * RX_HANDLER_PASS for everything else.
+ */
+static rx_handler_result_t tsn_rx_handler(struct sk_buff **pskb)
+{
+ struct sk_buff *skb = *pskb;
+ const struct ethhdr *ethhdr = eth_hdr(skb);
+ struct avtp_ch *ch;
+ struct tsn_link *link;
+ rx_handler_result_t ret = RX_HANDLER_PASS;
+
+ ch = tsnh_ch_from_skb(skb);
+ if (!ch)
+ return RX_HANDLER_PASS;
+ /* We do not (currently) touch control_data frames. */
+ if (ch->cd)
+ return RX_HANDLER_PASS;
+
+ link = tsn_find_by_stream_id(be64_to_cpu(ch->stream_id));
+ if (!link)
+ return RX_HANDLER_PASS;
+
+ tsn_lock(link);
+
+ if (!tsn_link_is_on(link))
+ goto out_unlock;
+
+ /* If link->ops is not set yet, there's nothing we can do, just
+ * ignore this frame
+ */
+ if (!link->ops)
+ goto out_unlock;
+
+ if (_tsnh_validate_du_header(link, ch, skb))
+ goto out_unlock;
+
+ trace_tsn_rx_handler(link, ethhdr, be64_to_cpu(ch->stream_id));
+
+ /* Handle dataunit, if it failes, pass on the frame and let
+ * userspace pick it up.
+ */
+ if (_tsnh_handle_du(link, ch) < 0)
+ goto out_unlock;
+
+ /* Done, data has been copied, free skb and return consumed */
+ consume_skb(skb);
+ ret = RX_HANDLER_CONSUMED;
+
+out_unlock:
+ tsn_unlock(link);
+ return ret;
+}
+
+int tsn_net_add_rx(struct tsn_list *tlist)
+{
+ struct tsn_nic *nic;
+
+ if (!tlist)
+ return -EINVAL;
+
+ /* Setup receive handler for TSN traffic.
+ *
+ * Receive will happen all the time, once a link is active as a
+ * Listener, we will add a hook into the receive-handler to
+ * steer the frames to the correct link.
+ *
+ * We try to add Rx-handlers to all the card listed in tlist (we
+ * assume core has filtered the NICs appropriatetly sothat only
+ * TSN-capable cards are present).
+ */
+ mutex_lock(&tlist->lock);
+ list_for_each_entry(nic, &tlist->head, list) {
+ rtnl_lock();
+ if (netdev_rx_handler_register(nic->dev, tsn_rx_handler, nic) < 0) {
+ pr_err("%s: could not attach an Rx-handler to %s, this link will not be able to accept TSN traffic\n",
+ __func__, nic->name);
+ rtnl_unlock();
+ continue;
+ }
+ rtnl_unlock();
+ pr_info("%s: attached rx-handler to %s\n",
+ __func__, nic->name);
+ nic->rx_registered = 1;
+ }
+ mutex_unlock(&tlist->lock);
+ return 0;
+}
+
+void tsn_net_remove_rx(struct tsn_list *tlist)
+{
+ struct tsn_nic *nic;
+
+ if (!tlist)
+ return;
+ mutex_lock(&tlist->lock);
+ list_for_each_entry(nic, &tlist->head, list) {
+ rtnl_lock();
+ if (nic->rx_registered)
+ netdev_rx_handler_unregister(nic->dev);
+ rtnl_unlock();
+ nic->rx_registered = 0;
+ pr_info("%s: RX-handler for %s removed\n",
+ __func__, nic->name);
+ }
+ mutex_unlock(&tlist->lock);
+}
+
+int tsn_net_prepare_tx(struct tsn_list *tlist)
+{
+ struct tsn_nic *nic;
+ struct device *dev;
+ int ret = 0;
+
+ if (!tlist)
+ return -EINVAL;
+
+ mutex_lock(&tlist->lock);
+ list_for_each_entry(nic, &tlist->head, list) {
+ if (!nic)
+ continue;
+ if (!nic->capable)
+ continue;
+
+ if (!nic->dev->netdev_ops)
+ continue;
+
+ dev = nic->dev->dev.parent;
+ nic->dma_mem = dma_alloc_coherent(dev, nic->dma_size,
+ &nic->dma_handle, GFP_KERNEL);
+ if (!nic->dma_mem) {
+ nic->capable = 0;
+ nic->dma_size = 0;
+ continue;
+ }
+ ret++;
+ }
+ mutex_unlock(&tlist->lock);
+ pr_info("%s: configured %d cards to use DMA\n", __func__, ret);
+ return ret;
+}
+
+void tsn_net_disable_tx(struct tsn_list *tlist)
+{
+ struct tsn_nic *nic;
+ struct device *dev;
+ int res = 0;
+
+ if (!tlist)
+ return;
+ mutex_lock(&tlist->lock);
+ list_for_each_entry(nic, &tlist->head, list) {
+ if (nic->capable && nic->dma_mem) {
+ dev = nic->dev->dev.parent;
+ dma_free_coherent(dev, nic->dma_size, nic->dma_mem,
+ nic->dma_handle);
+ res++;
+ }
+ }
+ mutex_unlock(&tlist->lock);
+ pr_info("%s: freed DMA regions from %d cards\n", __func__, res);
+}
+
+void tsn_net_close(struct tsn_link *link)
+{
+ /* struct tsn_rx_handler_data *rx_data; */
+
+ /* Careful! we need to make sure that we actually succeeded in
+ * registering the handler in open unless we want to unregister
+ * some random rx_handler..
+ */
+ if (!link->estype_talker) {
+ ;
+ /* Make sure we notify rx-handler so it doesn't write
+ * into NULL
+ */
+ }
+}
+
+int tsn_net_set_vlan(struct tsn_link *link)
+{
+ int err;
+ struct tsn_nic *nic = link->nic;
+ const struct net_device_ops *ops = nic->dev->netdev_ops;
+
+ int vf = 2;
+ u16 vlan = link->vlan_id;
+ u8 qos = link->class_a ? link->pcp_a : link->pcp_b;
+
+ pr_info("%s:%s Setting vlan=%u,vf=%d,qos=%u\n",
+ __func__, nic->name, vlan, vf, qos);
+ if (ops->ndo_set_vf_vlan) {
+ err = ops->ndo_set_vf_vlan(nic->dev, vf, vlan, qos);
+ if (err != 0) {
+ pr_err("%s:%s could not set VLAN to %u, got %d\n",
+ __func__, nic->name, vlan, err);
+ return -EINVAL;
+ }
+ return 0;
+ }
+ return -1;
+}
+
+static inline u16 _get_8021q_vid(struct tsn_link *link)
+{
+ u16 pcp = link->class_a ? link->pcp_a : link->pcp_b;
+ /* If not explicitly provided, use SR_PVID 0x2*/
+ return (link->vlan_id & VLAN_VID_MASK) | ((pcp & 0x7) << 13);
+}
+
+/* create and initialize a sk_buff with appropriate TSN Header values
+ *
+ * layout of frame:
+ * - Ethernet header
+ * dst (6) | src (6) | 802.1Q (4) | EtherType (2)
+ * - 1722 (sizeof struct avtpdu)
+ * - payload data
+ * - type header (e.g. iec61883-6 hdr)
+ * - payload data
+ *
+ * Required size:
+ * Ethernet: 18 -> VLAN_ETH_HLEN
+ * 1722: tsnh_len()
+ * payload: shim_hdr_size + data_bytes
+ *
+ * Note:
+ * - seqnr is not set
+ * - payload is not set
+ */
+static struct sk_buff *_skbuf_create_init(struct tsn_link *link,
+ size_t data_bytes,
+ size_t shim_hdr_size,
+ u64 ts_pres_ns, u8 more)
+{
+ struct sk_buff *skb = NULL;
+ struct avtpdu_header *avtpdu;
+ struct net_device *netdev = link->nic->dev;
+ int queue_idx;
+ int res = 0;
+ int hard_hdr_len;
+
+ /* length is size of AVTPDU + data
+ * +-----+ <-- head
+ * | - link layer header
+ * | - 1722 header (avtpdu_header)
+ * +-----+ <-- data
+ * | - shim_header
+ * | - data
+ * +-----+ <-- tail
+ * |
+ * +-----+ <--end
+ * We stuff all of TSN-related
+ * headers in the data-segment to make it easy
+ */
+ size_t hdr_len = VLAN_ETH_HLEN;
+ size_t avtpdu_len = tsnh_len() + shim_hdr_size + data_bytes;
+
+ skb = alloc_skb(hdr_len + avtpdu_len + netdev->needed_tailroom,
+ GFP_ATOMIC | GFP_DMA);
+ if (!skb)
+ return NULL;
+ skb_reserve(skb, hdr_len);
+
+ skb->protocol = htons(ETH_P_TSN);
+ skb->pkt_type = PACKET_OUTGOING;
+ skb->priority = (link->class_a ? link->pcp_a : link->pcp_b);
+ skb->dev = link->nic->dev;
+ skb_shinfo(skb)->tx_flags |= SKBTX_HW_TSTAMP;
+ skb->xmit_more = (more > 0 ? 1 : 0);
+ skb_set_mac_header(skb, 0);
+
+ /* We are using a ethernet-type frame (even though we could send
+ * TSN over other medium.
+ *
+ * - skb_push(skb, ETH_HLEN)
+ * - set header htons(header)
+ * - set source addr (netdev mac addr)
+ * - set dest addr
+ * - return ETH_HLEN
+ */
+ hard_hdr_len = dev_hard_header(skb, skb->dev, ETH_P_TSN,
+ link->remote_mac, NULL, 6);
+
+ skb = vlan_insert_tag(skb, htons(ETH_P_8021Q), _get_8021q_vid(link));
+ if (!skb) {
+ pr_err("%s: could not insert tag in buffer, aborting\n",
+ __func__);
+ return NULL;
+ }
+
+ /* tsnh_assemble_du() will deref avtpdu to find start of data
+ * segment and use that, this is to update the skb
+ * appropriately.
+ *
+ * tsnh_assemble_du() will grab tsn-lock before updating link
+ */
+ avtpdu = (struct avtpdu_header *)skb_put(skb, avtpdu_len);
+ res = tsnh_assemble_du(link, avtpdu, data_bytes, ts_pres_ns);
+ if (res < 0) {
+ pr_err("%s: Error initializing header (-> %d) , we are in an inconsistent state!\n",
+ __func__, res);
+ kfree_skb(skb);
+ return NULL;
+ }
+
+ /* FIXME: Find a suitable Tx-queue
+ *
+ * For igb, this returns -1
+ */
+ queue_idx = sk_tx_queue_get(skb->sk);
+ if (queue_idx < 0 || queue_idx >= netdev->real_num_tx_queues)
+ queue_idx = 0;
+ skb_set_queue_mapping(skb, queue_idx);
+ skb->queue_mapping = 0;
+
+ skb->csum = skb_checksum(skb, 0, hdr_len + data_bytes, 0);
+ return skb;
+}
+
+/**
+ * Send a set of frames as efficiently as possible
+ */
+int tsn_net_send_set(struct tsn_link *link, size_t num, u64 ts_base_ns,
+ u64 ts_delta_ns)
+{
+ struct sk_buff *skb;
+ struct net_device *dev;
+ size_t data_size;
+ int res;
+ struct netdev_queue *txq;
+ u64 ts_pres_ns = ts_base_ns;
+
+ if (!link)
+ return -EINVAL;
+ dev = link->nic->dev;
+
+ /* create and init sk_buff_head */
+ while (num-- > 0) {
+ data_size = tsn_shim_get_framesize(link);
+
+ skb = _skbuf_create_init(link, data_size,
+ tsn_shim_get_hdr_size(link),
+ ts_pres_ns, (num > 0));
+ if (!skb) {
+ pr_err("%s: could not allocate memory for skb\n",
+ __func__);
+ return -ENOMEM;
+ }
+
+ trace_tsn_pre_tx(link, skb, data_size);
+ txq = skb_get_tx_queue(dev, skb);
+ if (!txq) {
+ pr_err("%s: Could not get tx_queue, dropping sending\n",
+ __func__);
+ kfree_skb(skb);
+ return -EINVAL;
+ }
+ res = netdev_start_xmit(skb, dev, txq, (num > 0));
+ if (res != NETDEV_TX_OK) {
+ pr_err("%s: Tx FAILED\n", __func__);
+ return res;
+ }
+ ts_pres_ns += ts_delta_ns;
+ }
+ return 0;
+}
--
2.7.4
Henrik Austad
2016-06-11 23:03:31 UTC
Permalink
From: Henrik Austad <***@cisco.com>

This needs refactoring and should be updated to use TRACE_CLASS, but for
now it provides a fair debug-window into TSN.

Cc: "David S. Miller" <***@davemloft.net>
Cc: Steven Rostedt <***@goodmis.org> (maintainer:TRACING)
Cc: Ingo Molnar <***@redhat.com> (maintainer:TRACING)
Signed-off-by: Henrik Austad <***@cisco.com>
---
include/trace/events/tsn.h | 349 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 349 insertions(+)
create mode 100644 include/trace/events/tsn.h

diff --git a/include/trace/events/tsn.h b/include/trace/events/tsn.h
new file mode 100644
index 0000000..ac1f31b
--- /dev/null
+++ b/include/trace/events/tsn.h
@@ -0,0 +1,349 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM tsn
+
+#if !defined(_TRACE_TSN_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_TSN_H
+
+#include <linux/tsn.h>
+#include <linux/tracepoint.h>
+
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+/* #include <linux/skbuff.h> */
+
+/* FIXME: update to TRACE_CLASS to reduce overhead */
+TRACE_EVENT(tsn_buffer_write,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_buffer_write_net,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+
+TRACE_EVENT(tsn_buffer_read,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_refill,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t reported_avail),
+
+ TP_ARGS(link, reported_avail),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(size_t, reported_left)
+ __field(size_t, low_water)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->reported_left = reported_avail;
+ __entry->low_water = link->low_water_mark;
+ ),
+
+ TP_printk("stream_id=%llu, buffer=%zd, avail=%zd, reported=%zd, low=%zd",
+ __entry->stream_id, __entry->bsize, __entry->size_left, __entry->reported_left, __entry->low_water)
+ );
+
+TRACE_EVENT(tsn_send_batch,
+
+ TP_PROTO(struct tsn_link *link,
+ int num_send,
+ u64 ts_base_ns,
+ u64 ts_delta_ns),
+
+ TP_ARGS(link, num_send, ts_base_ns, ts_delta_ns),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(int, seqnr)
+ __field(int, num_send)
+ __field(u64, ts_base_ns)
+ __field(u64, ts_delta_ns)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->seqnr = (int)link->last_seqnr;
+ __entry->ts_base_ns = ts_base_ns;
+ __entry->ts_delta_ns = ts_delta_ns;
+ __entry->num_send = num_send;
+ ),
+
+ TP_printk("stream_id=%llu, seqnr=%d, num_send=%d, ts_base_ns=%llu, ts_delta_ns=%llu",
+ __entry->stream_id, __entry->seqnr, __entry->num_send, __entry->ts_base_ns, __entry->ts_delta_ns)
+ );
+
+
+TRACE_EVENT(tsn_rx_handler,
+
+ TP_PROTO(struct tsn_link *link,
+ const struct ethhdr *ethhdr,
+ u64 sid),
+
+ TP_ARGS(link, ethhdr, sid),
+
+ TP_STRUCT__entry(
+ __field(char *, name)
+ __field(u16, proto)
+ __field(u64, sid)
+ __field(u64, link_sid)
+ ),
+ TP_fast_assign(
+ __entry->name = link->nic->name;
+ __entry->proto = ethhdr->h_proto;
+ __entry->sid = sid;
+ __entry->link_sid = link->stream_id;
+ ),
+
+ TP_printk("name=%s, proto: 0x%04x, stream_id=%llu, link->sid=%llu",
+ __entry->name, ntohs(__entry->proto), __entry->sid, __entry->link_sid)
+ );
+
+TRACE_EVENT(tsn_du,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, link_sid)
+ __field(size_t, bytes)
+ ),
+ TP_fast_assign(
+ __entry->link_sid = link->stream_id;
+ __entry->bytes = bytes;
+ ),
+
+ TP_printk("stream_id=%llu,bytes=%zu",
+ __entry->link_sid, __entry->bytes)
+);
+
+TRACE_EVENT(tsn_set_buffer,
+
+ TP_PROTO(struct tsn_link *link, size_t bufsize),
+
+ TP_ARGS(link, bufsize),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bufsize;
+ ),
+
+ TP_printk("stream_id=%llu,buffer_size=%zu",
+ __entry->stream_id, __entry->size)
+
+ );
+
+TRACE_EVENT(tsn_free_buffer,
+
+ TP_PROTO(struct tsn_link *link),
+
+ TP_ARGS(link),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bufsize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bufsize = link->buffer_size;
+ ),
+
+ TP_printk("stream_id=%llu,size:%zd",
+ __entry->stream_id, __entry->bufsize)
+
+ );
+
+TRACE_EVENT(tsn_buffer_drain,
+
+ TP_PROTO(struct tsn_link *link, size_t used),
+
+ TP_ARGS(link, used),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, used)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->used = used;
+ ),
+
+ TP_printk("stream_id=%llu,used=%zu",
+ __entry->stream_id, __entry->used)
+
+);
+/* TODO: too long, need cleanup.
+ */
+TRACE_EVENT(tsn_pre_tx,
+
+ TP_PROTO(struct tsn_link *link, struct sk_buff *skb, size_t bytes),
+
+ TP_ARGS(link, skb, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(u32, vlan_tag)
+ __field(size_t, bytes)
+ __field(size_t, data_len)
+ __field(unsigned int, headlen)
+ __field(u16, protocol)
+ __field(u16, prot_native)
+ __field(int, tx_idx)
+ __field(u16, mac_len)
+ __field(u16, hdr_len)
+ __field(u16, vlan_tci)
+ __field(u16, mac_header)
+ __field(unsigned int, tail)
+ __field(unsigned int, end)
+ __field(unsigned int, truesize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->vlan_tag = (skb_vlan_tag_present(skb) ? skb_vlan_tag_get(skb) : 0);
+ __entry->bytes = bytes;
+ __entry->data_len = skb->data_len;
+ __entry->headlen = skb_headlen(skb);
+ __entry->protocol = ntohs(vlan_get_protocol(skb));
+ __entry->prot_native = ntohs(skb->protocol);
+ __entry->tx_idx = skb_get_queue_mapping(skb);
+
+ __entry->mac_len = skb->mac_len;
+ __entry->hdr_len = skb->hdr_len;
+ __entry->vlan_tci = skb->vlan_tci;
+ __entry->mac_header = skb->mac_header;
+ __entry->tail = (unsigned int)skb->tail;
+ __entry->end = (unsigned int)skb->end;
+ __entry->truesize = skb->truesize;
+ ),
+
+ TP_printk("stream_id=%llu,vlan_tag=0x%04x,data_size=%zd,data_len=%zd,headlen=%u,proto=0x%04x (0x%04x),tx_idx=%d,mac_len=%u,hdr_len=%u,vlan_tci=0x%02x,mac_header=0x%02x,tail=%u,end=%u,truesize=%u",
+ __entry->stream_id,
+ __entry->vlan_tag,
+ __entry->bytes,
+ __entry->data_len,
+ __entry->headlen,
+ __entry->protocol,
+ __entry->prot_native, __entry->tx_idx,
+ __entry->mac_len,
+ __entry->hdr_len,
+ __entry->vlan_tci,
+ __entry->mac_header,
+ __entry->tail,
+ __entry->end,
+ __entry->truesize)
+ );
+
+#endif /* _TRACE_TSN_H || TRACE_HEADER_MULTI_READ */
+
+#include <trace/define_trace.h>
--
2.7.4
Steven Rostedt
2016-06-12 16:58:30 UTC
Permalink
On Sun, 12 Jun 2016 01:01:34 +0200
Post by Henrik Austad
This needs refactoring and should be updated to use TRACE_CLASS, but for
now it provides a fair debug-window into TSN.
---
include/trace/events/tsn.h | 349 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 349 insertions(+)
create mode 100644 include/trace/events/tsn.h
diff --git a/include/trace/events/tsn.h b/include/trace/events/tsn.h
new file mode 100644
index 0000000..ac1f31b
--- /dev/null
+++ b/include/trace/events/tsn.h
@@ -0,0 +1,349 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM tsn
+
+#if !defined(_TRACE_TSN_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_TSN_H
+
+#include <linux/tsn.h>
+#include <linux/tracepoint.h>
+
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+/* #include <linux/skbuff.h> */
+
+/* FIXME: update to TRACE_CLASS to reduce overhead */
I'm curious to why I didn't do this now. A class would make less
duplication of typing too ;-)
Post by Henrik Austad
+TRACE_EVENT(tsn_buffer_write,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
Move this logic into the print statement, since you save head and tail.
Post by Henrik Austad
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
__entry->stream_id, __entry->size, __entry->bsize,
(__entry->head - __entry->tail) % __entry->bsize,
Post by Henrik Austad
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_buffer_write_net,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+
+TRACE_EVENT(tsn_buffer_read,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_refill,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t reported_avail),
+
+ TP_ARGS(link, reported_avail),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(size_t, reported_left)
+ __field(size_t, low_water)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
As you don't save head and tail here, this logic needs to remain.
Post by Henrik Austad
+ __entry->reported_left = reported_avail;
+ __entry->low_water = link->low_water_mark;
+ ),
+
+ TP_printk("stream_id=%llu, buffer=%zd, avail=%zd, reported=%zd, low=%zd",
+ __entry->stream_id, __entry->bsize, __entry->size_left, __entry->reported_left, __entry->low_water)
+ );
+
+TRACE_EVENT(tsn_send_batch,
+
+ TP_PROTO(struct tsn_link *link,
+ int num_send,
+ u64 ts_base_ns,
+ u64 ts_delta_ns),
+
+ TP_ARGS(link, num_send, ts_base_ns, ts_delta_ns),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(int, seqnr)
+ __field(int, num_send)
+ __field(u64, ts_base_ns)
+ __field(u64, ts_delta_ns)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->seqnr = (int)link->last_seqnr;
+ __entry->ts_base_ns = ts_base_ns;
+ __entry->ts_delta_ns = ts_delta_ns;
+ __entry->num_send = num_send;
+ ),
+
+ TP_printk("stream_id=%llu, seqnr=%d, num_send=%d, ts_base_ns=%llu, ts_delta_ns=%llu",
+ __entry->stream_id, __entry->seqnr, __entry->num_send, __entry->ts_base_ns, __entry->ts_delta_ns)
+ );
+
+
+TRACE_EVENT(tsn_rx_handler,
+
+ TP_PROTO(struct tsn_link *link,
+ const struct ethhdr *ethhdr,
+ u64 sid),
+
+ TP_ARGS(link, ethhdr, sid),
+
+ TP_STRUCT__entry(
+ __field(char *, name)
+ __field(u16, proto)
+ __field(u64, sid)
+ __field(u64, link_sid)
+ ),
+ TP_fast_assign(
+ __entry->name = link->nic->name;
+ __entry->proto = ethhdr->h_proto;
+ __entry->sid = sid;
+ __entry->link_sid = link->stream_id;
+ ),
+
+ TP_printk("name=%s, proto: 0x%04x, stream_id=%llu, link->sid=%llu",
+ __entry->name, ntohs(__entry->proto), __entry->sid, __entry->link_sid)
+ );
+
+TRACE_EVENT(tsn_du,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, link_sid)
+ __field(size_t, bytes)
+ ),
+ TP_fast_assign(
+ __entry->link_sid = link->stream_id;
+ __entry->bytes = bytes;
+ ),
+
+ TP_printk("stream_id=%llu,bytes=%zu",
+ __entry->link_sid, __entry->bytes)
+);
+
+TRACE_EVENT(tsn_set_buffer,
+
+ TP_PROTO(struct tsn_link *link, size_t bufsize),
+
+ TP_ARGS(link, bufsize),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bufsize;
+ ),
+
+ TP_printk("stream_id=%llu,buffer_size=%zu",
+ __entry->stream_id, __entry->size)
+
+ );
+
+TRACE_EVENT(tsn_free_buffer,
+
+ TP_PROTO(struct tsn_link *link),
+
+ TP_ARGS(link),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bufsize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bufsize = link->buffer_size;
+ ),
+
+ TP_printk("stream_id=%llu,size:%zd",
+ __entry->stream_id, __entry->bufsize)
+
+ );
+
+TRACE_EVENT(tsn_buffer_drain,
+
+ TP_PROTO(struct tsn_link *link, size_t used),
+
+ TP_ARGS(link, used),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, used)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->used = used;
+ ),
+
+ TP_printk("stream_id=%llu,used=%zu",
+ __entry->stream_id, __entry->used)
+
+);
+/* TODO: too long, need cleanup.
+ */
+TRACE_EVENT(tsn_pre_tx,
+
+ TP_PROTO(struct tsn_link *link, struct sk_buff *skb, size_t bytes),
+
+ TP_ARGS(link, skb, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(u32, vlan_tag)
+ __field(size_t, bytes)
+ __field(size_t, data_len)
+ __field(unsigned int, headlen)
+ __field(u16, protocol)
+ __field(u16, prot_native)
+ __field(int, tx_idx)
+ __field(u16, mac_len)
+ __field(u16, hdr_len)
+ __field(u16, vlan_tci)
+ __field(u16, mac_header)
+ __field(unsigned int, tail)
+ __field(unsigned int, end)
+ __field(unsigned int, truesize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->vlan_tag = (skb_vlan_tag_present(skb) ? skb_vlan_tag_get(skb) : 0);
+ __entry->bytes = bytes;
+ __entry->data_len = skb->data_len;
+ __entry->headlen = skb_headlen(skb);
+ __entry->protocol = ntohs(vlan_get_protocol(skb));
Maybe it would be better to do the ntohs() in the TP_printk() as well.
Post by Henrik Austad
+ __entry->prot_native = ntohs(skb->protocol);
here too.
Post by Henrik Austad
+ __entry->tx_idx = skb_get_queue_mapping(skb);
+
+ __entry->mac_len = skb->mac_len;
+ __entry->hdr_len = skb->hdr_len;
+ __entry->vlan_tci = skb->vlan_tci;
+ __entry->mac_header = skb->mac_header;
+ __entry->tail = (unsigned int)skb->tail;
+ __entry->end = (unsigned int)skb->end;
+ __entry->truesize = skb->truesize;
+ ),
+
+ TP_printk("stream_id=%llu,vlan_tag=0x%04x,data_size=%zd,data_len=%zd,headlen=%u,proto=0x%04x (0x%04x),tx_idx=%d,mac_len=%u,hdr_len=%u,vlan_tci=0x%02x,mac_header=0x%02x,tail=%u,end=%u,truesize=%u",
+ __entry->stream_id,
+ __entry->vlan_tag,
+ __entry->bytes,
+ __entry->data_len,
+ __entry->headlen,
+ __entry->protocol,
+ __entry->prot_native, __entry->tx_idx,
+ __entry->mac_len,
+ __entry->hdr_len,
+ __entry->vlan_tci,
+ __entry->mac_header,
Is this an ether mac header? If so we support %M. But as it's defined
as only u16, it doesn't seem like it can be.

-- Steve
Post by Henrik Austad
+ __entry->tail,
+ __entry->end,
+ __entry->truesize)
+ );
+
+#endif /* _TRACE_TSN_H || TRACE_HEADER_MULTI_READ */
+
+#include <trace/define_trace.h>
Henrik Austad
2016-06-12 21:25:29 UTC
Permalink
Post by Steven Rostedt
On Sun, 12 Jun 2016 01:01:34 +0200
Post by Henrik Austad
This needs refactoring and should be updated to use TRACE_CLASS, but for
now it provides a fair debug-window into TSN.
---
include/trace/events/tsn.h | 349 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 349 insertions(+)
create mode 100644 include/trace/events/tsn.h
diff --git a/include/trace/events/tsn.h b/include/trace/events/tsn.h
new file mode 100644
index 0000000..ac1f31b
--- /dev/null
+++ b/include/trace/events/tsn.h
@@ -0,0 +1,349 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM tsn
+
+#if !defined(_TRACE_TSN_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_TSN_H
+
+#include <linux/tsn.h>
+#include <linux/tracepoint.h>
+
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+/* #include <linux/skbuff.h> */
+
+/* FIXME: update to TRACE_CLASS to reduce overhead */
I'm curious to why I didn't do this now. A class would make less
duplication of typing too ;-)
Yeah, I found this in a really great article written by some tracing-dude,
I hear he talks really, really fast!

https://lwn.net/Articles/381064/
Post by Steven Rostedt
Post by Henrik Austad
+TRACE_EVENT(tsn_buffer_write,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
Move this logic into the print statement, since you save head and tail.
Ok, any particular reason?
Post by Steven Rostedt
Post by Henrik Austad
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
__entry->stream_id, __entry->size, __entry->bsize,
(__entry->head - __entry->tail) % __entry->bsize,
Ok, so is this about saving space by dropping one intermediate value, or is
it some other point I'm missing here?
Post by Steven Rostedt
Post by Henrik Austad
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_buffer_write_net,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+
+TRACE_EVENT(tsn_buffer_read,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+TRACE_EVENT(tsn_refill,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t reported_avail),
+
+ TP_ARGS(link, reported_avail),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(size_t, reported_left)
+ __field(size_t, low_water)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
As you don't save head and tail here, this logic needs to remain.
Post by Henrik Austad
+ __entry->reported_left = reported_avail;
+ __entry->low_water = link->low_water_mark;
+ ),
+
+ TP_printk("stream_id=%llu, buffer=%zd, avail=%zd, reported=%zd, low=%zd",
+ __entry->stream_id, __entry->bsize, __entry->size_left, __entry->reported_left, __entry->low_water)
+ );
+
+TRACE_EVENT(tsn_send_batch,
+
+ TP_PROTO(struct tsn_link *link,
+ int num_send,
+ u64 ts_base_ns,
+ u64 ts_delta_ns),
+
+ TP_ARGS(link, num_send, ts_base_ns, ts_delta_ns),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(int, seqnr)
+ __field(int, num_send)
+ __field(u64, ts_base_ns)
+ __field(u64, ts_delta_ns)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->seqnr = (int)link->last_seqnr;
+ __entry->ts_base_ns = ts_base_ns;
+ __entry->ts_delta_ns = ts_delta_ns;
+ __entry->num_send = num_send;
+ ),
+
+ TP_printk("stream_id=%llu, seqnr=%d, num_send=%d, ts_base_ns=%llu, ts_delta_ns=%llu",
+ __entry->stream_id, __entry->seqnr, __entry->num_send, __entry->ts_base_ns, __entry->ts_delta_ns)
+ );
+
+
+TRACE_EVENT(tsn_rx_handler,
+
+ TP_PROTO(struct tsn_link *link,
+ const struct ethhdr *ethhdr,
+ u64 sid),
+
+ TP_ARGS(link, ethhdr, sid),
+
+ TP_STRUCT__entry(
+ __field(char *, name)
+ __field(u16, proto)
+ __field(u64, sid)
+ __field(u64, link_sid)
+ ),
+ TP_fast_assign(
+ __entry->name = link->nic->name;
+ __entry->proto = ethhdr->h_proto;
+ __entry->sid = sid;
+ __entry->link_sid = link->stream_id;
+ ),
+
+ TP_printk("name=%s, proto: 0x%04x, stream_id=%llu, link->sid=%llu",
+ __entry->name, ntohs(__entry->proto), __entry->sid, __entry->link_sid)
+ );
+
+TRACE_EVENT(tsn_du,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, link_sid)
+ __field(size_t, bytes)
+ ),
+ TP_fast_assign(
+ __entry->link_sid = link->stream_id;
+ __entry->bytes = bytes;
+ ),
+
+ TP_printk("stream_id=%llu,bytes=%zu",
+ __entry->link_sid, __entry->bytes)
+);
+
+TRACE_EVENT(tsn_set_buffer,
+
+ TP_PROTO(struct tsn_link *link, size_t bufsize),
+
+ TP_ARGS(link, bufsize),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bufsize;
+ ),
+
+ TP_printk("stream_id=%llu,buffer_size=%zu",
+ __entry->stream_id, __entry->size)
+
+ );
+
+TRACE_EVENT(tsn_free_buffer,
+
+ TP_PROTO(struct tsn_link *link),
+
+ TP_ARGS(link),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, bufsize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->bufsize = link->buffer_size;
+ ),
+
+ TP_printk("stream_id=%llu,size:%zd",
+ __entry->stream_id, __entry->bufsize)
+
+ );
+
+TRACE_EVENT(tsn_buffer_drain,
+
+ TP_PROTO(struct tsn_link *link, size_t used),
+
+ TP_ARGS(link, used),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, used)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->used = used;
+ ),
+
+ TP_printk("stream_id=%llu,used=%zu",
+ __entry->stream_id, __entry->used)
+
+);
+/* TODO: too long, need cleanup.
+ */
+TRACE_EVENT(tsn_pre_tx,
+
+ TP_PROTO(struct tsn_link *link, struct sk_buff *skb, size_t bytes),
+
+ TP_ARGS(link, skb, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(u32, vlan_tag)
+ __field(size_t, bytes)
+ __field(size_t, data_len)
+ __field(unsigned int, headlen)
+ __field(u16, protocol)
+ __field(u16, prot_native)
+ __field(int, tx_idx)
+ __field(u16, mac_len)
+ __field(u16, hdr_len)
+ __field(u16, vlan_tci)
+ __field(u16, mac_header)
+ __field(unsigned int, tail)
+ __field(unsigned int, end)
+ __field(unsigned int, truesize)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->vlan_tag = (skb_vlan_tag_present(skb) ? skb_vlan_tag_get(skb) : 0);
+ __entry->bytes = bytes;
+ __entry->data_len = skb->data_len;
+ __entry->headlen = skb_headlen(skb);
+ __entry->protocol = ntohs(vlan_get_protocol(skb));
Maybe it would be better to do the ntohs() in the TP_printk() as well.
Post by Henrik Austad
+ __entry->prot_native = ntohs(skb->protocol);
here too.
Post by Henrik Austad
+ __entry->tx_idx = skb_get_queue_mapping(skb);
+
+ __entry->mac_len = skb->mac_len;
+ __entry->hdr_len = skb->hdr_len;
+ __entry->vlan_tci = skb->vlan_tci;
+ __entry->mac_header = skb->mac_header;
+ __entry->tail = (unsigned int)skb->tail;
+ __entry->end = (unsigned int)skb->end;
+ __entry->truesize = skb->truesize;
+ ),
+
+ TP_printk("stream_id=%llu,vlan_tag=0x%04x,data_size=%zd,data_len=%zd,headlen=%u,proto=0x%04x (0x%04x),tx_idx=%d,mac_len=%u,hdr_len=%u,vlan_tci=0x%02x,mac_header=0x%02x,tail=%u,end=%u,truesize=%u",
+ __entry->stream_id,
+ __entry->vlan_tag,
+ __entry->bytes,
+ __entry->data_len,
+ __entry->headlen,
+ __entry->protocol,
+ __entry->prot_native, __entry->tx_idx,
+ __entry->mac_len,
+ __entry->hdr_len,
+ __entry->vlan_tci,
+ __entry->mac_header,
Is this an ether mac header? If so we support %M. But as it's defined
as only u16, it doesn't seem like it can be.
Actually, looking at the output, I'm not quite sure what it is that I
wanted to grab with that, the skb->mac_header should give an offset into
the header-area of skb, so it should be a constant offset from skb->head
(that is an actual pointer).

I *think* I wanted to make sure I updated things correctly so that the
offset didn't suddenly change, but the fact that I'm no longer sure
indicates that I should just drop that one. That whole printout is too long
anyway..

Thanks for pointing a finger at this!


I'm still a bit stymied as to why logic should be in TP_printk() and not
TP_fast_assign(). Not that I really have any preferences either way, just
curious.
--
Henrik Austad
Steven Rostedt
2016-06-13 02:22:31 UTC
Permalink
On Sun, 12 Jun 2016 23:25:10 +0200
Post by Henrik Austad
Post by Steven Rostedt
Post by Henrik Austad
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+/* #include <linux/skbuff.h> */
+
+/* FIXME: update to TRACE_CLASS to reduce overhead */
I'm curious to why I didn't do this now. A class would make less
duplication of typing too ;-)
Yeah, I found this in a really great article written by some tracing-dude,
I hear he talks really, really fast!
I plead the 5th!
Post by Henrik Austad
https://lwn.net/Articles/381064/
Post by Steven Rostedt
Post by Henrik Austad
+TRACE_EVENT(tsn_buffer_write,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
Move this logic into the print statement, since you save head and tail.
Ok, any particular reason?
Because it removes calculations during the trace. The calculations done
in TP_printk() are done at the time of reading the trace, and
calculations done in TP_fast_assign() are done during the recording and
hence adding more overhead to the trace itself.
Post by Henrik Austad
Post by Steven Rostedt
Post by Henrik Austad
+ __entry->buffer = link->buffer;
+ __entry->head = link->head;
+ __entry->tail = link->tail;
+ __entry->end = link->end;
+ ),
+
+ TP_printk("stream_id=%llu, copy=%zd, buffer: %zd, avail=%zd, [buffer=%p, head=%p, tail=%p, end=%p]",
+ __entry->stream_id, __entry->size, __entry->bsize, __entry->size_left,
__entry->stream_id, __entry->size, __entry->bsize,
(__entry->head - __entry->tail) % __entry->bsize,
Ok, so is this about saving space by dropping one intermediate value, or is
it some other point I'm missing here?
Nope, just moving the overhead from the recording of the trace to the
reading of the trace.
Post by Henrik Austad
Post by Steven Rostedt
Post by Henrik Austad
+ __entry->buffer, __entry->head, __entry->tail, __entry->end)
+
+ );
+
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->vlan_tag = (skb_vlan_tag_present(skb) ? skb_vlan_tag_get(skb) : 0);
+ __entry->bytes = bytes;
+ __entry->data_len = skb->data_len;
+ __entry->headlen = skb_headlen(skb);
+ __entry->protocol = ntohs(vlan_get_protocol(skb));
Maybe it would be better to do the ntohs() in the TP_printk() as well.
Post by Henrik Austad
+ __entry->prot_native = ntohs(skb->protocol);
here too.
Post by Henrik Austad
+ __entry->tx_idx = skb_get_queue_mapping(skb);
+
+ __entry->mac_len = skb->mac_len;
+ __entry->hdr_len = skb->hdr_len;
+ __entry->vlan_tci = skb->vlan_tci;
+ __entry->mac_header = skb->mac_header;
+ __entry->tail = (unsigned int)skb->tail;
+ __entry->end = (unsigned int)skb->end;
+ __entry->truesize = skb->truesize;
+ ),
+
+ TP_printk("stream_id=%llu,vlan_tag=0x%04x,data_size=%zd,data_len=%zd,headlen=%u,proto=0x%04x (0x%04x),tx_idx=%d,mac_len=%u,hdr_len=%u,vlan_tci=0x%02x,mac_header=0x%02x,tail=%u,end=%u,truesize=%u",
+ __entry->stream_id,
+ __entry->vlan_tag,
+ __entry->bytes,
+ __entry->data_len,
+ __entry->headlen,
+ __entry->protocol,
+ __entry->prot_native, __entry->tx_idx,
+ __entry->mac_len,
+ __entry->hdr_len,
+ __entry->vlan_tci,
+ __entry->mac_header,
Is this an ether mac header? If so we support %M. But as it's defined
as only u16, it doesn't seem like it can be.
Actually, looking at the output, I'm not quite sure what it is that I
wanted to grab with that, the skb->mac_header should give an offset into
the header-area of skb, so it should be a constant offset from skb->head
(that is an actual pointer).
I *think* I wanted to make sure I updated things correctly so that the
offset didn't suddenly change, but the fact that I'm no longer sure
indicates that I should just drop that one. That whole printout is too long
anyway..
Thanks for pointing a finger at this!
I'm still a bit stymied as to why logic should be in TP_printk() and not
TP_fast_assign(). Not that I really have any preferences either way, just
curious.
As said above, it's simply just trying to make tracing have less of an
effect on what is being traced. The more you can do in the post
transactions the faster the trace becomes and less invasive the trace
is on the system performance.

-- Steve
Henrik Austad
2016-06-13 07:21:04 UTC
Permalink
Post by Steven Rostedt
On Sun, 12 Jun 2016 23:25:10 +0200
Post by Henrik Austad
Post by Steven Rostedt
Post by Henrik Austad
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+/* #include <linux/skbuff.h> */
+
+/* FIXME: update to TRACE_CLASS to reduce overhead */
I'm curious to why I didn't do this now. A class would make less
duplication of typing too ;-)
Yeah, I found this in a really great article written by some tracing-dude,
I hear he talks really, really fast!
I plead the 5th!
Post by Henrik Austad
https://lwn.net/Articles/381064/
Post by Steven Rostedt
Post by Henrik Austad
+TRACE_EVENT(tsn_buffer_write,
+
+ TP_PROTO(struct tsn_link *link,
+ size_t bytes),
+
+ TP_ARGS(link, bytes),
+
+ TP_STRUCT__entry(
+ __field(u64, stream_id)
+ __field(size_t, size)
+ __field(size_t, bsize)
+ __field(size_t, size_left)
+ __field(void *, buffer)
+ __field(void *, head)
+ __field(void *, tail)
+ __field(void *, end)
+ ),
+
+ TP_fast_assign(
+ __entry->stream_id = link->stream_id;
+ __entry->size = bytes;
+ __entry->bsize = link->used_buffer_size;
+ __entry->size_left = (link->head - link->tail) % link->used_buffer_size;
Move this logic into the print statement, since you save head and tail.
Ok, any particular reason?
Because it removes calculations during the trace. The calculations done
in TP_printk() are done at the time of reading the trace, and
calculations done in TP_fast_assign() are done during the recording and
hence adding more overhead to the trace itself.
Aha! that makes sense, thanks!
(/me goes and updates the tracing-part)

-Henrik
Henrik Austad
2016-06-11 23:04:02 UTC
Permalink
From: Henrik Austad <***@cisco.com>

This defines the general TSN headers for network packets, the
shim-interface and the central 'tsn_list' structure.

Cc: "David S. Miller" <***@davemloft.net>
Signed-off-by: Henrik Austad <***@cisco.com>
---
include/linux/tsn.h | 806 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 806 insertions(+)
create mode 100644 include/linux/tsn.h

diff --git a/include/linux/tsn.h b/include/linux/tsn.h
new file mode 100644
index 0000000..0e1f732b
--- /dev/null
+++ b/include/linux/tsn.h
@@ -0,0 +1,806 @@
+/* TSN - Time Sensitive Networking
+ *
+ * Copyright (C) 2016- Henrik Austad <***@cisco.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef _TSN_H
+#define _TSN_H
+#include <linux/list.h>
+#include <linux/configfs.h>
+#include <linux/hrtimer.h>
+
+/* The naming here can be a bit confusing as we call it TSN but naming
+ * suggests 'AVB'. Reason: IEE 1722 was written before the working group
+ * was renamed to Time Sensitive Networking.
+ *
+ * To be precise. TSN describes the protocol for shipping data, AVB is a
+ * medialayer which you can build on top of TSN.
+ *
+ * For this reason the frames are given avb-names whereas the functions
+ * use tsn_-naming.
+ */
+
+/* 7 bit value 0x00 - 0x7F */
+enum avtp_subtype {
+ AVTP_61883_IIDC = 0,
+ AVTP_MMA = 0x1,
+ AVTP_MAAP = 0x7e,
+ AVTP_EXPERIMENTAL = 0x7f,
+};
+
+/* NOTE NOTE NOTE !!
+ * The headers below use bitfields extensively and verifications
+ * are needed when using little-endian vs big-endian systems.
+ */
+
+/* Common part of avtph header
+ *
+ * AVB Transport Protocol Common Header
+ *
+ * Defined in 1722-2011 Sec. 5.2
+ */
+struct avtp_ch {
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ /* use avtp_subtype enum.
+ */
+ u8 subtype:7;
+
+ /* Controlframe: 1
+ * Dataframe : 0
+ */
+ u8 cd:1;
+
+ /* Type specific data, part 1 */
+ u8 tsd_1:4;
+
+ /* In current version of AVB, only 0 is valid, all other values
+ * are reserved for future versions.
+ */
+ u8 version:3;
+
+ /* Valid StreamID in frame
+ *
+ * ControlData not related to a specific stream should clear
+ * this (and have stream_id = 0), _all_ other values should set
+ * this to 1.
+ */
+ u8 sv:1;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+ u8 cd:1;
+ u8 subtype:7;
+ u8 sv:1;
+ u8 version:3;
+ u8 tsd_1:4;
+#else
+#error "Unknown Endianness, cannot determine bitfield ordering"
+#endif
+ /* Type specific data (adjacent to tsd_1, but split due to bitfield) */
+ u16 tsd_2;
+ u64 stream_id;
+
+ /*
+ * payload by subtype
+ */
+ u8 pbs[0];
+} __packed;
+
+/* AVTPDU Common Control header format
+ * IEEE 1722#5.3
+ */
+struct avtpc_header {
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 subtype:7;
+ u8 cd:1;
+ u8 control_data:4;
+ u8 version:3;
+ u8 sv:1;
+ u16 control_data_length:11;
+ u16 status:5;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+ u8 cd:1;
+ u8 subtype:7;
+ u8 sv:1;
+ u8 version:3;
+ u8 control_data:4;
+ u16 status:5;
+ u16 control_data_length:11;
+#else
+#error "Unknown Endianness, cannot determine bitfield ordering"
+#endif
+ u64 stream_id;
+} __packed;
+
+/* AVTP common stream data AVTPDU header format
+ * IEEE 1722#5.4
+ */
+struct avtpdu_header {
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 subtype:7;
+ u8 cd:1;
+
+ /* avtp_timestamp valid */
+ u8 tv: 1;
+
+ /* gateway_info valid */
+ u8 gv:1;
+
+ /* reserved */
+ u8 r:1;
+
+ /*
+ * Media clock Restart toggle
+ */
+ u8 mr:1;
+
+ u8 version:3;
+
+ /* StreamID valid */
+ u8 sv:1;
+ u8 seqnr;
+
+ /* Timestamp uncertain */
+ u8 tu:1;
+ u8 r2:7;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+ u8 cd:1;
+ u8 subtype:7;
+
+ u8 sv:1;
+ u8 version:3;
+ u8 mr:1;
+ u8 r:1;
+ u8 gv:1;
+ u8 tv: 1;
+
+ u8 seqnr;
+ u8 r2:7;
+ u8 tu:1;
+#else
+#error "Unknown Endianness, cannot determine bitfield ordering"
+#endif
+
+ u64 stream_id;
+
+ u32 avtp_timestamp;
+ u32 gateway_info;
+
+ /* Stream Data Length */
+ u16 sd_len;
+
+ /* Protocol specific header, derived from avtp_subtype */
+ u16 psh;
+
+ /* Stream Payload Data 0 to n octets
+ * n so that total size < MTU
+ */
+ u8 data[0];
+} __packed;
+
+
+/**
+ * struct tsn_list - The top level container of TSN
+ *
+ * This is what tsn_configfs refers to as 'tier-0'
+ *
+ * @head List of TSN cards
+ * @lock lock protecting global entries
+ * @tsn_subsys Ref to ConfigFS subsystem
+ *
+ * @running: hrtimer is running driving data out
+ * @tsn_timer: hrtimer container
+ * @num_avail Number of available TSN NICs exposed through ConfigFS
+ */
+struct tsn_list {
+ struct list_head head;
+ struct mutex lock;
+ struct configfs_subsystem tsn_subsys;
+
+ /*
+ * TSN-timer is running. Not to be confused with the per-link
+ * disabled flag which indicates if a remote client, like aplay,
+ * is pushing data to it.
+ */
+ atomic_t running;
+ struct hrtimer tsn_timer;
+ unsigned int period_ns;
+
+
+ size_t num_avail;
+};
+
+/**
+ * struct tsn_nic
+ *
+ * Individual TSN-capable NICs, or 'tier-1' struct
+ *
+ * @list linked list of all TSN NICs
+ * @group configfs group
+ * @dev corresponding net_device
+ * @dma_size : size of the DMA buffer
+ * @dma_handle: housekeeping DMA-stuff
+ * @dma_mem : pointer to memory region we're using for DMAing to the NIC
+ * @name Name of NIC (same as name in dev), TO BE REMOVED
+ * @txq Size of Tx-queue. TO BE REMOVED
+ * @rx_registered flag indicating if a handler is registered for the nic
+ * @capable: if the NIC is capable for proper TSN traffic or if it must
+ * be emulated in software.
+ *
+ */
+struct tsn_nic {
+ struct list_head list;
+ struct config_group group;
+ struct net_device *dev;
+ struct tsn_list *tsn_list;
+
+ size_t dma_size;
+ dma_addr_t dma_handle;
+ void *dma_mem;
+
+ char *name;
+ int txq;
+ u8 rx_registered:1;
+ u8 capable:1;
+ u8 reserved:6;
+};
+
+struct tsn_shim_ops;
+/**
+ * tsn_link - Structure describing a single TSN link
+ *
+ */
+struct tsn_link {
+ /*
+ * Lock for protecting the buffer
+ */
+ spinlock_t lock;
+
+ struct config_group group;
+ struct tsn_nic *nic;
+ struct hlist_node node;
+
+ /* The link itself is active, and the tsn_core will treat it as
+ * an active participant and feed data from it to the
+ * network. This places some restrictions on which attributes
+ * can be changed.
+ *
+ * 1: active
+ * 0: inactive
+ */
+ atomic_t active;
+
+ u64 timer_period_ns;
+
+ /* Pointer to media-specific data.
+ * e.g. struct avb_chip
+ */
+ void *media_chip;
+
+ u64 stream_id;
+
+ /*
+ * The max required size for a _single_ TSN frame.
+ *
+ * To be used instead of channels and sample_freq.
+ */
+ u16 max_payload_size;
+ u16 shim_header_size;
+
+ /*
+ * Size of buffer (in bytes) to use when handling data to/from
+ * NIC.
+ *
+ * Smaller size will result in client being called more often
+ * but also provides lower latencies.
+ */
+ size_t buffer_size;
+ size_t used_buffer_size;
+
+ /*
+ * Used when frames are constructed and shipped to the network
+ * layer. If this is true, 0-frames will be sent insted of data
+ * from the buffer.
+ */
+ atomic_t buffer_active;
+
+ /*
+ * ringbuffer for incoming or outging traffic
+ * +-----------------------------------+
+ * | ########## |
+ * +-----------------------------------+
+ * ^ ^ ^ ^
+ * buffer tail head end
+ *
+ * Buffer: start of memory area
+ * tail: first byte of data in buffer
+ * head: first unused slot in which to store new data
+ *
+ * head,tail is used to represent the position of 'live data' in
+ * the buffer.
+ */
+ void *buffer;
+ void *head;
+ void *tail;
+ void *end;
+
+ /* Number of bytes to run refill/drain callbacks */
+ size_t low_water_mark;
+ size_t high_water_mark;
+
+
+ /*
+ * callback ops.
+ */
+ struct tsn_shim_ops *ops;
+
+ /*
+ * EndStation Type
+ *
+ * Either Talker or Listener
+ *
+ * 1: We are *Talker*, i.e. producing data to send
+ * 0: We are *Listener*, i.e. we receive data from another ES.
+ *
+ * This is for a single link, so even though an end-station can
+ * be both Talker *and* Listener, a link can only be one.
+ */
+ u8 estype_talker;
+
+ /*
+ * Link will use buffer managed by the shim. For this to work,
+ * the shim must:
+ *
+ * - call tsn_use_external_buffer(link, size);
+ * - provide tsn_shim_buffer_swap(link) in tsn_shim_ops
+ */
+ u8 external_buffer;
+
+ u8 last_seqnr;
+
+ /*
+ * Class can be either A or B
+ *
+ * ClassA: every 125us
+ * ClassB: every 250us
+ *
+ * This will also affect how large each frame will be.
+ */
+ u8 class_a:1;
+
+ /*
+ * Any AVTP data stream must set the 802.1Q vlan id and priority
+ * Code point. This should be obtained from MSRP, default values
+ * are:
+ *
+ * pvid: SR_PVID 2
+ * pcp: Class A: 3
+ * Class B: 2
+ *
+ * See IEEE 802.1Q-2011, Sec 35.2.2.9.3 and table 6-6 in 6.6.2
+ * for details
+ */
+ u8 pcp_a:3;
+ u8 pcp_b:3;
+ u16 vlan_id:12;
+
+ u8 remote_mac[6];
+};
+
+/**
+ * tsn_link_on - make link active
+ *
+ * This cause most of the attributes to be treated read-only since we
+ * will have to re-negotiate with the network if most of these
+ * parameters change.
+ *
+ * Note: this means that the link will be handled by the rx-handler or
+ * the timer callback, but until the link_buffer is set active (via
+ * tsn_lb_on()), actual data is not moved.
+ *
+ * @link: link being set to active
+ */
+static inline void tsn_link_on(struct tsn_link *link)
+{
+ if (link)
+ atomic_set(&link->active, 1);
+}
+
+/**
+ * tsn_link_off - make link inactive
+ *
+ * The link will now be ignored by timer callback or the
+ * rx-handler. Attributes can be mostly freely changed (we assume that
+ * userspace sets values that are negotiated properly).
+ *
+ * @link: link to deactivate
+ */
+static inline void tsn_link_off(struct tsn_link *link)
+{
+ if (link)
+ atomic_set(&link->active, 0);
+}
+
+/**
+ * tsn_link_is_on - query link to see if it is active
+ *
+ * Mostly used by tsn_configfs to respect the "read-only" once link is
+ * configured and made active.
+ *
+ * @link active link
+ * @returns 1 if active/on, 0 otherwise
+ */
+static inline int tsn_link_is_on(struct tsn_link *link)
+{
+ if (link)
+ return atomic_read(&link->active);
+ return 0;
+}
+
+/**
+ * tsn_set_buffer_size - adjust buffersize to match a shim
+ *
+ * This will not allocate (or deallcoate) memory, just adjust how much
+ * of the buffer allocated in tsn_prepare_link is being used. tsn_
+ * expects tsn_clear_buffer_size() to be invoked when stream is closed.
+ */
+int tsn_set_buffer_size(struct tsn_link *link, size_t bsize);
+int tsn_clear_buffer_size(struct tsn_link *link);
+
+/**
+ * tsn_buffer_write write data into the buffer from shim
+ *
+ * This is called from the shim-driver when more data is available and
+ * data needs to be pushed out to the network.
+ *
+ * NOTE: This is used when TSN handles the databuffer. This will not be
+ * needed for "shim-hosted" buffers.
+ *
+ * _If_ this function is called when the link is inactive, it will
+ * _enable_ the link (i.e. link will mark the buffer as 'active'). Do
+ * not copy data into the buffer unless you are ready to start sending
+ * frames!
+ *
+ * @link active link
+ * @src the buffer to copy data from
+ * @bytes bytes to copy
+ * @return bytes copied from link->buffer or negative error
+ */
+int tsn_buffer_write(struct tsn_link *link, void *src, size_t bytes);
+
+
+/**
+ * tsn_buffer_read - read data from link->buffer and give to shim
+ *
+ * When we act as a listener, this is what the shim (should|will) call
+ * to grab data. It typically grabs much more data than the _net
+ * equivalent. It also do not trigger a refill-event the same way
+ * buffer_read_net does.
+ *
+ * @param link current link that holds the buffer
+ * @param buffer the buffer to copy into, must be at least of size bytes
+ * @param bytes number of bytes.
+ *
+ * Note that this routine does NOT CARE about channels, samplesize etc,
+ * it is a _pure_ copy that handles ringbuffer wraps etc.
+ *
+ * This function have side-effects as it will update internal tsn_link
+ * values.
+ *
+ * @return Bytes copied into link->buffer, negative value upon error.
+ */
+int tsn_buffer_read(struct tsn_link *link, void *buffer, size_t bytes);
+
+/**
+ * tsn_lb_enable - TSN Link Buffer Enable
+ *
+ * Mark the link as "buffer-enabled" which will let the core start
+ * shifting data in/out of the buffer instead of ignoring incoming
+ * frames or sending "nullframes".
+ *
+ * This is for the network-end of the tsn-buffer, i.e.
+ * - when enabled frames *from* the network will be inserted into the buffer,
+ * - or frames going *out* will include data from the buffer instead of sending
+ * null-frames.
+ *
+ * When disabled, data will be zero'd, e.g Tx will send NULL-frames and
+ * Rx will silently drop the frames.
+ *
+ * @link: active link
+ */
+static inline void tsn_lb_enable(struct tsn_link *link)
+{
+ if (link)
+ atomic_set(&link->buffer_active, 1);
+}
+
+/**
+ * tsn_lb_disable - stop using the buffer for the net-side of TSN
+ *
+ * When we close a stream, we do not necessarily tear down the link, and
+ * we need to handle the data in some way.
+ */
+static inline void tsn_lb_disable(struct tsn_link *link)
+{
+ if (link)
+ atomic_set(&link->buffer_active, 0);
+}
+
+/**
+ * tsn_lb() - query if we have disabled pushing of data to/from link-buffer
+ *
+ * @param struct tsn_link *link - active link
+ * @returns 1 if link is enabled
+ */
+static inline int tsn_lb(struct tsn_link *link)
+{
+ if (link)
+ return atomic_read(&link->buffer_active);
+
+ /* if link is NULL; buffer not active */
+ return 0;
+}
+
+
+/**
+ * Shim ops - what tsn_core use when calling back into the shim. All ops
+ * must be reentrant.
+ */
+#define SHIM_NAME_SIZE 32
+struct tsn_shim_ops {
+
+ /* internal linked list used by tsn_core to keep track of all
+ * shims.
+ */
+ struct list_head head;
+
+ /**
+ * name - a unique name identifying this shim
+ *
+ * This is what userspace use to indicate to core what SHIM a
+ * particular link will use. If the name is already present,
+ * core will reject this name.
+ */
+ char shim_name[SHIM_NAME_SIZE];
+
+ /**
+ * probe - callback when a new link of this type is instantiated.
+ *
+ * When a new link is brought online, this is called once the
+ * essential parts of tsn_core has finiesh. Once probe_cb has
+ * finisehd, the shim _must_ be ready to accept data to/from
+ * tsn_core. On the other hand, due to the final steps of setup,
+ * it cannot expect to be called into action immediately after
+ * probe has finished.
+ *
+ * In other words, shim must be ready, but core doesn't have to
+ *
+ * @param : a particular link to pass along to the probe-function.
+ */
+ int (*probe)(struct tsn_link *link);
+
+ /**
+ * buffer_swap - set a new buffer for the link. [OPTIONAL]
+ *
+ * Used when external buffering is enabled.
+ *
+ * When called, a new buffer must be returned WITHOUT blocking
+ * as this will be called from interrupt context.
+ *
+ * The buffer returned from the shim must be at least the size
+ * of used_buffer_size.
+ *
+ * @param current link
+ * @param old_buffer the buffer that are no longer needed
+ * @param used number of bytes in buffer that has been filled with data.
+ * @return new buffer to use
+ */
+ void * (*buffer_swap)(struct tsn_link *link, void *old_buffer,
+ size_t used);
+
+ /**
+ * buffer_refill - signal shim that more data is required
+ * @link Active link
+ *
+ * This function should not do anything that can preempt the
+ * task (kmalloc, sleeping lock) or invoke actions that can take
+ * a long time to complete.
+ *
+ * This will be called from tsn_buffer_read_net() when available
+ * data in the buffer drops below low_water_mark. It will be
+ * called with the link-lock *held*
+ */
+ size_t (*buffer_refill)(struct tsn_link *link);
+
+ /**
+ * buffer_drain - shim need to copy data from buffer
+ *
+ * This will be called from tsn_buffer_write_net() when data in
+ * the buffer exceeds high_water_mark.
+ *
+ * The expected behavior is for the shim to then fill data into
+ * the buffer via tsn_buffer_write()
+ */
+ size_t (*buffer_drain)(struct tsn_link *link);
+
+ /**
+ * media_close - shut down media controller properly
+ *
+ * when the link is closed/removed for some reason
+ * external to the media controller (ALSA soundcard, v4l2 driver
+ * etc), we call this to clean up.
+ *
+ * Normal operation is stopped before media_close is called, but
+ * all references should be valid. TSN core expects media_close
+ * to handle any local cleanup, once returned, any references in
+ * stale tsn_links cannot be trusted.
+ *
+ * @link: current link where data is stored
+ * @returns: 0 upon success, negative on error.
+ */
+ int (*media_close)(struct tsn_link *link);
+
+ /**
+ * hdr_size - ask shim how large the header is
+ *
+ * Needed when reserving space in skb for transmitting data.
+ *
+ * @link: current link where data is stored
+ * @return: size of header for this shim
+ */
+ size_t (*hdr_size)(struct tsn_link *link);
+
+ /**
+ * copy_size - ask client how much from the buffer to include in
+ * the next frame.
+ *
+ * This is for *outgoing* frames, incoming frames
+ * have 'sd_len' set in the header.
+ *
+ * Note: copy_size should not return a size larger
+ * than link->max_payload_size
+ */
+ size_t (*copy_size)(struct tsn_link *link);
+
+ /**
+ * validate_header - let the shim validate subtype-header
+ *
+ * Both psh and data may (or may not) contain headers that need
+ * validating. This is the responsibility of the shim to
+ * validate, and ops->valdiate_header() will be called before
+ * any data is copied from the incoming frame and into the
+ * buffer.
+ *
+ * Important: tsn_core expects validate_header to _not_ alter
+ * the contents of the frame, and ideally, validate_header could
+ * be called multiple times and give the same result.
+ *
+ * @param: active link owning the new data
+ * @param: start of data-unit header
+ *
+ * This function will be called from interrupt-context and MUST
+ * NOT take any locks.
+ */
+ int (*validate_header)(struct tsn_link *link,
+ struct avtpdu_header *header);
+
+ /**
+ * assemble_header - add shim-specific headers
+ *
+ * This adds the headers required by the current shim after the
+ * generic 1722-header.
+ *
+ * @param: active link
+ * @param: start of data-unit header
+ * @param: size of data to send in this frame
+ * @return void
+ */
+ void (*assemble_header)(struct tsn_link *link,
+ struct avtpdu_header *header, size_t bytes);
+
+ /**
+ * get_payload_data - get a pointer to where the data is stored
+ *
+ * core will use the pointer (or drop it if NULL is returned)
+ * and copy header->sd_len bytes of *consecutive* data from the
+ * target memory and into the buffer memory.
+ *
+ * This is called with relevant locks held, from interrupt context.
+ *
+ * @param link active link
+ * @param header header of frame, which contains data
+ * @returns pointer to memory to copy from
+ */
+ void * (*get_payload_data)(struct tsn_link *link,
+ struct avtpdu_header *header);
+};
+/**
+ * tsn_shim_register_ops - register shim-callbacks for a given shim
+ *
+ * @param shim_ops - callbacks. The ops-struct should be kept intact for
+ * as long as the driver is running.
+ *
+ *
+ */
+int tsn_shim_register_ops(struct tsn_shim_ops *shim_ops);
+
+/**
+ * tsn_shim_deregister_ops - remove callback for module
+ *
+ * Completely remove shim_ops. This will close any links currently using
+ * this shim. Note: the links will be closed, but _not_ removed.
+ *
+ * @param shim_ops ops associated with this shim
+ */
+void tsn_shim_deregister_ops(struct tsn_shim_ops *shim_ops);
+
+/**
+ * tsn_shim_get_active : return the name of the currently loaded shim
+ *
+ * @param current link
+ * @return name of shim (matches an entry from exported triggers)
+ */
+char *tsn_shim_get_active(struct tsn_link *link);
+
+/**
+ * tsn_shim_find_by_name find shim_ops by name
+ *
+ * @param name of shim
+ * @return shim or NULL if not found/error.
+ */
+struct tsn_shim_ops *tsn_shim_find_by_name(const char *name);
+
+/**
+ * tsn_shim_export_probe_triggers - export a list of registered shims
+ *
+ * @param page to write content into
+ * @returns length of data written to page
+ */
+ssize_t tsn_shim_export_probe_triggers(char *page);
+
+/**
+ * tsn_get_framesize - get the size of the next TSN frame to send
+ *
+ * This will call into the shim to get the next chunk of data to
+ * read. Some sanitychecking is performed, i.e.
+ *
+ * 0 <= size <= max_payload_size
+ *
+ * @param struct tsn_link *link active link
+ * @returns size of frame in bytes or negative on error.
+ */
+static inline size_t tsn_shim_get_framesize(struct tsn_link *link)
+{
+ size_t ret;
+
+ ret = link->ops->copy_size(link);
+ if (ret <= link->max_payload_size)
+ return ret;
+ return link->max_payload_size;
+}
+
+/**
+ * tsn_get_hdr_size - get the size of the shim-specific header size
+ *
+ * The shim will add it's own header to the frame.
+ */
+static inline size_t tsn_shim_get_hdr_size(struct tsn_link *link)
+{
+ size_t ret;
+
+ if (!link || !link->ops->hdr_size)
+ return -EINVAL;
+ ret = link->ops->hdr_size(link);
+ if (ret > link->max_payload_size)
+ return -EINVAL;
+ return ret;
+}
+
+#endif /* _TSN_H */
--
2.7.4
Richard Cochran
2016-06-13 11:47:26 UTC
Permalink
Henrik,
Post by Henrik Austad
There are at least one AVB-driver (the AV-part of TSN) in the kernel
already,
Which driver is that?
Post by Henrik Austad
however this driver aims to solve a wider scope as TSN can do
much more than just audio. A very basic ALSA-driver is added to the end
that allows you to play music between 2 machines using aplay in one end
and arecord | aplay on the other (some fiddling required) We have plans
for doing the same for v4l2 eventually (but there are other fishes to
fry first). The same goes for a TSN_SOCK type approach as well.
Please, no new socket type for this.
Post by Henrik Austad
What remains
- tie to (g)PTP properly, currently using ktime_get() for presentation
time
- get time from shim into TSN and vice versa
.. and a whole lot more, see below.
Post by Henrik Austad
- let shim create/manage buffer
(BTW, shim is a terrible name for that.)

[sigh]

People have been asking me about TSN and Linux, and we've made some
thoughts about it. The interest is there, and so I am glad to see
discussion on this topic.

Having said that, your series does not even begin to address the real
issues. I did not review the patches too carefully (because the
important stuff is missing), but surely configfs is the wrong
interface for this. In the end, we will be able to support TSN using
the existing networking and audio interfaces, adding appropriate
extensions.

Your patch features a buffer shared by networking and audio. This
isn't strictly necessary for TSN, and it may be harmful. The
Listeners are supposed to calculate the delay from frame reception to
the DA conversion. They can easily include the time needed for a user
space program to parse the frames, copy (and combine/convert) the
data, and re-start the audio transfer. A flexible TSN implementation
will leave all of the format and encoding task to the userland. After
all, TSN will some include more that just AV data, as you know.

Lets take a look at the big picture. One aspect of TSN is already
fully supported, namely the gPTP. Using the linuxptp user stack and a
modern kernel, you have a complete 802.1AS-2011 solution.

Here is what is missing to support audio TSN:

* User Space

1. A proper userland stack for AVDECC, MAAP, FQTSS, and so on. The
OpenAVB project does not offer much beyond simple examples.

2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.

* Kernel Space

1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)

2. Time based qdisc for transmitted frames. For MACs that support
this (like the i210), we only have to place the frame into the
correct queue. For normal HW, we want to be able to reserve a time
window in which non-TSN frames are blocked. This is some work, but
in the end it should be a generic solution that not only works
"perfectly" with TSN HW but also provides best effort service using
any NIC.

3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.

4. ALSA support for time triggered playback. The patch series
completely ignore the critical issue of media clock recovery. The
Listener must buffer the stream in order to play it exactly at a
specified time. It cannot simply send the stream ASAP to the audio
HW, because some other Listener might need longer. AFAICT, there
is nothing in ALSA that allows you to say, sample X should be
played at time Y.

These are some ideas about implementing TSN. Maybe some of it is
wrong (especially about ALSA), but we definitely need a proper design
to get the kernel parts right. There is plenty of work to do, but we
really don't need some hacky, in-kernel buffer with hard coded audio
formats.

Thanks,
Richard
Henrik Austad
2016-06-13 13:01:14 UTC
Permalink
Post by Richard Cochran
Henrik,
Hi Richard,
Post by Richard Cochran
Post by Henrik Austad
There are at least one AVB-driver (the AV-part of TSN) in the kernel
already,
Which driver is that?
drivers/net/ethernet/renesas/
Post by Richard Cochran
Post by Henrik Austad
however this driver aims to solve a wider scope as TSN can do
much more than just audio. A very basic ALSA-driver is added to the end
that allows you to play music between 2 machines using aplay in one end
and arecord | aplay on the other (some fiddling required) We have plans
for doing the same for v4l2 eventually (but there are other fishes to
fry first). The same goes for a TSN_SOCK type approach as well.
Please, no new socket type for this.
The idea was to create a tsn-driver and then allow userspace to use it
either for media or for whatever else they'd like - and then a socket made
sense. Or so I thought :)

What is the rationale for no new sockets? To avoid cluttering? or do
sockets have a drawback I'm not aware of?
Post by Richard Cochran
Post by Henrik Austad
What remains
- tie to (g)PTP properly, currently using ktime_get() for presentation
time
- get time from shim into TSN and vice versa
... and a whole lot more, see below.
Post by Henrik Austad
- let shim create/manage buffer
(BTW, shim is a terrible name for that.)
So something thin that is placed between to subystems should rather be
called.. flimsy? The point of the name was to indicate that it glued 2
pieces together. If you have a better suggestion, I'm all ears.
Post by Richard Cochran
[sigh]
People have been asking me about TSN and Linux, and we've made some
thoughts about it. The interest is there, and so I am glad to see
discussion on this topic.
I'm not aware of any such discussions, could you point me to where TSN has
been discussed, it would be nice to see other peoples thought on the matter
(which was one of the ideas behind this series in the first place)
Post by Richard Cochran
Having said that, your series does not even begin to address the real
issues.
Well, in all honesty, I did say so :) It is marked as "very-RFC", and not
for being included in the kernel as-is. I also made a short list of the
most crucial bits missing.

I know there are real issues, but solving these won't matter if you don't
have anything useful to do with it. I decided to start by adding a thin
ALSA-driver and then continue to work with the kernel infrastructure.
Having something that works-ish makes it a lot easier to test and get
others interested in, especially when you are not deeply involved in a
subsystem.

At one point you get to where you need input from other more intimate with
then inner workings of the different subsystems to see how things should be
created without making too much of a mess. So where we are :)

My primary motivation was to
a) gather feedback (which you have provided, and for which I am very
grateful)
b) get the discussion going on how/if TSN should be added to the kernel
Post by Richard Cochran
I did not review the patches too carefully (because the
important stuff is missing), but surely configfs is the wrong
interface for this.
Why is configfs wrong?

Unless you want to implement discovery and enumeration and srp-negotiation
in the kernel, you need userspace to handle this. Once userspace has done
all that (found priority-codes, streamIDs, vlanIDs and all the required
bits), then userspace can create a new link. For that I find ConfigFS to be
quite useful and up to the task.

In my opinion, it also makes for a much tidier and saner interface than
some obscure dark-magic ioctl()
Post by Richard Cochran
In the end, we will be able to support TSN using
the existing networking and audio interfaces, adding appropriate
extensions.
I surely hope so, but as I'm not deep into the networking part of the
kernel finding those appropriate extensions is hard - which is why we
started writing a standalone module-
Post by Richard Cochran
Your patch features a buffer shared by networking and audio. This
isn't strictly necessary for TSN, and it may be harmful.
At one stage, data has to flow in/out of the network, and whoever's using
TSN probably need to store data somewhere as well, so you need some form of
buffering at one place in the path the data flows through.

That being said, one of the bits on my plate is to remove the
"TSN-hosted-buffer" and let TSN read/write data via the shim_ops. What the
best set of functions where are, remain to be seen, but it should provide a
way to move data from either a single frame or a "few frames" to the shime
(err.. <descriptive word for a thin layer slapped between 2 largers
subsystems in the kernel> ;)
Post by Richard Cochran
The
Listeners are supposed to calculate the delay from frame reception to
the DA conversion. They can easily include the time needed for a user
space program to parse the frames, copy (and combine/convert) the
data, and re-start the audio transfer. A flexible TSN implementation
will leave all of the format and encoding task to the userland. After
all, TSN will some include more that just AV data, as you know.
Yes, or a ALSA-driver capable of same task. But yes, you need a way to
propagate the presentation-time (and maybe a timestamp for when a frame was
received) to the final destination of the samples. As far as I've been able
to tell, this is not possible in the kernel at the moment.
Post by Richard Cochran
Lets take a look at the big picture. One aspect of TSN is already
fully supported, namely the gPTP. Using the linuxptp user stack and a
modern kernel, you have a complete 802.1AS-2011 solution.
Yes, I thought so, which is also why I have put that to the side and why
I'm using ktime_get() for timestamps at the moment. There's also the issue
of hooking the time into ALSA/V4L2
Post by Richard Cochran
* User Space
1. A proper userland stack for AVDECC, MAAP, FQTSS, and so on. The
OpenAVB project does not offer much beyond simple examples.
yes, I've noticed. I've refered to an imaginary 'tsnctl' in the code, which
is supposed to be a "userspace catch-all for TSN-housekeeping". You
probably need a tsnd or similar as well to send keepalive frames etc.
Post by Richard Cochran
2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.
Why? the whole point should be to make it as easy for userspace as
possible. If you need to tailor each individual media-appliation to use
AVB, it is not going to be very useful outside pro-Audio. Sure, there will
be challenges, but one key element here should be to *not* require
upgrading every single media application.

Then, back to the suggestion of adding a TSN_SOCKET (which you didn't like,
but can we agree on a term "raw interface to TSN", and mode of transport
can be defined later? ), was to let those applications that are TSN-aware
to do what they need to do, whether it is controlling robots or media
streams.
Post by Richard Cochran
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
Ah, I was unaware of this, both CMESG and mmap buffers.

What is the accuracy of deferred transmit? If you have a class A stream,
you push out a new frame every 125 us, you may end up with
accuracy-constraints lower than that if you want to be able to state "send
frame X at time Y".
Post by Richard Cochran
2. Time based qdisc for transmitted frames. For MACs that support
this (like the i210), we only have to place the frame into the
correct queue. For normal HW, we want to be able to reserve a time
window in which non-TSN frames are blocked. This is some work, but
in the end it should be a generic solution that not only works
"perfectly" with TSN HW but also provides best effort service using
any NIC.
Yes, that would be very nice, and something like that is the ultimate goal
of the netdev_ops I added, even though it is a far way away from that now.
Post by Richard Cochran
3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.
Yes, this is something missing that must be adressed. And yes, I know
sharing a buffer the way the alsa-shim is currently doing is bad.
Post by Richard Cochran
4. ALSA support for time triggered playback. The patch series
completely ignore the critical issue of media clock recovery. The
Listener must buffer the stream in order to play it exactly at a
specified time. It cannot simply send the stream ASAP to the audio
HW, because some other Listener might need longer. AFAICT, there
is nothing in ALSA that allows you to say, sample X should be
played at time Y.
These are some ideas about implementing TSN. Maybe some of it is
wrong (especially about ALSA), but we definitely need a proper design
to get the kernel parts right. There is plenty of work to do, but we
really don't need some hacky, in-kernel buffer with hard coded audio
formats.
Well, the hard-coded audio format you refer to is placed with the avb_alsa
shim, avtp_du is part of the actual TSN-header so that is not audio-only.
And yes, it must be separated.

Apart from requiring media-applications to know about AVB, I don't think
we really disagree on anything. As I said, the main motivation for
submitting this now was to kick off a discussion, get some critical
response (your email was awesome - thanks!) and start steering the
development in the right direction.

Regards,
--
Henrik Austad
Richard Cochran
2016-06-13 19:32:29 UTC
Permalink
Post by Henrik Austad
Post by Richard Cochran
Which driver is that?
drivers/net/ethernet/renesas/
That driver is merely a PTP capable MAC driver, nothing more.
Although AVB is in the device name, the driver doesn't implement
anything beyond the PTP bits.
Post by Henrik Austad
What is the rationale for no new sockets? To avoid cluttering? or do
sockets have a drawback I'm not aware of?
The current raw sockets will work just fine. Again, there should be a
application that sits in between with the network socket and the audio
interface.
Post by Henrik Austad
Why is configfs wrong?
Because the application will use the already existing network and
audio interfaces to configure the system.
Post by Henrik Austad
Post by Richard Cochran
Lets take a look at the big picture. One aspect of TSN is already
fully supported, namely the gPTP. Using the linuxptp user stack and a
modern kernel, you have a complete 802.1AS-2011 solution.
Yes, I thought so, which is also why I have put that to the side and why
I'm using ktime_get() for timestamps at the moment. There's also the issue
of hooking the time into ALSA/V4L2
So lets get that issue solved before anything else. It is absolutely
essential for TSN. Without the synchronization, you are only playing
audio over the network. We already have software for that.
Post by Henrik Austad
Post by Richard Cochran
2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.
Why?
Because user space is right place to place the knowledge of the myriad
formats and options.
Post by Henrik Austad
the whole point should be to make it as easy for userspace as
possible. If you need to tailor each individual media-appliation to use
AVB, it is not going to be very useful outside pro-Audio. Sure, there will
be challenges, but one key element here should be to *not* require
upgrading every single media application.
Then, back to the suggestion of adding a TSN_SOCKET (which you didn't like,
but can we agree on a term "raw interface to TSN", and mode of transport
can be defined later? ), was to let those applications that are TSN-aware
to do what they need to do, whether it is controlling robots or media
streams.
First you say you don't want ot upgrade media applications, but then
you invent a new socket type. That is a contradiction in terms.

Audio apps already use networking, and they already use the audio
subsystem. We need to help them get their job done by providing the
missing kernel interfaces. They don't need extra magic buffering the
kernel. They already can buffer audio data by themselves.
Post by Henrik Austad
Post by Richard Cochran
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
Ah, I was unaware of this, both CMESG and mmap buffers.
What is the accuracy of deferred transmit? If you have a class A stream,
you push out a new frame every 125 us, you may end up with
accuracy-constraints lower than that if you want to be able to state "send
frame X at time Y".
I have no idea what you are asking here.

Sorry,
Richard
Henrik Austad
2016-06-14 09:30:22 UTC
Permalink
Post by Richard Cochran
Post by Henrik Austad
Post by Richard Cochran
Which driver is that?
drivers/net/ethernet/renesas/
That driver is merely a PTP capable MAC driver, nothing more.
Although AVB is in the device name, the driver doesn't implement
anything beyond the PTP bits.
Yes, I think they do the rest from userspace, not sure though :)
Post by Richard Cochran
Post by Henrik Austad
What is the rationale for no new sockets? To avoid cluttering? or do
sockets have a drawback I'm not aware of?
The current raw sockets will work just fine. Again, there should be a
application that sits in between with the network socket and the audio
interface.
So loop data from kernel -> userspace -> kernelspace and finally back to
userspace and the media application? I agree that you need a way to pipe
the incoming data directly from the network to userspace for those TSN
users that can handle it. But again, for media-applications that don't know
(or care) about AVB, it should be fed to ALSA/v4l2 directly and not jump
between kernel and userspace an extra round.

I get the point of not including every single audio/video encoder in the
kernel, but raw audio should be piped directly to alsa. V4L2 has a way of
piping encoded video through the system and to the media application (in
order to support cameras that to encoding). The same approach should be
doable for AVB, no? (someone from alsa/v4l2 should probably comment on
this)
Post by Richard Cochran
Post by Henrik Austad
Why is configfs wrong?
Because the application will use the already existing network and
audio interfaces to configure the system.
Configuring this via the audio-interface is going to be a challenge since
you need to configure the stream through the network before you can create
the audio interface. If not, you will have to either drop data or block the
caller until the link has been fully configured.

This is actually the reason why configfs is used in the series now, as it
allows userspace to figure out all the different attributes and configure
the link before letting ALSA start pushing data.
Post by Richard Cochran
Post by Henrik Austad
Post by Richard Cochran
Lets take a look at the big picture. One aspect of TSN is already
fully supported, namely the gPTP. Using the linuxptp user stack and a
modern kernel, you have a complete 802.1AS-2011 solution.
Yes, I thought so, which is also why I have put that to the side and why
I'm using ktime_get() for timestamps at the moment. There's also the issue
of hooking the time into ALSA/V4L2
So lets get that issue solved before anything else. It is absolutely
essential for TSN. Without the synchronization, you are only playing
audio over the network. We already have software for that.
Yes, I agree, presentation-time and local time needs to be handled
properly. The same for adjusting sample-rate etc. This is a lot of work, so
I hope you can understand why I started out with a simple approach to spark
a discussion before moving on to the larger bits.
Post by Richard Cochran
Post by Henrik Austad
Post by Richard Cochran
2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.
Why?
Because user space is right place to place the knowledge of the myriad
formats and options.
Se response above, better to let anything but uncompressed raw data trickle
through.
Post by Richard Cochran
Post by Henrik Austad
the whole point should be to make it as easy for userspace as
possible. If you need to tailor each individual media-appliation to use
AVB, it is not going to be very useful outside pro-Audio. Sure, there will
be challenges, but one key element here should be to *not* require
upgrading every single media application.
Then, back to the suggestion of adding a TSN_SOCKET (which you didn't like,
but can we agree on a term "raw interface to TSN", and mode of transport
can be defined later? ), was to let those applications that are TSN-aware
to do what they need to do, whether it is controlling robots or media
streams.
First you say you don't want ot upgrade media applications, but then
you invent a new socket type. That is a contradiction in terms.
Hehe, no, bad phrasing on my part. I want *both* (hence the shim-interface)
:)
Post by Richard Cochran
Audio apps already use networking, and they already use the audio
subsystem. We need to help them get their job done by providing the
missing kernel interfaces. They don't need extra magic buffering the
kernel. They already can buffer audio data by themselves.
Yes, I know some audio apps "use networking", I can stream netradio, I can
use jack to connect devices using RTP and probably a whole lot of other
applications do similar things. However, AVB is more about using the
network as a virtual sound-card. For the media application, it should not
have to care if the device it is using is a soudncard inside the box or a
set of AVB-capable speakers somewhere on the network.
Post by Richard Cochran
Post by Henrik Austad
Post by Richard Cochran
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
Ah, I was unaware of this, both CMESG and mmap buffers.
What is the accuracy of deferred transmit? If you have a class A stream,
you push out a new frame every 125 us, you may end up with
accuracy-constraints lower than that if you want to be able to state "send
frame X at time Y".
I have no idea what you are asking here.
I assumed that when you had a mmap'd buffer you'd have to specify a point
in time at which the frame should be sent. And since a class A has a 125us
interval of sending frames, you have to be able to send frames with enough
accuray to that. That's a pretty strict deadline coming from userspace. But
as they say, never assume.

I have a lot to dig into, and I've gotten a lot of very useful pointers. I
should be busy for a while
--
Henrik Austad
Richard Cochran
2016-06-14 18:26:27 UTC
Permalink
Post by Henrik Austad
So loop data from kernel -> userspace -> kernelspace and finally back to
userspace and the media application?
Huh? I wonder where you got that idea. Let me show an example of
what I mean.

void listener()
{
int in = socket();
int out = open("/dev/dsp");
char buf[];

while (1) {
recv(in, buf, packetsize);
write(out, buf + offset, datasize);
}
}

See?
Post by Henrik Austad
Yes, I know some audio apps "use networking", I can stream netradio, I can
use jack to connect devices using RTP and probably a whole lot of other
applications do similar things. However, AVB is more about using the
network as a virtual sound-card.
That is news to me. I don't recall ever having seen AVB described
like that before.
Post by Henrik Austad
For the media application, it should not
have to care if the device it is using is a soudncard inside the box or a
set of AVB-capable speakers somewhere on the network.
So you would like a remote listener to appear in the system as a local
PCM audio sink? And a remote talker would be like a local media URL?
Sounds unworkable to me, but even if you were to implement it, the
logic would surely belong in alsa-lib and not in the kernel. Behind
the enulated device, the library would run a loop like the example,
above.

In any case, your patches don't implement that sort of thing at all,
do they?

Thanks,
Richard
Henrik Austad
2016-06-14 20:38:24 UTC
Permalink
Post by Richard Cochran
Post by Henrik Austad
So loop data from kernel -> userspace -> kernelspace and finally back to
userspace and the media application?
Huh? I wonder where you got that idea. Let me show an example of
what I mean.
void listener()
{
int in = socket();
int out = open("/dev/dsp");
char buf[];
while (1) {
recv(in, buf, packetsize);
write(out, buf + offset, datasize);
}
}
See?
Where is your media-application in this? You only loop the audio from
network to the dsp, is the media-application attached to the dsp-device?

Whereas I want to do

aplay some_song.wav
or mplayer
or spotify
or ..
Post by Richard Cochran
Post by Henrik Austad
Yes, I know some audio apps "use networking", I can stream netradio, I can
use jack to connect devices using RTP and probably a whole lot of other
applications do similar things. However, AVB is more about using the
network as a virtual sound-card.
That is news to me. I don't recall ever having seen AVB described
like that before.
Post by Henrik Austad
For the media application, it should not
have to care if the device it is using is a soudncard inside the box or a
set of AVB-capable speakers somewhere on the network.
So you would like a remote listener to appear in the system as a local
PCM audio sink? And a remote talker would be like a local media URL?
Sounds unworkable to me, but even if you were to implement it, the
logic would surely belong in alsa-lib and not in the kernel. Behind
the enulated device, the library would run a loop like the example,
above.
In any case, your patches don't implement that sort of thing at all,
do they?
Subject: [very-RFC 7/8] AVB ALSA - Add ALSA shim for TSN

Did you even bother to look?
--
Henrik Austad
Richard Cochran
2016-06-15 07:04:53 UTC
Permalink
Post by Henrik Austad
Whereas I want to do
aplay some_song.wav
Can you please explain how your patches accomplish this?

Thanks,
Richard
Henrik Austad
2016-06-15 07:50:16 UTC
Permalink
Post by Richard Cochran
Post by Henrik Austad
Whereas I want to do
aplay some_song.wav
Can you please explain how your patches accomplish this?
In short:

modprobe tsn
modprobe avb_alsa
mkdir /sys/kernel/config/eth0/link
cd /sys/kernel/config/eth0/link
<set approriate values vor the attributes>
echo alsa > enabled
aplay -Ddefault:CARD=avb some_song.wav

Likewise on the receiver side, except add 'Listener' to end_station
attribute

arecord -c2 -r48000 -f S16_LE -Ddefault:CARD=avb > some_recording.wav

I've not had time to fully fix the hw-aprams for alsa, so some manual
tweaking of arecord is required.


Again, this is a very early attempt to get something useful done with TSN,
I know there are rough edges, I know buffer handling and timestamping is
not finished


Note: if you don't have an intel-card, load tsn in debug-mode and it will
let you use all NICs present.

modprobe tsn in_debug=1
--
Henrik Austad
Richard Cochran
2016-06-15 11:42:06 UTC
Permalink
Post by Richard Cochran
Post by Henrik Austad
Whereas I want to do
aplay some_song.wav
Can you please explain how your patches accomplish this?
Never mind. Looking back, I found it in patch #7.

Thanks,
Richard
Richard Cochran
2016-06-15 07:12:05 UTC
Permalink
Post by Henrik Austad
Where is your media-application in this?
Um, that *is* a media application. It plays music on the sound card.
Post by Henrik Austad
You only loop the audio from
network to the dsp, is the media-application attached to the dsp-device?
Sorry, I thought the old OSS API would be familiar and easy to
understand. The /dev/dsp is the sound card.

Thanks,
Richard
Richard Cochran
2016-06-13 19:37:56 UTC
Permalink
Post by Henrik Austad
Post by Richard Cochran
People have been asking me about TSN and Linux, and we've made some
thoughts about it. The interest is there, and so I am glad to see
discussion on this topic.
I'm not aware of any such discussions, could you point me to where TSN has
been discussed, it would be nice to see other peoples thought on the matter
(which was one of the ideas behind this series in the first place)
To my knowledge, there hasn't been any previous TSN talk on lkml.

(You have just now started the discussion ;)

Sorry for not being clear.

Richard
Arnd Bergmann
2016-06-13 13:11:06 UTC
Permalink
Post by Richard Cochran
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
After some back and forth, I think the conclusion for now was that
the timestamps in the current v3 format are sufficient until 2106
as long as we treat them as 'unsigned', so we don't need the new
format for y2038, but if we get a new format, that should definitely
use 64-bit timestamps because that is the right thing to do.

Arnd
John Fastabend
2016-06-13 15:57:18 UTC
Permalink
Post by Richard Cochran
Henrik,
Post by Henrik Austad
There are at least one AVB-driver (the AV-part of TSN) in the kernel
already,
Which driver is that?
Post by Henrik Austad
however this driver aims to solve a wider scope as TSN can do
much more than just audio. A very basic ALSA-driver is added to the end
that allows you to play music between 2 machines using aplay in one end
and arecord | aplay on the other (some fiddling required) We have plans
for doing the same for v4l2 eventually (but there are other fishes to
fry first). The same goes for a TSN_SOCK type approach as well.
Please, no new socket type for this.
Post by Henrik Austad
What remains
- tie to (g)PTP properly, currently using ktime_get() for presentation
time
- get time from shim into TSN and vice versa
... and a whole lot more, see below.
Post by Henrik Austad
- let shim create/manage buffer
(BTW, shim is a terrible name for that.)
[sigh]
People have been asking me about TSN and Linux, and we've made some
thoughts about it. The interest is there, and so I am glad to see
discussion on this topic.
Having said that, your series does not even begin to address the real
issues. I did not review the patches too carefully (because the
important stuff is missing), but surely configfs is the wrong
interface for this. In the end, we will be able to support TSN using
the existing networking and audio interfaces, adding appropriate
extensions.
Your patch features a buffer shared by networking and audio. This
isn't strictly necessary for TSN, and it may be harmful. The
Listeners are supposed to calculate the delay from frame reception to
the DA conversion. They can easily include the time needed for a user
space program to parse the frames, copy (and combine/convert) the
data, and re-start the audio transfer. A flexible TSN implementation
will leave all of the format and encoding task to the userland. After
all, TSN will some include more that just AV data, as you know.
Lets take a look at the big picture. One aspect of TSN is already
fully supported, namely the gPTP. Using the linuxptp user stack and a
modern kernel, you have a complete 802.1AS-2011 solution.
* User Space
1. A proper userland stack for AVDECC, MAAP, FQTSS, and so on. The
OpenAVB project does not offer much beyond simple examples.
2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
2. Time based qdisc for transmitted frames. For MACs that support
this (like the i210), we only have to place the frame into the
correct queue. For normal HW, we want to be able to reserve a time
window in which non-TSN frames are blocked. This is some work, but
in the end it should be a generic solution that not only works
"perfectly" with TSN HW but also provides best effort service using
any NIC.
When I looked at this awhile ago I convinced myself that it could fit
fairly well into the DCB stack (DCB is also part of 802.1Q). A lot of
the traffic class to queue mappings and priories could be handled here.
It might be worth taking a look at ./net/sched/mqprio.c and ./net/dcb/.

Unfortunately I didn't get too far along but we probably don't want
another mechanism to map hw queues/tcs/etc if the existing interfaces
work or can be extended to support this.
Post by Richard Cochran
3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.
4. ALSA support for time triggered playback. The patch series
completely ignore the critical issue of media clock recovery. The
Listener must buffer the stream in order to play it exactly at a
specified time. It cannot simply send the stream ASAP to the audio
HW, because some other Listener might need longer. AFAICT, there
is nothing in ALSA that allows you to say, sample X should be
played at time Y.
These are some ideas about implementing TSN. Maybe some of it is
wrong (especially about ALSA), but we definitely need a proper design
to get the kernel parts right. There is plenty of work to do, but we
really don't need some hacky, in-kernel buffer with hard coded audio
formats.
Thanks,
Richard
Henrik Austad
2016-06-14 08:36:12 UTC
Permalink
Post by John Fastabend
Post by Richard Cochran
[...]
* User Space
1. A proper userland stack for AVDECC, MAAP, FQTSS, and so on. The
OpenAVB project does not offer much beyond simple examples.
2. A user space audio application that puts it all together, making
use of the services in #1, the linuxptp gPTP service, the ALSA
services, and the network connections. This program will have all
the knowledge about packet formats, AV encodings, and the local HW
capabilities. This program cannot yet be written, as we still need
some kernel work in the audio and networking subsystems.
* Kernel Space
1. Providing frames with a future transmit time. For normal sockets,
this can be in the CMESG data. For mmap'ed buffers, we will need a
new format. (I think Arnd is working on a new layout.)
2. Time based qdisc for transmitted frames. For MACs that support
this (like the i210), we only have to place the frame into the
correct queue. For normal HW, we want to be able to reserve a time
window in which non-TSN frames are blocked. This is some work, but
in the end it should be a generic solution that not only works
"perfectly" with TSN HW but also provides best effort service using
any NIC.
When I looked at this awhile ago I convinced myself that it could fit
fairly well into the DCB stack (DCB is also part of 802.1Q). A lot of
the traffic class to queue mappings and priories could be handled here.
It might be worth taking a look at ./net/sched/mqprio.c and ./net/dcb/.
Interesting, I'll have a look at dcb and mqprio, I'm not familiar with
those systems. Thanks for pointing those out!

I hope that the complexity doesn't run crazy though, TSN is not aimed at
datacentra, a lot of the endpoints are going to be embedded devices,
introducing a massive stack for handling every eventuality in 802.1q is
going to be counter productive.
Post by John Fastabend
Unfortunately I didn't get too far along but we probably don't want
another mechanism to map hw queues/tcs/etc if the existing interfaces
work or can be extended to support this.
Sure, I get that, as long as the complexity for setting up a link doesn't
go through the roof :)

Thanks!
--
Henrik Austad
Richard Cochran
2016-06-13 19:52:07 UTC
Permalink
Post by Richard Cochran
3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.
Actually, we already have support for tunable clock-like HW elements,
namely the dynamic posix clock API. It is trivial to write a driver
for VCO or the like. I am just not too familiar with the latest high
end audio devices.

I have seen audio PLL/multiplier chips that will take, for example, a
10 kHz input and produce your 48 kHz media clock. With the right HW
design, you can tell your PTP Hardware Clock to produce a 10000 PPS,
and you will have a synchronized AVB endpoint. The software is all
there already. Somebody should tell the ALSA guys about it.

I don't know if ALSA has anything for sample rate conversion or not,
but haven't seen anything that addresses distributed synchronized
audio applications.

Thanks,
Richard
One Thousand Gnomes
2016-06-14 11:19:01 UTC
Permalink
On Mon, 13 Jun 2016 21:51:36 +0200
Post by Richard Cochran
Post by Richard Cochran
3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.
Actually, we already have support for tunable clock-like HW elements,
namely the dynamic posix clock API. It is trivial to write a driver
for VCO or the like. I am just not too familiar with the latest high
end audio devices.
Why high end ? Even the most basic USB audio is frame based and
isosynchronous to the USB clock. It also reports back the delay
properties.

Alan
Richard Cochran
2016-06-14 17:05:01 UTC
Permalink
Post by One Thousand Gnomes
On Mon, 13 Jun 2016 21:51:36 +0200
Post by Richard Cochran
Actually, we already have support for tunable clock-like HW elements,
namely the dynamic posix clock API. It is trivial to write a driver
for VCO or the like. I am just not too familiar with the latest high
end audio devices.
Why high end ? Even the most basic USB audio is frame based and
isosynchronous to the USB clock. It also reports back the delay
properties.
Well, I guess I should have said, I am not too familiar with the
breadth of current audio hardware, high end or low end. Of course I
would like to see even consumer devices work with AVB, but it is up to
the ALSA people to make that happen. So far, nothing has been done,
afaict.

Thanks,
Richard
Takashi Sakamoto
2016-06-15 03:22:46 UTC
Permalink
Hi Richard,
Post by Richard Cochran
3. ALSA support for tunable AD/DA clocks. The rate of the Listener's
DA clock must match that of the Talker and the other Listeners.
Either you adjust it in HW using a VCO or similar, or you do
adaptive sample rate conversion in the application. (And that is
another reason for *not* having a shared kernel buffer.) For the
Talker, either you adjust the AD clock to match the PTP time, or
you measure the frequency offset.
I have seen audio PLL/multiplier chips that will take, for example, a
10 kHz input and produce your 48 kHz media clock. With the right HW
design, you can tell your PTP Hardware Clock to produce a 10000 PPS,
and you will have a synchronized AVB endpoint. The software is all
there already. Somebody should tell the ALSA guys about it.
Just from my curiosity, could I ask you more explanation for it in ALSA
side?

The similar mechanism to synchronize endpoints was also applied to audio
and music unit on IEEE 1394 bus. According to IEC 61883-1/6, some of
these actual units can generate presentation-timestamp from header
information of 8,000 packet per sec, and utilize the signal as sampling
clock[1].

There's much differences between IEC 61883-1/6 on IEEE 1394 bus and
Audio and Video Bridge on Ethernet[2], especially for synchronization,
but in this point of transferring synchnization signal and time-based
data, we have the similar requirements of software implementations, I think.

My motivation to join in this discussion is to consider about to make it
clear to implement packet-oriented drivers in ALSA kernel-land, and
enhance my work for drivers to handle IEC 61883-1/6 on IEEE 1394 bus.
Post by Richard Cochran
I don't know if ALSA has anything for sample rate conversion or not,
but haven't seen anything that addresses distributed synchronized
audio applications.
In ALSA, sampling rate conversion should be in userspace, not in kernel
land. In alsa-lib, sampling rate conversion is implemented in shared
object. When userspace applications start playbacking/capturing,
depending on PCM node to access, these applications load the shared
object and convert PCM frames from buffer in userspace to mmapped
DMA-buffer, then commit them.

Before establishing a PCM substream, userspace applications and
in-kernel drivers communicate to decide sampling rate, PCM frame format,
the size of PCM buffer, and so on. (see snd_pcm_hw_params() and
ioctl(SNDRV_PCM_IOCTL_HW_PARAMS)). Thus, as long as in-kernel drivers
know specifications of endpoints, userspace applications can start PCM
substreams correctly.


[1] In detail, please refer to specification of 1394TA I introduced:
http://www.spinics.net/lists/netdev/msg381259.html
[2] I guess that IEC 61883-1/6 packet for Ethernet-AVB is a mutant from
original specifications.


Regards

Takashi Sakamoto
Richard Cochran
2016-06-15 08:06:21 UTC
Permalink
Post by Takashi Sakamoto
Post by Richard Cochran
I have seen audio PLL/multiplier chips that will take, for example, a
10 kHz input and produce your 48 kHz media clock. With the right HW
design, you can tell your PTP Hardware Clock to produce a 10000 PPS,
and you will have a synchronized AVB endpoint. The software is all
there already. Somebody should tell the ALSA guys about it.
Just from my curiosity, could I ask you more explanation for it in ALSA
side?
(Disclaimer: I really don't know too much about ALSA, expect that is
fairly big and complex ;)

Here is what I think ALSA should provide:

- The DA and AD clocks should appear as attributes of the HW device.

- There should be a method for measuring the DA/AD clock rate with
respect to both the system time and the PTP Hardware Clock (PHC)
time.

- There should be a method for adjusting the DA/AD clock rate if
possible. If not, then ALSA should fall back to sample rate
conversion.

- There should be a method to determine the time delay from the point
when the audio data are enqueued into ALSA until they pass through
the D/A converter. If this cannot be known precisely, then the
library should provide an estimate with an error bound.

- I think some AVB use cases will need to know the time delay from A/D
until the data are available to the local application. (Distributed
microphones? I'm not too sure about that.)

- If the DA/AD clocks are connected to other clock devices in HW,
there should be a way to find this out in SW. For example, if SW
can see the PTP-PHC-PLL-DA relationship from the above example, then
it knows how to synchronize the DA clock using the network.

[ Implementing this point involves other subsystems beyond ALSA. It
isn't really necessary for people designing AVB systems, since
they know their designs, but it would be nice to have for writing
generic applications that can deal with any kind of HW setup. ]
Post by Takashi Sakamoto
In ALSA, sampling rate conversion should be in userspace, not in kernel
land. In alsa-lib, sampling rate conversion is implemented in shared object.
When userspace applications start playbacking/capturing, depending on PCM
node to access, these applications load the shared object and convert PCM
frames from buffer in userspace to mmapped DMA-buffer, then commit them.
The AVB use case places an additional requirement on the rate
conversion. You will need to adjust the frequency on the fly, as the
stream is playing. I would guess that ALSA doesn't have that option?

Thanks,
Richard
Takashi Sakamoto
2016-06-18 05:22:42 UTC
Permalink
Hi,

Sorry to be late. In this weekday, I have little time for this thread
because working for alsa-lib[1]. Besides, I'm not full-time developer
for this kind of work. In short, I use my limited private time for this
discussion.
Post by Richard Cochran
Post by Takashi Sakamoto
Post by Richard Cochran
I have seen audio PLL/multiplier chips that will take, for example, a
10 kHz input and produce your 48 kHz media clock. With the right HW
design, you can tell your PTP Hardware Clock to produce a 10000 PPS,
and you will have a synchronized AVB endpoint. The software is all
there already. Somebody should tell the ALSA guys about it.
Just from my curiosity, could I ask you more explanation for it in ALSA
side?
(Disclaimer: I really don't know too much about ALSA, expect that is
fairly big and complex ;)
In this morning, I read IEEE 1722:2011 and realized that it quite
roughly refers to IEC 61883-1/6 and includes much ambiguities to end
applications.

(In my opinion, the author just focuses on packet with timestamps,
without enough considering about how to implement endpoint applications
which perform semi-real sampling, fetching and queueing and so on, so as
you. They're satisfied just by handling packet with timestamp, without
enough consideration about actual hardware/software applications.)
Post by Richard Cochran
- The DA and AD clocks should appear as attributes of the HW device.
- There should be a method for measuring the DA/AD clock rate with
respect to both the system time and the PTP Hardware Clock (PHC)
time.
- There should be a method for adjusting the DA/AD clock rate if
possible. If not, then ALSA should fall back to sample rate
conversion.
- There should be a method to determine the time delay from the point
when the audio data are enqueued into ALSA until they pass through
the D/A converter. If this cannot be known precisely, then the
library should provide an estimate with an error bound.
- I think some AVB use cases will need to know the time delay from A/D
until the data are available to the local application. (Distributed
microphones? I'm not too sure about that.)
- If the DA/AD clocks are connected to other clock devices in HW,
there should be a way to find this out in SW. For example, if SW
can see the PTP-PHC-PLL-DA relationship from the above example, then
it knows how to synchronize the DA clock using the network.
[ Implementing this point involves other subsystems beyond ALSA. It
isn't really necessary for people designing AVB systems, since
they know their designs, but it would be nice to have for writing
generic applications that can deal with any kind of HW setup. ]
Depends on which subsystem decides "AVTP presentation time"[3]. This
value is dominant to the number of events included in an IEC 61883-1
packet. If this TSN subsystem decides it, most of these items don't need
to be in ALSA.

As long as I know, the number of AVTPDU per second seems not to be
fixed. So each application is not allowed to calculate the timestamp by
its own way unless TSN implementation gives the information to each
applications.

For your information, in current ALSA implementation of IEC 61883-1/6 on
IEEE 1394 bus, the presentation timestamp is decided in ALSA side. The
number of isochronous packet transmitted per second is fixed by 8,000 in
IEEE 1394, and the number of data blocks in an IEC 61883-1 packet is
deterministic according to 'sampling transfer frequency' in IEC 61883-6
and isochronous cycle count passed from Linux FireWire subsystem.

In the TSN subsystem, like FireWire subsystem, callback for filling
payload should have information of 'when the packet is scheduled to be
transmitted'. With the information, each application can calculate the
number of event in the packet and presentation timestamp. Of cource,
this timestamp should be handled as 'avtp_timestamp' in packet queueing.
Post by Richard Cochran
Post by Takashi Sakamoto
In ALSA, sampling rate conversion should be in userspace, not in kernel
land. In alsa-lib, sampling rate conversion is implemented in shared object.
When userspace applications start playbacking/capturing, depending on PCM
node to access, these applications load the shared object and convert PCM
frames from buffer in userspace to mmapped DMA-buffer, then commit them.
The AVB use case places an additional requirement on the rate
conversion. You will need to adjust the frequency on the fly, as the
stream is playing. I would guess that ALSA doesn't have that option?
In ALSA kernel/userspace interfaces , the specification cannot be
supported, at all.

Please explain about this requirement, where it comes from, which
specification and clause describe it (802.1AS or 802.1Q?). As long as I
read IEEE 1722, I cannot find such a requirement.

(When considering about actual hardware codecs, on-board serial bus such
as Inter-IC Sound, corresponding controller, immediate change of
sampling rate is something imaginary for semi-realtime applications. And
the idea has no meaning for typical playback/capture softwares.)


[1] [alsa-lib][PATCH 0/9 v3] ctl: add APIs for control element set
http://mailman.alsa-project.org/pipermail/alsa-devel/2016-June/109274.html
[2] IEEE 1722-2011
http://ieeexplore.ieee.org/servlet/opac?punumber=5764873
[3] 5.5 Timing and Synchronization
op. cit.
[4] 1394 Open Host Controller Interface Specification
http://download.microsoft.com/download/1/6/1/161ba512-40e2-4cc9-843a-923143f3456c/ohci_11.pdf


Regards

Takashi Sakamoto
Henrik Austad
2016-06-18 22:46:04 UTC
Permalink
Post by Takashi Sakamoto
Hi,
Hi Takashi,

You raise a lot of valid points and questions, I'll try to answer them.

edit: this turned out to be a somewhat lengthy answer. I have tried to
shorten it down somewhere. it is getting late and I'm getting increasingly
incoherent (Richard probably knows what I'm talking about ;) so I'll stop
for now.

Plase post a follow-up with everything that's not clear!
Thanks!
Post by Takashi Sakamoto
Sorry to be late. In this weekday, I have little time for this thread
because working for alsa-lib[1]. Besides, I'm not full-time developer
for this kind of work. In short, I use my limited private time for this
discussion.
Thank you for taking the time to reply to this thread then, it is much
appreciated
Post by Takashi Sakamoto
Post by Richard Cochran
Post by Takashi Sakamoto
Post by Richard Cochran
I have seen audio PLL/multiplier chips that will take, for example, a
10 kHz input and produce your 48 kHz media clock. With the right HW
design, you can tell your PTP Hardware Clock to produce a 10000 PPS,
and you will have a synchronized AVB endpoint. The software is all
there already. Somebody should tell the ALSA guys about it.
Just from my curiosity, could I ask you more explanation for it in ALSA
side?
(Disclaimer: I really don't know too much about ALSA, expect that is
fairly big and complex ;)
In this morning, I read IEEE 1722:2011 and realized that it quite
roughly refers to IEC 61883-1/6 and includes much ambiguities to end
applications.
As far as I know, 1722 aims to describe how the data is wrapped in AVTPDU
(and likewise for control-data), not how the end-station should implement
it.

If there are ambiguities, would you mind listing a few? It would serve as a
useful guide as to look for other pitfalls as well (thanks!)
Post by Takashi Sakamoto
(In my opinion, the author just focuses on packet with timestamps,
without enough considering about how to implement endpoint applications
which perform semi-real sampling, fetching and queueing and so on, so as
you. They're satisfied just by handling packet with timestamp, without
enough consideration about actual hardware/software applications.)
You are correct, none of the standards explain exactly how it should be
implemented, only what the end result should look like. One target of this
collection of standards are embedded, dedicated AV equipment and the
authors have no way of knowing (nor should they care I think) the
underlying architecture of these.
Post by Takashi Sakamoto
Post by Richard Cochran
- The DA and AD clocks should appear as attributes of the HW device.
This would be very useful and helpful when determining if the clock of the
HW time is falling behind or racing ahead of the gPTP time domain. It will
also help finding the capture time or calculating when a sample in the
buffer will be played back by the device.
Post by Takashi Sakamoto
Post by Richard Cochran
- There should be a method for measuring the DA/AD clock rate with
respect to both the system time and the PTP Hardware Clock (PHC)
time.
as above.
Post by Takashi Sakamoto
Post by Richard Cochran
- There should be a method for adjusting the DA/AD clock rate if
possible. If not, then ALSA should fall back to sample rate
conversion.
This is not a requirement from the standard, but will help avoid costly
resampling. At least it should be possible to detect the *need* for
resampling so that we can try to avoid underruns.
Post by Takashi Sakamoto
Post by Richard Cochran
- There should be a method to determine the time delay from the point
when the audio data are enqueued into ALSA until they pass through
the D/A converter. If this cannot be known precisely, then the
library should provide an estimate with an error bound.
- I think some AVB use cases will need to know the time delay from A/D
until the data are available to the local application. (Distributed
microphones? I'm not too sure about that.)
yes, if you have multiple microphones that you want to combine into a
stream and do signal processing, some cases require sample-sync (so within
1 us accuracy for 48kHz).
Post by Takashi Sakamoto
Post by Richard Cochran
- If the DA/AD clocks are connected to other clock devices in HW,
there should be a way to find this out in SW. For example, if SW
can see the PTP-PHC-PLL-DA relationship from the above example, then
it knows how to synchronize the DA clock using the network.
[ Implementing this point involves other subsystems beyond ALSA. It
isn't really necessary for people designing AVB systems, since
they know their designs, but it would be nice to have for writing
generic applications that can deal with any kind of HW setup. ]
Depends on which subsystem decides "AVTP presentation time"[3].
Presentation time is either set by
a) Local sound card performing capture (in which case it will be 'capture
time')
b) Local media application sending a stream accross the network
(time when the sample should be played out remotely)
c) Remote media application streaming data *to* host, in which case it will
be local presentation time on local soundcard
Post by Takashi Sakamoto
This value is dominant to the number of events included in an IEC 61883-1
packet. If this TSN subsystem decides it, most of these items don't need
to be in ALSA.
Not sure if I understand this correctly.

TSN should have a reference to the timing-domain of each *local*
sound-device (for local capture or playback) as well as the shared
time-reference provided by gPTP.

Unless an End-station acts as GrandMaster for the gPTP-domain, time set
forth by gPTP is inmutable and cannot be adjusted. It follows that the
sample-frequency of the local audio-devices must be adjusted, or the
audio-streams to/from said devices must be resampled.
Post by Takashi Sakamoto
As long as I know, the number of AVTPDU per second seems not to be
fixed. So each application is not allowed to calculate the timestamp by
its own way unless TSN implementation gives the information to each
applications.
Before initiating a stream, an application needs to reserve a path and
bandwidth through the network. Every bridge (switch/router) must accept
this for the stream-allocation to succeed. If a single bridge along the way
declies, the entire stream is denied. The StreamID combined with traffic
class and destination address is used to uniquely identify the stream.

Once ready, frames leaving the End-station with the same StreamID will be
forwarded through the bridges to the End-station(s).

If you choose to transmit *less* than the bandwidth you reserved, that is
fine, but you cannot transmit *more*.

As to timestamps. When a talker transmit a frame, the timestamp in the
AVTPDU describes the presentation-time.

1) The Talker is a mic, and the timestamp will then be the capture-time
of the sample.
2) For a Listener, the timestamp will be the presentation-time,
the time when the *first* sample in the sample-set should be played (or
aligned in an offline format with other samples).

The application should be part of the same gPTP-domain as all the other
nodes in the domain, and all the nodes share a common sense of time. That
means that time X will be the exact same time (or, within a sub-microsecond
error) for all the nodes in the same domain.
Post by Takashi Sakamoto
For your information, in current ALSA implementation of IEC 61883-1/6 on
IEEE 1394 bus, the presentation timestamp is decided in ALSA side. The
number of isochronous packet transmitted per second is fixed by 8,000 in
IEEE 1394, and the number of data blocks in an IEC 61883-1 packet is
deterministic according to 'sampling transfer frequency' in IEC 61883-6
and isochronous cycle count passed from Linux FireWire subsystem.
For an audio-stream, it will be very similar. The difference is the split
between class A and class B, the former is 8kHz frame-rate and a guaranteed
2ms latency accross the network (think required buffering at end-stations),
class B is 4kHz and a 50ms max latency. Class B is used for links
traversing 1 or 2 wireless links.

If you look at the avb-shim in the series, you see that for 48kHz, 2ch,
S16_LE, every frame is of the same size, 6 samples per frame, total of 24
bytes / frame. For class B, size doubles to 48 bytes as it transmits frames
4000 times / sec.

The 44.1 part is a bit more painful/messy/horrible, but is doable because
the stream-reservation only gives an *upper* bound of bandwidth.
Post by Takashi Sakamoto
In the TSN subsystem, like FireWire subsystem, callback for filling
payload should have information of 'when the packet is scheduled to be
transmitted'.
[ Given that you are part of a gPTP domain and that you share a common
sense of what time it is *now* with all the other devices ]

A frame should be transmittet so that it will not arrive too late for it to
be presented. A class A link guarantees that a frame will be delivered
within 2ms. Then, by looking at the timestamp, you subtract the
delivery-time and you get when the frame should be sent at the latest.
Post by Takashi Sakamoto
With the information, each application can calculate the
number of event in the packet and presentation timestamp. Of cource,
this timestamp should be handled as 'avtp_timestamp' in packet queueing.
Not sure if I understand what you are asking, but I think maybe I've
answered this above (re. 48kHz, 44.1khz and upper bound of framesize?)
Post by Takashi Sakamoto
Post by Richard Cochran
Post by Takashi Sakamoto
In ALSA, sampling rate conversion should be in userspace, not in kernel
land. In alsa-lib, sampling rate conversion is implemented in shared object.
When userspace applications start playbacking/capturing, depending on PCM
node to access, these applications load the shared object and convert PCM
frames from buffer in userspace to mmapped DMA-buffer, then commit them.
The AVB use case places an additional requirement on the rate
conversion. You will need to adjust the frequency on the fly, as the
stream is playing. I would guess that ALSA doesn't have that option?
In ALSA kernel/userspace interfaces , the specification cannot be
supported, at all.
Please explain about this requirement, where it comes from, which
specification and clause describe it (802.1AS or 802.1Q?). As long as I
read IEEE 1722, I cannot find such a requirement.
1722 only describes how the L2 frames are constructed and transmittet. You
are correct that it does not mention adjustable clocks there.

- 802.1BA gives an overview of AVB

- 802.1Q-2011 Sec 34 and 35 describes forwarding and queueing and Stream
Reservation (basically what the network needs in order to correctly
prioritize TSN streams)

- 802.1AS-2011 (gPTP) describes the timing in great detail (from a PTP
point of vew) and describes in more detail how the clocks should be
syntonized (802.1AS-2011, 7.3.3).

Since the clock that drives the sample-rate for the DA/AD must be
controlled by the shared clock, the fact that gPTP can adjust the time
means that the DA/AD circuit needs to be adjustable as well.

note that an adjustable sample-clock is not a *requirement* but in general
you'd want to avoid resampling in software.
Post by Takashi Sakamoto
(When considering about actual hardware codecs, on-board serial bus such
as Inter-IC Sound, corresponding controller, immediate change of
sampling rate is something imaginary for semi-realtime applications. And
the idea has no meaning for typical playback/capture softwares.)
Yes, and no. When you play back a stored file to your soundcard, data is
pulled by the card from memory. So you only have a single timing-domain to
worry about. So I'd say the idea has meaning in normal scenarios as well,
you don't have to worry about it.

When you send a stream accross the network, you cannot let the Listener
pull data from you, you have to have some common sense of time in order to
send just enough data, and that is why the gPTP domain is so important.

802.1Q gives you low latency through the network, but more importantly, no
dropped frames. gPTP gives you a central reference to time.
Post by Takashi Sakamoto
[1] [alsa-lib][PATCH 0/9 v3] ctl: add APIs for control element set
http://mailman.alsa-project.org/pipermail/alsa-devel/2016-June/109274.html
[2] IEEE 1722-2011
http://ieeexplore.ieee.org/servlet/opac?punumber=5764873
[3] 5.5 Timing and Synchronization
op. cit.
[4] 1394 Open Host Controller Interface Specification
http://download.microsoft.com/download/1/6/1/161ba512-40e2-4cc9-843a-923143f3456c/ohci_11.pdf
I hope this cleared some of the questions
--
Henrik Austad
Richard Cochran
2016-06-19 09:46:51 UTC
Permalink
Post by Henrik Austad
edit: this turned out to be a somewhat lengthy answer. I have tried to
shorten it down somewhere. it is getting late and I'm getting increasingly
incoherent (Richard probably knows what I'm talking about ;) so I'll stop
for now.
Thanks for your responses, Henrik. I think your explanations are on spot.
Post by Henrik Austad
note that an adjustable sample-clock is not a *requirement* but in general
you'd want to avoid resampling in software.
Yes, but..

Adjusting the local clock rate to match the AVB network rate is
essential. You must be able to *continuously* adjust the rate in
order to compensate drift. Again, there are exactly two ways to do
it, namely in hardware (think VCO) or in software (dynamic
resampling).

What you cannot do is simply buffer the AV data and play it out
blindly at the local clock rate.

Regarding the media clock, if I understand correctly, there the talker
has two possibilities. Either the talker samples the stream at the
gPTP rate, or the talker must tell the listeners the relationship
(phase offset and frequency ratio) between the media clock and the
gPTP time. Please correct me if I got the wrong impression...

Thanks,
Richard
Henrik Austad
2016-06-20 09:40:56 UTC
Permalink
Post by Richard Cochran
Post by Henrik Austad
edit: this turned out to be a somewhat lengthy answer. I have tried to
shorten it down somewhere. it is getting late and I'm getting increasingly
incoherent (Richard probably knows what I'm talking about ;) so I'll stop
for now.
Thanks for your responses, Henrik. I think your explanations are on spot.
Post by Henrik Austad
note that an adjustable sample-clock is not a *requirement* but in general
you'd want to avoid resampling in software.
Yes, but..
Adjusting the local clock rate to match the AVB network rate is
essential. You must be able to *continuously* adjust the rate in
order to compensate drift. Again, there are exactly two ways to do
it, namely in hardware (think VCO) or in software (dynamic
resampling).
Don't get me wrong, having an adjustable clock for the sampling is
essential -but it si not -required-.
Post by Richard Cochran
What you cannot do is simply buffer the AV data and play it out
blindly at the local clock rate.
No, that you cannot do that, that would not be pretty :)
Post by Richard Cochran
Regarding the media clock, if I understand correctly, there the talker
has two possibilities. Either the talker samples the stream at the
gPTP rate, or the talker must tell the listeners the relationship
(phase offset and frequency ratio) between the media clock and the
gPTP time. Please correct me if I got the wrong impression...
Last first; AFAIK, there is no way for the Talker to tell a Listener the
phase offset/freq ratio other than how each end-station/bridge in the
gPTP-domain calculates this on psync_update event messages. I could be
wrong though, and different encoding formats can probably convey such
information. I have not seen any such mechanisms in the underlying 1722
format though.

So a Talker should send a stream sampled as if the gPTP time drove the
AD/DA sample frequency directly. Whether the local sampling is driven by
gPTP or resampled to match gPTP-time prior to transmit is left as an
implementation detail for the end-station.

Did all that make sense?

Thanks!
--
Henrik Austad
Pierre-Louis Bossart
2016-06-20 11:14:20 UTC
Permalink
Post by Henrik Austad
Presentation time is either set by
a) Local sound card performing capture (in which case it will be 'capture
time')
b) Local media application sending a stream accross the network
(time when the sample should be played out remotely)
c) Remote media application streaming data *to* host, in which case it will
be local presentation time on local soundcard
Post by Takashi Sakamoto
This value is dominant to the number of events included in an IEC 61883-1
packet. If this TSN subsystem decides it, most of these items don't need
to be in ALSA.
Not sure if I understand this correctly.
TSN should have a reference to the timing-domain of each *local*
sound-device (for local capture or playback) as well as the shared
time-reference provided by gPTP.
Unless an End-station acts as GrandMaster for the gPTP-domain, time set
forth by gPTP is inmutable and cannot be adjusted. It follows that the
sample-frequency of the local audio-devices must be adjusted, or the
audio-streams to/from said devices must be resampled.
The ALSA API provides support for 'audio' timestamps (playback/capture
rate defined by audio subsystem) and 'system' timestamps (typically
linked to TSC/ART) with one option to take synchronized timestamps
should the hardware support them.
The intent was that the 'audio' timestamps are translated to a shared
time reference managed in userspace by gPTP, which in turn would define
if (adaptive) audio sample rate conversion is needed. There is no
support at the moment for a 'play_at' function in ALSA, only means to
control a feedback loop.
Henrik Austad
2016-06-20 11:50:37 UTC
Permalink
Post by Pierre-Louis Bossart
Post by Henrik Austad
Presentation time is either set by
a) Local sound card performing capture (in which case it will be 'capture
time')
b) Local media application sending a stream accross the network
(time when the sample should be played out remotely)
c) Remote media application streaming data *to* host, in which case it will
be local presentation time on local soundcard
Post by Takashi Sakamoto
This value is dominant to the number of events included in an IEC 61883-1
packet. If this TSN subsystem decides it, most of these items don't need
to be in ALSA.
Not sure if I understand this correctly.
TSN should have a reference to the timing-domain of each *local*
sound-device (for local capture or playback) as well as the shared
time-reference provided by gPTP.
Unless an End-station acts as GrandMaster for the gPTP-domain, time set
forth by gPTP is inmutable and cannot be adjusted. It follows that the
sample-frequency of the local audio-devices must be adjusted, or the
audio-streams to/from said devices must be resampled.
The ALSA API provides support for 'audio' timestamps
(playback/capture rate defined by audio subsystem) and 'system'
timestamps (typically linked to TSC/ART) with one option to take
synchronized timestamps should the hardware support them.
Ok, this sounds promising, and very much in line with what AVB would need.
Post by Pierre-Louis Bossart
The intent was that the 'audio' timestamps are translated to a
shared time reference managed in userspace by gPTP, which in turn
would define if (adaptive) audio sample rate conversion is needed.
There is no support at the moment for a 'play_at' function in ALSA,
only means to control a feedback loop.
Ok, I understand that the 'play_at' is difficult to obtain, but it sounds
like it is doable to achieve something useful.

Looks like I will be looking into what to put in the .trigger-handler in
the ALSA shim and experimenting with this to see how it make sense to
connect it from the TSN-stream.

Thanks!
--
Henrik Austad
Richard Cochran
2016-06-20 15:38:14 UTC
Permalink
Where is this "audio_time" program of which you speak?
Never mind, found it in alsa-lib.

I still would appreciate an answer to my other questions, though...

Thanks,
Richard
Takashi Iwai
2016-06-21 05:54:45 UTC
Permalink
On Mon, 20 Jun 2016 17:21:26 +0200,
Post by Richard Cochran
Where is this "audio_time" program of which you speak?
Never mind, found it in alsa-lib.
I still would appreciate an answer to my other questions, though...
Currently HD-audio (both ASoC and legacy ones) are the only drivers
providing the link timestamp. In the recent code, it's PCM
get_time_info ops, so you can easily grep it.


HTH,

Takashi
Richard Cochran
2016-06-21 06:39:21 UTC
Permalink
Post by Takashi Iwai
Post by Richard Cochran
I still would appreciate an answer to my other questions, though...
Currently HD-audio (both ASoC and legacy ones) are the only drivers
providing the link timestamp. In the recent code, it's PCM
get_time_info ops, so you can easily grep it.
Yes, I found that myself, thanks.
Post by Takashi Iwai
HTH,
No it doesn't help me, because I asked three questions, and none were
about the link timestamp.

Thanks,
Richard
Takashi Iwai
2016-06-21 06:46:17 UTC
Permalink
On Tue, 21 Jun 2016 08:38:57 +0200,
Post by Richard Cochran
Post by Takashi Iwai
Post by Richard Cochran
I still would appreciate an answer to my other questions, though...
Currently HD-audio (both ASoC and legacy ones) are the only drivers
providing the link timestamp. In the recent code, it's PCM
get_time_info ops, so you can easily grep it.
Yes, I found that myself, thanks.
Post by Takashi Iwai
HTH,
No it doesn't help me, because I asked three questions, and none were
about the link timestamp.
?? The extended audio timpestamp is essentially to return the link
timestamp. Just the term has changed along time...


Takashi
Pierre-Louis Bossart
2016-06-21 17:19:55 UTC
Permalink
1. DMA timestamp, no compensation for DMA+analog delay
$ ./audio_time -p --ts_type=1
Where is this "audio_time" program of which you speak?
alsa-lib/test
Pierre-Louis Bossart
2016-06-21 17:49:41 UTC
Permalink
The ALSA API provides support for 'audio' timestamps (playback/capture rate
defined by audio subsystem) and 'system' timestamps (typically linked to
TSC/ART) with one option to take synchronized timestamps should the hardware
support them.
Thanks for the info. I just skimmed Documentation/sound/alsa/timestamping.txt.
That is fairly new, only since v4.1. Are then any apps in the wild
that I can look at? AFAICT, OpenAVB, gstreamer, etc, don't use the
new API.
The ALSA API supports a generic .get_time_info callback, its
implementation is for now limited to a regular 'DMA' or 'link' timestamp
for HDaudio - the difference being which counters are used and how close
they are to the link serializer. The synchronized part is still WIP but
should come 'soon'
The intent was that the 'audio' timestamps are translated to a shared time
reference managed in userspace by gPTP, which in turn would define if
(adaptive) audio sample rate conversion is needed. There is no support at
the moment for a 'play_at' function in ALSA, only means to control a
feedback loop.
If supported in hardware, the absolute link time could also be used
to define a precise start time (patches WIP)
1. Where are the patches? (If some are coming, I would appreciate
being on CC!)
2. Can you mention specific HW that would support this?
You can experiment with the 'dma' and 'link' timestamps today on any
HDaudio-based device. Like I said the synchronized part has not been
upstreamed yet (delays + dependency on ART-to-TSC conversions that made
it in the kernel recently)
Richard Cochran
2016-06-21 21:18:49 UTC
Permalink
Post by Pierre-Louis Bossart
You can experiment with the 'dma' and 'link' timestamps today on any
HDaudio-based device. Like I said the synchronized part has not been
upstreamed yet (delays + dependency on ART-to-TSC conversions that made it
in the kernel recently)
Can you point me to any open source apps using the dma/link
timestamps?

Thanks,
Richard
Pierre-Louis Bossart
2016-06-22 12:38:18 UTC
Permalink
Post by Richard Cochran
Post by Pierre-Louis Bossart
You can experiment with the 'dma' and 'link' timestamps today on any
HDaudio-based device. Like I said the synchronized part has not been
upstreamed yet (delays + dependency on ART-to-TSC conversions that made it
in the kernel recently)
Can you point me to any open source apps using the dma/link
timestamps?
Those timestamps are only used in custom applications at the moment, not
'mainstream' open source.
It takes time for new kernel capabilities to trickle into userspace,
applications usually align on the lowest hardware common denominator. In
addition, most applications don't access the kernel directly but go
through an audio server or HAL which needs to be updated as well so it's
a two-level dependency. These timestamps can be directly mappped to the
Android AudioTrack.getTimeStamp API though.
Henrik Austad
2016-06-23 10:39:01 UTC
Permalink
The ALSA API provides support for 'audio' timestamps (playback/capture rate
defined by audio subsystem) and 'system' timestamps (typically linked to
TSC/ART) with one option to take synchronized timestamps should the hardware
support them.
Thanks for the info. I just skimmed Documentation/sound/alsa/timestamping.txt.
That is fairly new, only since v4.1. Are then any apps in the wild
that I can look at? AFAICT, OpenAVB, gstreamer, etc, don't use the
new API.
The ALSA API supports a generic .get_time_info callback, its implementation
is for now limited to a regular 'DMA' or 'link' timestamp for HDaudio - the
difference being which counters are used and how close they are to the link
serializer. The synchronized part is still WIP but should come 'soon'
Interesting, would you mind CCing me in on those patches?
The intent was that the 'audio' timestamps are translated to a shared time
reference managed in userspace by gPTP, which in turn would define if
(adaptive) audio sample rate conversion is needed. There is no support at
the moment for a 'play_at' function in ALSA, only means to control a
feedback loop.
If supported in hardware, the absolute link time could also be used
to define a precise start time (patches WIP)
1. Where are the patches? (If some are coming, I would appreciate
being on CC!)
2. Can you mention specific HW that would support this?
You can experiment with the 'dma' and 'link' timestamps today on any
HDaudio-based device. Like I said the synchronized part has not been
upstreamed yet (delays + dependency on ART-to-TSC conversions that made it
in the kernel recently)
Ok, I think I see a way to hook this into timestamps from the skbuf on
incoming frames and a somewhat messy way on outgoing. Having time coupled
with 'avail' and 'delay' is useful, and from the looks of it, 'link'-time
is the appropriate level to add this.

I'm working on storing the time in the tsn_link struct I use, and then read
that from the avb_alsa-shim. Details are still a bit fuzzy though, but I
plan to do that and then see what audio-time gives me once it is up and
running.

Richard: is it fair to assume that if ptp4l is running and is part of a PTP
domain, ktime_get() will return PTP-adjusted time for the system? -Or do I
also need to run phc2sys in order to sync the system-time to PTP-time? Note
that this is for outgoing traffic, Rx should perhaps use the timestamp
in skb.

Hooking into ktime_get() instead of directly to the PTP-subsystem (if that
is even possible) makes it a lot easier to debug when running this in a VM
as it doesn't *have* to use PTP-time when I'm crashing a new kernel :)

Thanks!
--
Henrik Austad
Richard Cochran
2016-06-23 13:28:18 UTC
Permalink
Post by Henrik Austad
Richard: is it fair to assume that if ptp4l is running and is part of a PTP
domain, ktime_get() will return PTP-adjusted time for the system?
No.
Post by Henrik Austad
Or do I also need to run phc2sys in order to sync the system-time
to PTP-time?
Yes, unless you are using SW time stamping, in which case ptp4l will
steer the system clock directly.

HTH,
Richard

Takashi Sakamoto
2016-06-15 03:27:56 UTC
Permalink
Hi Richard,
Post by Richard Cochran
Well, I guess I should have said, I am not too familiar with the
breadth of current audio hardware, high end or low end. Of course I
would like to see even consumer devices work with AVB, but it is up to
the ALSA people to make that happen. So far, nothing has been done,
afaict.
In OSS world, there's few developers for this kind of devices, even if
it's alsa-project. Furthermore, manufacturerer for recording equipments
have no interests in OSS.

In short, what we can do for these devices is just to
reverse-engineering. For models of Ethernet-AVB, it might be just to
transfer or receive packets, and read them. The devices are still
black-boxes and we have no ways to reveal their details.

So when you require the details to implement something in your side, few
developers can tell you, I think.


Regards

Takashi Sakamoto
Loading...