Modern USB gadget on Linux & how to integrate it with systemd (Part 2)

Andrzej Pietrasiewicz avatar

Posted on 27/03/2019 by Andrzej Pietrasiewicz

Share this post:

In the previous post I introduced you to the subject of USB gadgets implemented as machines running Linux. We also talked about modern style of USB gadget creation and integrating that with systemd. In this post, we look at how to implement your very own USB function with FunctionFS and how to integrate that with systemd.

The general idea of FunctionFS

The idea is to delegate actual USB function implementation to userspace, using a filesystem interface (read()/write() etc). This way, USB out traffic (from host) is available at a file descriptor ready for reading, and USB in traffic (to host) is accepted at a file descriptor ready for writing.

What you need to know before using FunctionFS

In USB all communication is performed through so called endpoints. At the device side the endpoints are (hardware) FIFO queues associated with the UDC. There are 4 types of USB endpoints: control, bulk, iso and interrupt. Control endpoint is endpoint 0 and all USB devices shall have at least endpoint 0. This is a bi-directional communication channel between the host and the device used for enumerating and then controlling the device - remember, the USB is a host centric bus. So even if a device has some new data to be sent to the host it is the host that actually asks for data and the communication required to arrange this happens on the endpoint 0. All other types of endpoints are uni-directional. Bulk endpoints are meant for tansferring (potentially) large amounts of data on a "best effort" basis, that is, as available bandwith allows, so there are no timing guarantees for bulk data.

On the other hand bulk data is guaranteed to be transmitted error-free (perhaps using re-transmission under the hood). Iso(chronous) endpoints are meant for time-sensitive data (such as audio or video) with guaranteed bandwitdh and bounded latency. Neither data delivery nor integrity is guarenteed, though, and it is on purpose: in multimedia transmission much more harm is done by non-timely data delivery rather than by occasional non-integrity or even loss of a frame. Interrupt endpoints are for non-periodic data transfers "initiated" by the device (such as mouse movements or keystrokes) and have guaranteed latency. The "initiation" by the device is in fact queuing the data on an endpoint, and then the host polls it when it considers appropriate. In USB it's a host's world. Get used to it or quit using USB.

Why am I telling you this? You need to know that each USB function requires its own set of endpoints, which must not be used at the same time by any other function. This makes UDC's endpoints a scarce resource and imposes a limit on a number of functions which can be provided by your gadget at a time (in one configuration, but you can "overcommit" by specifying multiple configurations - you still remember, that only one of them can be active at a time?). If you want to roll your own USB function, you need to know how many and what kind of endpoints you need, and, especially if you want to compose it with other functions, whether the number of available endpoints is not exceeded. Modern UDCs usually do provide enough endpoints to compose 2-3 moderately "endpoint-hungry" functions into one configuration without any problems.

More detailed idea of FunctionFS

Now that you know about endpoints, you can learn more about FunctionFS. From the point of view of the gadget it is yet another USB function available for composing a gadget from. But it is special, because each instance of FunctionFS provides an instance of a 'functionfs' filesystem to be mounted. If you mount it you will notice there is only one entry: ep0. Its name suggests it is associated with endpoint 0. Indeed it is. However, the good news is that you don't need to implement all the endpoint 0 handling, because most of that is already taken care of for you by the composite layer. You only need to handle control requests directed at your particular USB function. ep0 is also (ab)used to specify the endpoints and USB strings. FunctionFS is not considered ready for binding until endpoint descriptors and strings are written to ep0. The gadget as a whole is not ready for binding if any of its functions is not ready. After the descriptors and strings are written, FunctionFS creates appropriate number of ep<number> files for you to use to implement the desired data streams between the host and the device.

An example function can be found in kernel sources: tools/usb/ffs-test.c The file also contains an example of how to specify USB descriptors and strings to be written to ep0 to make FunctionFS ready. Actually I'm using its modified and simplified version, which passes to the host whatever the host writes to it. We will get to its code later in this post.

And how to integrate THAT with systemd?

Compared to the previous scenario there are more steps involved before the gadget becomes operational:

  • load gadget scheme and don't enable yet (because it contains FunctionFS)
  • mount FunctionFS instance
  • write descriptors and strings
  • start the userspace daemon (e.g. ffs-test)
  • enable the gadget (now that it's ready, bind it to UDC)

It turns out that systemd already has means to automate writing the descriptors and strings to FunctionFS, and starting a userspace daemon! However, the remaining steps need to be taken care of.

The scheme

Let's call the scheme ffs_test.scheme. It describes a minimal gadget with FunctionFS, leaving some attributes at their default values:

 attrs : { idVendor = 0xABCD; idProduct = 0x1234; }; strings = ( ); functions : { ffs_loopback : { instance = "loopback"; type = "ffs"; }; }; configs = ( { id = 1; name = "c"; functions = ( { name = "ffs.loopback"; function = "ffs_loopback"; } ); } ); 

One important thing to note is that the instance name (here: "loopback") becomes a "device" name to be used when mounting this FunctionFS instance.

The service

Let's call our example service unit usb-gadget-ffs.service. This unit is almost identical to the usb-gadget.service, except the ExecStart line:

ExecStart=/bin/gt load -o ffs_test.scheme ffs_test

The difference is that we tell gt to only (-o) load gadget's composition but not bind it, which would fail anyway because the FunctionFS instance is not ready yet.

Another difference is that the Type cannot be "simple" this time, because we need systemd not to schedule loading of dependent modules until gadget scheme is fully loaded - and, consequently, FunctionFS instance registered and made available for mounting. Units of type "simple" are considered completely run immediately, so we need to change to:


This change ensures that the dependent modules will be started only after full completion of the gt command.

Complete usb-gadget-ffs.service code:

[Unit] Description=Load USB gadget scheme Requires=sys-kernel-config.mount After=sys-kernel-config.mount [Service] ExecStart=/bin/gt load -o ffs_test.scheme ffs_test RemainAfterExit=yes ExecStop=/bin/gt rm -rf ffs_test Type=oneshot [Install] 

The mount

We can use a mount unit to automatically mount our FunctionFS instance. We choose to mount it at /run/ffs_test, so the mount unit must be named run-ffs_test.mount:

[Unit] Description=Mount FunctionFS instance Requires=usb-gadget-ffs.service After=usb-gadget-ffs.service Before=ffs.socket [Mount] # "device" name (FunctionFS instance name) What=loopback Where=/run/ffs_test Type=functionfs Options=defaults TimeoutSec=5 [Install] 

and then use systemctl enable run-ffs_test.mount

The above two units are enough to load our gadget composition into memory at UDC's appearance and mount its accompanied FunctionFS instance.

Descriptors, strings and the userspace daemon

systemd supports socket units capable of listening to usb traffic directed at FunctionFS. Such a socket unit must be pointed at an already mounted FunctionFS instance. The socket unit contains ListenUSBFunction entry in its [Socket] section to specify exactly that. It also contains a Service entry pointing to the service unit associated with this unit and such a socket unit must have its associated service unit.

When the socket unit is started, it starts its associated service unit, which contains USBFunctionDescriptors and USBFunctionStrings entries. The two specify locations of files containing binary blobs representing USB descriptors and strings, respectively. The service unit's job is then to write those to ep0 of the corresponding FunctionFS instance. And here the systemd's lazy daemon start comes into play: the descriptors and strings are already passed to FunctionFS, so it becomes ready _but_ the daemon is _not_ started until some traffic directed at this FunctionFS instance happens on USB. And it is exactly the job of the socket unit to make it work. The daemon binary is specified with the usual ExecStart entry in the service unit.

The socket unit

Let's call it ffs.socket:

[Unit] Description=USB function fs socket Requires=run-ffs_test.mount After=run-ffs_test.mount DefaultDependencies=no [Socket] ListenUSBFunction=/run/ffs_test Service=functionfs-daemon.service # we will get to ExecStartPost later ExecStartPost=/bin/gt enable ffs_test [Install] 

And then systemctl enable functionfs.socket.

The service unit accompanying the socket unit

We call it functionfs-daemon.service as per the Service entry above.

[Service] ExecStart=/root/bin/ffs-test USBFunctionDescriptors=/root/descriptors-ffs-test.bin USBFunctionStrings=/root/strings-ffs-test.bin 

A question you likely ask yourself is where to get the .bin files from? include/uapi/linux/usb/functionfs.h in the kernel sources provides the format description (line 89 in v5.0-rc6). It is up to you how you create these blobs. An example of how to create the descriptors (and strings) in a C program can be found in tools/usb/ffs-test.c. Please note that you are supposed to use usb_functionfs_descs_head_v2, because the other format is now deprecated. You can modify the program so that it doesn't do anything except writing the descriptors/strings to standard output and then capture the result in a file.

Here is a hex dump of descriptors and strings blobs for a high-speed-only example funtion:

# hd descriptors-ffs-test.bin 00000000 03 00 00 00 27 00 00 00 02 00 00 00 03 00 00 00 |....'...........| 00000010 09 04 00 00 02 ff 00 00 01 07 05 81 02 00 02 00 |................| 00000020 07 05 02 02 00 02 01 |.......| 00000027 # hd strings-ffs-test.bin 00000000 02 00 00 00 1d 00 00 00 01 00 00 00 01 00 00 00 |................| 00000010 09 04 55 53 42 20 46 69 6c 74 65 72 00 |..USB Filter.| 0000001d 

Please note that the descriptors are created only in the unmodified version of ffs-test.c. As promised, we will get to the modified version later in this post.

Enabling the gadget

If the functionfs.socket did not contain the ExecStartPost entry, then at this point we would have a gadget ready to be bound, but not actually bound. The ExecStartPost contains a command which executes our gadget binding _after_ the service unit is started, which is the last missing piece in this puzzle. Your gadget is now composed and activated with systemd.

Can I use existing daemon code to integrate with sytsemd?

No, you can't. However, usually patching userspace code to be able to be passed open file descriptors by systemd is a trivial task. And to make it even easier for you to play with FunctionFS and systemd I have modified the ffs-test.c program for using with systemd (it purposedly contains some simplifications in order for you to be able to follow the code easily, so you can't take it as a full-fledged implementation):

look for tools/usb/ffs-test.c on the ffs-systemd branch.

Giving it a try

In order to test the above mentioned modified function you need the following:

  • Scheme of the example gagdet from this post
  • All systemd units from this post
  • ffs-test binary compiled from the modified sources
  • FunctionFS endpoint descriptors and strings binary blobs
  • usbserial module at host with generic serial support

Set up your gadget and connect it to the host, it should enumerate correctly. The idea is to bind generic usb serial driver at the host to our device. This results in automatic creation of ttyUSB<X> at the host side. And then whatever you write to the ttyUSB<X> you can read back from it. That's what the modified ffs-test.c does - a loopback function.

# at the host echo 0xabcd 0x1234 > /sys/bus/usb-serial/drivers/generic/new_id dmesg <...> usbserial_generic 3- The "generic" usb-serial driver is only for testing and one-off prototypes. usbserial_generic 3- Tell to add your device to a proper driver. usbserial_generic 3- generic converter detected usb 3- generic converter now attached to ttyUSB1 while true; do dd if=/dev/ttyUSB1 bs=1 iflag=nocache status=none 2>/dev/null; done 

And still at the host, at some other console:

man -P cat bash > /dev/ttyUSB1

The contents of bash man page should appear at the console where the "while" loop runs. You can further modify ffs-test.c to e.g. capitalize all the letters, or use some encryption algorithm to cipher your text, effectively creating a hardware crypto device on USB!

What's next?

In the next installment I will write about using dummy_hcd. And in yet another one we will add some systemd templatization to make the above solutions more generic.