Building a Tiny CLI Shell for Tiny Firmware | Interrupt

[tyler]

@hbehrens

Are you aware of a standard way of doing this (e.g. file size and suggested file name upfront, transfer of multiple files at once, maybe even compression)? In the past, we were particularly struggling with log messages being interleaved with logs from unrelated sources. In our system, logs (e.g. crash dumps of processes) could even produce a file “spontaneously” at the same time one would interact with the CLI. We ultimately decided to store those files away instead of logging them (and later fetching them using a different method).

I recognize that this could be too unrelated to your use-case. In that case, would you recommend trying to isolate the CLI output from everything else during an interaction to make the CLI use-case itself as simple as possible?

It’s a fantastic question, and one that is definitely relevant. I hope the community can give their inputs as well if they have any solutions!

I’ve seen a few ways to do this.

1. Forfeit the idea that both the serial console and a file transfer can happen at the same time. To send or receive files, you need to send a command to the device that tells it to kill any interactivity or printing to the shell, and then send or receive a raw stream of bytes.

   To send a file to the device’s filesystem, something like:

   fw_shell$ fs_receive <file_name> <file_size>
   

   where file_name is the name of the file to save on the device’s firmware, such as littlefs. This would be wrapped locally with a Python script which would drive the process of reading the file from the host machine over the serial port (using PySerial).

   Then, to receive a file, similary

   fw_shell$ fw_send <file_name>
   

   where file_name is again the name of the file on the devices filesystem. There would again be a Python script driving this, and would read the entire contents of the stream from the device into a file specified.

   $ invoke fs.receive --name <fs_file_name> --output <host_file_name>
   $ invoke fs.send --name <fs_file_name> --input <host_file_name>
   

   The above method has previously been used to decent success at a previous employer, but obviously it’s limited. It does not contain compression and the shell would otherwise be inoperable while a file transfer is taking place. It also relies on the fact that each and every byte is received and is in tact, which isn’t a guarantee that is easily met.

2. Do the above, but use a known simple transfer library on top of it. I’ve seen Kermit and (X/Z)Modem used in the context of embedded devices. They perform some error checking and light compression on the data, which helps. Nothing preventing a developer from doing simple RLE compression as well!

   Unfortunately, I haven’t seen many public implementations of this.

3. Build a simple framed protocol on top of the UART, where each command is prefixed with a protocol number. Unfortunately, this would make using the shell from a simple serial library quite difficult, but creating a wrapper with PySerial’s Miniterm would not be out of the question.