The most common and simplest interface to a computer and operating system is a command-line shell. It allows for quick operations, the ability to automate operations through scripting, and is light on bandwidth usage.
Thank you for this post, @tyler! I love the follow-up step of wrapping this into invoke scripts 
Among other opportunities you mentioned
receive files from a device’s file system
Are you aware of a standard way of doing this (e.g. file size and suggested file name upfront, transfer of multiple files at once, maybe even compression)? In the past, we were particularly struggling with log messages being interleaved with logs from unrelated sources. In our system, logs (e.g. crash dumps of processes) could even produce a file “spontaneously” at the same time one would interact with the CLI. We ultimately decided to store those files away instead of logging them (and later fetching them using a different method).
I recognize that this could be too unrelated to your use-case. In that case, would you recommend trying to isolate the CLI output from everything else during an interaction to make the CLI use-case itself as simple as possible?
It’s a fantastic question, and one that is definitely relevant. I hope the community can give their inputs as well if they have any solutions!
I’ve seen a few ways to do this.
Forfeit the idea that both the serial console and a file transfer can happen at the same time. To send or receive files, you need to send a command to the device that tells it to kill any interactivity or printing to the shell, and then send or receive a raw stream of bytes.
To send a file to the device’s filesystem, something like:
fw_shell$ fs_receive <file_name> <file_size>
where file_name is the name of the file to save on the device’s firmware, such as littlefs. This would be wrapped locally with a Python script which would drive the process of reading the file from the host machine over the serial port (using PySerial).
Then, to receive a file, similary
fw_shell$ fw_send <file_name>
where file_name is again the name of the file on the devices filesystem. There would again be a Python script driving this, and would read the entire contents of the stream from the device into a file specified.
$ invoke fs.receive --name <fs_file_name> --output <host_file_name>
$ invoke fs.send --name <fs_file_name> --input <host_file_name>
The above method has previously been used to decent success at a previous employer, but obviously it’s limited. It does not contain compression and the shell would otherwise be inoperable while a file transfer is taking place. It also relies on the fact that each and every byte is received and is in tact, which isn’t a guarantee that is easily met.
Do the above, but use a known simple transfer library on top of it. I’ve seen Kermit and (X/Z)Modem used in the context of embedded devices. They perform some error checking and light compression on the data, which helps. Nothing preventing a developer from doing simple RLE compression as well!
Unfortunately, I haven’t seen many public implementations of this.
Build a simple framed protocol on top of the UART, where each command is prefixed with a protocol number. Unfortunately, this would make using the shell from a simple serial library quite difficult, but creating a wrapper with PySerial’s Miniterm would not be out of the question.
+1 for ZMODEM. It’s standard (on Mac, the Serial app supports it) and works reasonably well.
Here’s an open source implementation bundled with the RT-Thread OS: https://github.com/RT-Thread/rt-thread/tree/master/components/utilities/zmodem
I think there’s a bug in this section:
int cli_cmd_kv_write(int argc, char *argv[]) {
// We expect 3 arguments:
// 1. Command name
// 2. Key
// 3. Value
if (argc != 3) {
shell_put_line("> FAIL,1");
}
const char *key = argv[1];
const char *value = argv[2];
bool result = kv_store_write(key, value, strlen(value));
if (!result) {
shell_put_line("> FAIL,2");
}
shell_put_line("> OK");
return 0;
}
The part that has shell_put_line("> FAIL, 1"); should probably return afterwards otherwise you’ll be accessing past the bounds of the array. The other fail case probably needs to return as afterwards as well, though less serious consequences 
Oops! Good catch, On a previous project, this was so common I believe we had a macro for it, something like FAIL_AND_RETURN(1); 
Thanks for the report. I’ve pushed a fix.https://github.com/memfault/interrupt/commit/21737def3ede5f3093f22c2f756cd1ea33aacfda
Thanks for the excellent post! I would like to use this shell in an open-source project I’m working on right now. It looks like the interrupt github repository is licensed under cc by-sa, which I believe requires attribution. To whom should I accredit these files? “Memfault, Inc.”?
Yes, Memfault, Inc. should work. We also plan on placing the shell code in a separate repo with a more standard source code license soon.
There is an overflow by 1 // off by one error in your first example of console_gets.
int console_gets(char *s, int len) {
char *t = s;
char c;
*t = '\000';
/* read until a <LF> is received */
while ((c = console_getc()) != '\n') {
*t = c; // this overflows when t-s == len
console_putc(c);
if ((t - s) < len) {
t++;
}
/* update end of string with NUL */
*t = '\000'; // this overflows when t-s == len
}
return t - s;
}
Simple example assumption buffer of one length. First char is put at index 0, t - s is then smaller then len, t will be incremented and from that moment on all writes are writing at index 1.
I am having trouble locating the ‘sdk_common.h’ needed to make the executable. I searched the firmware-shell directory for the file and couldn’t find it.
Is there a step I am not understanding in the build process?
p.s. I enjoyed reading your posts so much I bought the dev board you’re using to try out the examples. It’s been a steep learning curve so far, but it’s a good challenge!
Hello @csluiter! Welcome.
It actually is included in the nRF5 SDK that Nordic includes. The instructions aren’t front and center in the post, but they can be found on the Github example page: https://github.com/memfault/interrupt/tree/master/example/firmware-shell#setup
$ cd nRF5_SDK_15.3.0_59ac345/
$ find . -name "sdk_common.h"
./components/libraries/util/sdk_common.h
p.s. I enjoyed reading your posts so much I bought the dev board you’re using to try out the examples. It’s been a steep learning curve so far, but it’s a good challenge!
That’s awesome to hear! The nRF52 board is great and easy to use. You won’t be disappointed.
Ah, found it. Thanks for taking time to answer a beginner’s question.
Thanks for the great article and tip regarding OK or FAIL response for automation 
CLI shells for embedded systems are great!
Shameless plug: if someone’s not sold on the idea yet and have access to an Arduino Uno then they can try this CLI shell. Documentation HERE
Regarding file transfers, I use XMODEM. Documentation HERE
Regards,
Pieter
I really enjoyed the tutorial and the code. I believe I found a bug in shell_receive_char:
if (c == '\b') {
s_shell.rx_buffer[--s_shell.rx_size] = '\0';
return;
}
There are no checks for underflow here. When it does underflow, it will corrupt one byte somewhere in memory. After that, the buffer will appear full and will not accept any more characters, locking up the shell indefinitely.