Better Firmware with LLVM/Clang | Interrupt

If you have ever done software development that involves Apple products, FreeBSD, WebAssembly, or Rust, you have used the wildly popular compiler toolchain known as LLVM. However, LLVM and the open source C-language compiler built on top of it (Clang) do not get a lot of attention in the embedded world.


This is a companion discussion topic for the original entry at https://interrupt.memfault.com/blog/arm-cortexm-with-llvm-clang
1 Like

That’s a great post! I have also tried to experiment with Clang few months ago to solve the problem related to iostream C++ STL header.

The problem is that if one would like to include iostream or sstream to implement e.g. nice streaming to a logger, or to some other peripheral, using operator<< for example, then the binary size increases so much, that it will not fit into Flash memory.

This is really annoying, because I can’t simply use std::ostringstream, to reuse easy string conversion, and I need to add quite a lot of boilerplate to be able to convert various types to string, using e.g. std::to_string.

As I researched, I have found out that it is caused by the standard library bundled with GCC. After iostream is included some static objects are instantiated, even though they are not used at all.

In this post it is explained that the STL from ARM Toolchain is used. Do you know whether the use of the lld or the -Oz flag resolves the problem related to too big size of the binary?

When I try even a simple program, which had main() and used std::ostringstream to stringify some numbers, and was compiled for a ARM Cortex M CPU (with e.g. 256 kB of Flash), the Flash area was overflown.

I hope that this: https://interrupt.memfault.com/blog/arm-cortexm-with-llvm-clang#compiling-libclang_rtbuiltinsa-for-arm-cortex-m
also resolves the problem.

Normally I would try it on my own, but currently I don’t have a set up to try it out.

Thanks for the post, really interesting. I gave this a try (had to make some tweaks to my CMake builds first) but can’t seem to get it to work properly.

I get a bunch of errors similar to this:

In file included from /Users/kamil/xxxxx/../Drivers/CMSIS/CMSIS/Include/cmsis_compiler.h:48:
/Users/kamil/xxx/../Drivers/CMSIS/CMSIS/Include/cmsis_gcc.h:868:19: error: invalid instruction mnemonic 'isb'
  __ASM volatile ("isb 0xF":::"memory");
                  ^
<inline asm>:1:2: note: instantiated into assembly here
        isb 0xF
        ^~~

In any parts of the code that are trying to compile the hardware drivers (from ST in this case)

Hi @kisielk,

Glad to hear you enjoyed the article! The error you hit above suggests you may not be targeting the right architecture. What CFLAGs are you using? (For example, if you are compiling for a cortex M4, you want to make sure you have something like -mthumb -mcpu=cortex-m4).

I’m using all the same flags I normally use with gcc.

The a typical compile command looks something like this:

/Users/kamil/tmp/clang+llvm-10.0.0-x86_64-apple-darwin/bin/clang --sysroot=/usr/local/bin/../arm-none-eabi -DARM -DSTM32F401VETx -DSTM32F401xC -DSTM32F4xx -DUSE_FULL_LL_DRIVER -I/Users/kamil/xxx/cmake/../Drivers/STM32F4xx_HAL_Driver/Inc -I/Users/kamil/xxx/Project/Inc -I/Users/kamil/xxx/cmake/../Drivers/CMSIS/Device/ST/STM32F4xx/Include -I/Users/kamil/xxx/cmake/../Drivers/CMSIS/CMSIS/Include -Og -DDEBUG -g -gdwarf-2 -fdata-sections -ffunction-sections -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mcpu=cortex-m4 -o CMakeFiles/stm32hal.dir/Users/kamil/xxx/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c.o -c /Users/kamil/xxx/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c

I noticed I also get these warnings during the process:

clang-10: **warning:** argument unused during compilation: '-mthumb' [-Wunused-command-line-argument]

clang-10: **warning:** argument unused during compilation: '-mfloat-abi=hard' [-Wunused-command-line-argument]

clang-10: **warning:** argument unused during compilation: '-mfpu=fpv4-sp-d16' [-Wunused-command-line-argument]

clang-10: **warning:** argument unused during compilation: '-mcpu=cortex-m4' [-Wunused-command-line-argument]

Ah, looks like you are missing --target=arm-none-eabi to tell clang to cross-compile which will result in those warnings and the assembly one you are seeing.

Hey @KKoovalsky, you’re right that using the STL on embedded in general is tricky. I don’t use C++ much myself, but Phillip at Embedded Artistry does and writes about it. Check out his article about the Embedded Templates Library: https://embeddedartistry.com/blog/2018/12/13/embedded-template-library/. It might be what you’re looking for!

Ah ha, I was missing some flags. I think I’ve got that sorted out now, but I’m hitting a snag with includes:

cd /Users/kamil/xxx/Project/sbuild/haldrivers && /Users/kamil/tmp/clang+llvm-10.0.0-x86_64-apple-darwin/bin/clang++ --sysroot=/usr/local/Cellar/arm-none-eabi-gcc/9-2019-q4-major/gcc/bin/../arm-none-eabi  -DARM -DSTM32F401VETx -DSTM32F401xC -DSTM32F4xx -DUSE_FULL_LL_DRIVER -I/Users/kamil/xxx/lib/haldrivers/.. -I/Users/kamil/xxx/lib/haldrivers/../../Drivers/STM32_USB_Device_Library/Core/Inc -I/Users/kamil/xxx/lib/haldrivers/usb_midi -I/Users/kamil/xxx/lib/haldrivers/../lib -I/Users/kamil/xxx/cmake/../Drivers/STM32F4xx_HAL_Driver/Inc -I/Users/kamil/xxx/Project/Inc -I/Users/kamil/xxx/cmake/../Drivers/CMSIS/Device/ST/STM32F4xx/Include -I/Users/kamil/xxx/cmake/../Drivers/CMSIS/CMSIS/Include  -Og -fno-exceptions -DDEBUG -g -gdwarf-2 -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mcpu=cortex-m4 -fdata-sections -ffunction-sections -mthumb --target=arm-none-eabi -fno-rtti -Wno-register   -Wall -std=gnu++17 -o CMakeFiles/haldrivers.dir/assert.cpp.o -c /Users/kamil/xxx/lib/haldrivers/assert.cpp
/Users/kamil/xxx/lib/haldrivers/assert.cpp:1:10: fatal error: 'cstdint' file not found

I think you’ll want to add the -stdlib=libc++ flag to fix that

Hm, I tried adding that but it just gives clang-10: **warning:** argument unused during compilation: '-stdlib=libc++' [-Wunused-command-line-argument]

I wonder if it’s because CMake is adding -std=gnu++17 instead of -std=c++17

Another great post at Interrupt, thank you @chrisc! This is the first time I am reading about ownership_returns and ownership_takes and it looks like some powerful tool beyond malloc-like functions (especially in combination with ownership_holds).

I failed to find real-world examples on Github. Are you aware of any non-trivial examples I could learn from?

2 Likes

Thanks @hbehrens!

Unfortunately I have not found any examples either. I think today it’s really only used within the llvm-project for the unix MallocChecker but agree there seems like a lot of potential for other types of passes with the annotation in general!

Something I’d like to try is implementing a custom plugin that uses the ownership_ type attribute.

1 Like

Great article! The majority of my company’s firmware (with the exception of PIC-based systems) is developed & shipped with clang; when we switched 5-6 years ago the primary motivation was the superior quality of its diagnostic messages compared to GCC. I understand GCC’s output has improved drastically since then, but we haven’t had any practical reasons to reconsider that decision. We’re currently building with c++17 and I’m hoping to get clang 10 up and running soon so we can take advantage of some c++20 features.

I think clang-tidy also belongs in this conversation: it includes the static analyzer checks performed by scan-build, and additionally can perform a whole raft of checks against various coding & style guidelines including the C++ core guidelines. Running it is a little bit more involved than scan-build as it works best with a compile_commands database and introducing it to an existing codebase can be a big task, but it’s got some great checks: https://releases.llvm.org/10.0.0/tools/clang/tools/extra/docs/clang-tidy/checks/list.html

3 Likes

Welcome to Interrupt @gdharris! Exciting to hear from more folks using clang on embedded. We had evaluated clang 5-6 years ago, but we found that it wasn’t quite as good at optimizing for code size then. Was that a concern for your use case?

Thank you for bringing up clang-tidy. I have not had the change to try it, but I will. If you’re ever inspired to write a short post about it, we’d love to publish it :-).

Hi I follow your posts regularly. I am interested in converting some of my gcc projects to clang. This article is a great starting point. However, I am stuck early on in the scan-build step. I am getting the following errors:
~/.../example/freertos-example-llvm >>> scan-build make
scan-build: Using '/usr/bin/clang-10' for static analysis
/usr/bin/../lib/clang/ccc-analyzer
===GCC Compiler Detected===
Compiling src/builtin_scanbuild_examples.c
gcc: warning: ‘-mcpu=’ is deprecated; use ‘-mtune=’ or ‘-march=’ instead
gcc: error: unrecognized command line option ‘-mthumb’
gcc: error: unrecognized command line option ‘-mfloat-abi=hard’
gcc: error: unrecognized command line option ‘-mfpu=fpv4-sp-d16’
make: *** [Makefile:167: /home/rajah/workspace/interrupt/example/freertos-example- llvm/build/src/builtin_scanbuild_examples.o] Error 1
scan-build: Removing directory '/tmp/scan-build-2020-05-28-170744-12711-1' because it contains no reports.
scan-build: No bugs found.
~/.../example/freertos-example-llvm >>>

I have installed clang 10 on arch and I also have arm-none-eabi-gcc installed. What am I missing? Thanks in advance for any pointers.

Rajah

Hi @mcrajah,

Under the hood scan-build works by searching for a compiler to use, overriding CC in the makefile, and then running a static analysis pass with Clang. In your case it looks like it found the wrong compiler (“gcc” instead of “arm-none-eabi-gcc” or “clang”) which is why you are seeing the error.

Can you try using the --use-cc argument with scan-build to tell it the compiler to exact compiler to use instead: i.e scan-build --use-cc=<path_to_clang>/clang make?

Thanks for the question, I’ll add a note to the article about it!

Hi Chris,
Thank you very much for the prompt response!!
I was able to get it working (mostly) with the -use-cc switch. After compile, I got a linker error.

Compiling freertos_kernel/portable/GCC/ARM_CM4F/port.c
Compiling freertos_kernel/portable/MemMang/heap_1.c
Linking library
clang-10: error: unable to execute command: Executable "ld.lld" doesn't exist!
clang-10: error: ld.lld command failed with exit code 1 (use -v to see invocation)

I got around this error by installing the the linker package (lld on arch). Once I installed this, I got a library related error:

Compiling freertos_kernel/portable/MemMang/heap_1.c
Linking library
ld.lld: error: unable to find library -lgcc
clang-10: error: ld.lld command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:148: /home/rajah/workspace/interrupt/example/freertos-example-``llvm/build/nrf52.elf] Error 1
When I just run make instead of scan-build make, an elf output is generated. However, even with this error, I can still run scan-view as all the errors have been captured as part of the compile process. I am very impressed with how the errors are presented. I will continue with rest of the blog post as scan-build is working now. Any idea why scan-build its not able to find the gcc library?

This is a very interesting post. I have tried to get scan-build to run in the past without success… That’s still the case, but I think I got a lot closer today.

Have you had any success running this on Windows? I can get LLVM installed no problem and scan-build runs, but fails at various points. The closest I got was specifying the analyzer, target, and compiler, but it still failed due to what appears to be a space in the path of ccc-analyzer… Anyway, has anyone run this on Windows with success? My makefile runs no problem.

Any thoughts Chris?

Thanks,
Rajah

hi @Rajah

Can you try adding COMPILER=clang to the beginning of your invocation to see if it fixes the problem?

COMPILER=clang scan-build --use-cc=<path_to_clang>/clang make