Just found the following information in the MCUboot documentation available here under the threat model section. mcuboot/encrypted_images.md at master · mcu-tools/mcuboot · GitHub
It says:
Since decrypting requires a private key (or secret if using symmetric crypto) to reside inside the device, it is the responsibility of the device manufacturer to guarantee that this key is already in the device and not possible to extract.
So I guess there is no getting around needing to hardcode a key in the device.