Firmware over the air (FOTA) and FCB support in 1.11.0


ashish.shukla@corvi.com <ashish.shukla@...>
 

Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development


Please consider the environment before printing this e-mail or its attachments.

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi


Carles Cufi
 

Hi Ashish,

 

I plan to submit a PR with documentation as soon as possible.

 

Please let me know your GitHub ID so I can include you in the review and you can let me know if the documentation is enough.

 

Regards,

 

Carles

 

From: zephyr-devel-bounces@... <zephyr-devel-bounces@...> On Behalf Of ashish.shukla@...
Sent: 21 March 2018 05:15
To: zephyr-devel@...; zephyr-users@...
Subject: [Zephyr-devel] Firmware over the air (FOTA) and FCB support in 1.11.0

 

Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Any help regarding the same would be of great help.

 

--

Warm regards,
Ashish Shukla

Jr. Embedded Engineer

Research & Development

 

Please consider the environment before printing this e-mail or its attachments.

 

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi

 


Michael Scott
 



On 03/20/2018 09:14 PM, ashish.shukla@... wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Hi Ashish,

Your question is a bit open-ended, and might be difficult to answer without some details regarding your paricular use-case (BLE update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for receiving a firmware update in the LwM2M client, but the implementation of where to store the incoming binary data is up to you.  See https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208 for a callback example that is triggered on each incoming block of data.  Documentation for the sample itself doesn't discuss the firmware update mechanism, but it's here for reference: http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html

Then, there is a robust DFU (Device Firmware Update) subsystem to help implement the image writing portion of a firmware update as well as integrate with mcuboot (an MCU bootloader) which would check an image for validity and then move it into the bootable application slot.   See: https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu for sources.

Regarding FCB: the initial implementation is done in v1.11, but the APIs are fairly complex and IMHO it's meant to be used as a base layer for other higher level implementations to manage persistent data such as device configuration, system logs and other uses.  There are several pull requests in-progress which aim to add these higher level services.  (The first PR will probably be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
https://github.com/zephyrproject-rtos/zephyr/pull/6408

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development


Please consider the environment before printing this e-mail or its attachments.

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@...
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Vikrant More <vikrant8051@...>
 

Hi,
>> Regarding FCB: the initial implementation is done in v1.11,
>>but the APIs are fairly complex and IMHO it's meant to be used as a base layer for other higher level implementations to manage persistent data such as device configuration, system logs and other uses. 
>>There are several pull requests in-progress which aim to add these higher level services.  (The first PR will probably be re-written to use FCB as it's base layer):
>> https://github.com/zephyrproject-rtos/zephyr/pull/6391
>> https://github.com/zephyrproject-rtos/zephyr/pull/6408

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on flash which updates frequently ?

Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott <michael@...> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@... wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Hi Ashish,

Your question is a bit open-ended, and might be difficult to answer without some details regarding your paricular use-case (BLE update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for receiving a firmware update in the LwM2M client, but the implementation of where to store the incoming binary data is up to you.  See https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208 for a callback example that is triggered on each incoming block of data.  Documentation for the sample itself doesn't discuss the firmware update mechanism, but it's here for reference: http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html

Then, there is a robust DFU (Device Firmware Update) subsystem to help implement the image writing portion of a firmware update as well as integrate with mcuboot (an MCU bootloader) which would check an image for validity and then move it into the bootable application slot.   See: https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu for sources.

Regarding FCB: the initial implementation is done in v1.11, but the APIs are fairly complex and IMHO it's meant to be used as a base layer for other higher level implementations to manage persistent data such as device configuration, system logs and other uses.  There are several pull requests in-progress which aim to add these higher level services.  (The first PR will probably be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
https://github.com/zephyrproject-rtos/zephyr/pull/6408

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development


Please consider the environment before printing this e-mail or its attachments.

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel



Michael Scott
 



On 03/21/2018 10:09 AM, Vikrant More wrote:
Hi,
>> Regarding FCB: the initial implementation is done in v1.11,
>>but the APIs are fairly complex and IMHO it's meant to be used as a base layer for other higher level implementations to manage persistent data such as device configuration, system logs and other uses. 
>>There are several pull requests in-progress which aim to add these higher level services.  (The first PR will probably be re-written to use FCB as it's base layer):
>> https://github.com/zephyrproject-rtos/zephyr/pull/6391
>> https://github.com/zephyrproject-rtos/zephyr/pull/6408

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on flash which updates frequently ?

IIRC, the idea is to settle on 1 solution which would be appropriate for most use-cases (including Mesh Sequence Numbers).   You might want to have 2 separate storage locations if you have data that only updates occasionally *and* data that updates frequently.

The guys from Nordic (and possibly Johan) are looking at this issue as well.

- Mike


Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott <michael@...> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@... wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Hi Ashish,

Your question is a bit open-ended, and might be difficult to answer without some details regarding your paricular use-case (BLE update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for receiving a firmware update in the LwM2M client, but the implementation of where to store the incoming binary data is up to you.  See https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208 for a callback example that is triggered on each incoming block of data.  Documentation for the sample itself doesn't discuss the firmware update mechanism, but it's here for reference: http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html

Then, there is a robust DFU (Device Firmware Update) subsystem to help implement the image writing portion of a firmware update as well as integrate with mcuboot (an MCU bootloader) which would check an image for validity and then move it into the bootable application slot.   See: https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu for sources.

Regarding FCB: the initial implementation is done in v1.11, but the APIs are fairly complex and IMHO it's meant to be used as a base layer for other higher level implementations to manage persistent data such as device configuration, system logs and other uses.  There are several pull requests in-progress which aim to add these higher level services.  (The first PR will probably be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
https://github.com/zephyrproject-rtos/zephyr/pull/6408

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development


Please consider the environment before printing this e-mail or its attachments.

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel




laczenJMS
 

Hi,

As the author of NVS (PR6391
https://github.com/zephyrproject-rtos/zephyr/pull/6391) I am probably
a bit biased. NVS as is can be used directly for managing persistent
data. I am looking into adding NVS as a backend to the configuration
system, but the configuration system comes with a major drawback:
flash usage.

I did a little calculation to compare the configuration system with
FCB as backend to direct use of NVS. Lets make the following
assumptions: Device is nrf51822, flash erase size is 1k, flash erase
cycles is 20000. Flash available for storage is 2k (2 pages). Data
without the sequence number takes up 512 bytes so 512 bytes is
available to store the sequence number.

Now lets see what happens when storing the sequence number, we will
only store it after 8192 messages (this means we only need to store a
11 bit number). This number can be increased but will reduce the
number of reboots that are allowed (to keep the IV index from changing
to fast). The worst case scenario is 24.3 messages/s (see nordicsemi
mesh-sdk).

Calculation for the config system: each store of the sequence number
is a string looking like:
"bt/btmesh/seq=XXXX" which takes up 18 bytes. FCB overhead takes
another 8 bytes, so in total 26 bytes for a write of the sequence
number. After 19 (512/26) writes a sector is full and it will be
erased (using a 1K scratch sector). So with the worst case of 24.3
messages/s: after 19 * 8192 /24.3 s a sector will be erased. Or every
1.78 h. With the erase cycles of 20000 flash will be worn out after
about 4 years.

Calculation for NVS: the sequence number is given an id for storage.
This id is used to differentiate between other data that is stored
(and is part of the NVS overhead). The sequence number takes up only 2
bytes. NVS overhead takes up 8 bytes, so in total 10 bytes for a
write. After 51 (512/10) messages the first page (1k) is full and the
second page is taken into use. After 51 messages the second sector is
full and the first page is erased. So after 102 messages the same page
is erased. With the erase cycle of 20000 flash will be worn out after
about 21.5 years.

Modifying the above assumptions should allow you to get an estimate of
the expected life of your specific application.

To summarize: the configuration system comes at a price, the use of
human readable strings as storage items takes up more space, and
reduces the life of the device.

I am currently working on a solution for the configuration system that
would use NVS as a backend but would also allow some data to be
directly written to NVS. This would allow combination of human
readability for some data with reduced flash wear for other. But the
human readability will always take up more space.

Kind regards,

Jehudi

Message: 1
Date: Wed, 21 Mar 2018 14:02:39 -0700
From: Michael Scott <michael@opensourcefoundries.com>
To: Vikrant More <vikrant8051@gmail.com>
Cc: zephyr-users@lists.zephyrproject.org,
zephyr-devel@lists.zephyrproject.org, "ashish.shukla@corvi.com"
<ashish.shukla@corvi.com>
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB
support in 1.11.0
Message-ID:
<7d94daad-9272-4517-f236-50255c352f00@opensourcefoundries.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"



On 03/21/2018 10:09 AM, Vikrant More wrote:
Hi,
Regarding FCB: the initial implementation is done in v1.11,
but the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage persistent
data such as device configuration, system logs and other uses.
There are several pull requests in-progress which aim to add these
higher level services.? (The first PR will probably be re-written to
use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on
flash which updates frequently ?
IIRC, the idea is to settle on 1 solution which would be appropriate for
most use-cases (including Mesh Sequence Numbers).?? You might want to
have 2 separate storage locations if you have data that only updates
occasionally *and* data that updates frequently.

The guys from Nordic (and possibly Johan) are looking at this issue as well.

- Mike


Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott
<michael@opensourcefoundries.com
<mailto:michael@opensourcefoundries.com>> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@corvi.com
<mailto:ashish.shukla@corvi.com> wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when
it is supported, I cannot see any samples available or proper
documentation to use these features in my project.
Hi Ashish,

Your question is a bit open-ended, and might be difficult to
answer without some details regarding your paricular use-case (BLE
update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for
receiving a firmware update in the LwM2M client, but the
implementation of where to store the incoming binary data is up to
you.? See
https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208
<https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208>
for a callback example that is triggered on each incoming block of
data.? Documentation for the sample itself doesn't discuss the
firmware update mechanism, but it's here for reference:
http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html
<http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html>

Then, there is a robust DFU (Device Firmware Update) subsystem to
help implement the image writing portion of a firmware update as
well as integrate with mcuboot (an MCU bootloader) which would
check an image for validity and then move it into the bootable
application slot. ? See:
https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu
<https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu>
for sources.

Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and
other uses.? There are several pull requests in-progress which aim
to add these higher level services.? (The first PR will probably
be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development
www.corvi.com <http://www.corvi.com/>


Please consider the environment before printing this e-mail or
its attachments.

Disclaimer: The information contained herein (including any
accompanying documents) is confidential and is intended solely
for the addressee(s). If you have erroneously received this
message, please immediately delete it and notify the sender.
Also, if you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution or taking any
action in reliance on the contents of this message or any
accompanying document is strictly prohibited and is unlawful. The
organization is not responsible for any damage caused by a virus
or alteration of the e-mail by a third party or otherwise. The
contents of this message may not necessarily represent the views
or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>


Johan Hedberg
 

Hi Jehudi,

That's some good analysis you've done. Note that this is for "frequent
sender" mesh nodes. For "requent receiver" nodes it's the Replay
Protection List (RPL) that will need to be updated frequently. What's
worse for the RPL is that you can't make similar jumps of 8192 messages,
rather you have to update the RPL for every message that the node
processes (note that relaying does not count as "processing" in this
sense).

The implications of failing to keep an updated Seq vs failing to keep an
updated RPL are similar, but not quite the same: in the case of the Seq,
you end up in a situation where other nodes ignore packets from you,
since they think someone is doing a replay attack with old seq values.
In the case of not updating the RPL it's the local node that gets
exposed to potential replay attacks.

Johan

On Thu, Mar 22, 2018, Laczen JMS wrote:
Hi,

As the author of NVS (PR6391
https://github.com/zephyrproject-rtos/zephyr/pull/6391) I am probably
a bit biased. NVS as is can be used directly for managing persistent
data. I am looking into adding NVS as a backend to the configuration
system, but the configuration system comes with a major drawback:
flash usage.

I did a little calculation to compare the configuration system with
FCB as backend to direct use of NVS. Lets make the following
assumptions: Device is nrf51822, flash erase size is 1k, flash erase
cycles is 20000. Flash available for storage is 2k (2 pages). Data
without the sequence number takes up 512 bytes so 512 bytes is
available to store the sequence number.

Now lets see what happens when storing the sequence number, we will
only store it after 8192 messages (this means we only need to store a
11 bit number). This number can be increased but will reduce the
number of reboots that are allowed (to keep the IV index from changing
to fast). The worst case scenario is 24.3 messages/s (see nordicsemi
mesh-sdk).

Calculation for the config system: each store of the sequence number
is a string looking like:
"bt/btmesh/seq=XXXX" which takes up 18 bytes. FCB overhead takes
another 8 bytes, so in total 26 bytes for a write of the sequence
number. After 19 (512/26) writes a sector is full and it will be
erased (using a 1K scratch sector). So with the worst case of 24.3
messages/s: after 19 * 8192 /24.3 s a sector will be erased. Or every
1.78 h. With the erase cycles of 20000 flash will be worn out after
about 4 years.

Calculation for NVS: the sequence number is given an id for storage.
This id is used to differentiate between other data that is stored
(and is part of the NVS overhead). The sequence number takes up only 2
bytes. NVS overhead takes up 8 bytes, so in total 10 bytes for a
write. After 51 (512/10) messages the first page (1k) is full and the
second page is taken into use. After 51 messages the second sector is
full and the first page is erased. So after 102 messages the same page
is erased. With the erase cycle of 20000 flash will be worn out after
about 21.5 years.

Modifying the above assumptions should allow you to get an estimate of
the expected life of your specific application.

To summarize: the configuration system comes at a price, the use of
human readable strings as storage items takes up more space, and
reduces the life of the device.

I am currently working on a solution for the configuration system that
would use NVS as a backend but would also allow some data to be
directly written to NVS. This would allow combination of human
readability for some data with reduced flash wear for other. But the
human readability will always take up more space.

Kind regards,

Jehudi
Message: 1
Date: Wed, 21 Mar 2018 14:02:39 -0700
From: Michael Scott <michael@opensourcefoundries.com>
To: Vikrant More <vikrant8051@gmail.com>
Cc: zephyr-users@lists.zephyrproject.org,
zephyr-devel@lists.zephyrproject.org, "ashish.shukla@corvi.com"
<ashish.shukla@corvi.com>
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB
support in 1.11.0
Message-ID:
<7d94daad-9272-4517-f236-50255c352f00@opensourcefoundries.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"



On 03/21/2018 10:09 AM, Vikrant More wrote:
Hi,
Regarding FCB: the initial implementation is done in v1.11,
but the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage persistent
data such as device configuration, system logs and other uses.
There are several pull requests in-progress which aim to add these
higher level services.? (The first PR will probably be re-written to
use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on
flash which updates frequently ?
IIRC, the idea is to settle on 1 solution which would be appropriate for
most use-cases (including Mesh Sequence Numbers).?? You might want to
have 2 separate storage locations if you have data that only updates
occasionally *and* data that updates frequently.

The guys from Nordic (and possibly Johan) are looking at this issue as well.

- Mike


Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott
<michael@opensourcefoundries.com
<mailto:michael@opensourcefoundries.com>> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@corvi.com
<mailto:ashish.shukla@corvi.com> wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when
it is supported, I cannot see any samples available or proper
documentation to use these features in my project.
Hi Ashish,

Your question is a bit open-ended, and might be difficult to
answer without some details regarding your paricular use-case (BLE
update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for
receiving a firmware update in the LwM2M client, but the
implementation of where to store the incoming binary data is up to
you.? See
https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208
<https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208>
for a callback example that is triggered on each incoming block of
data.? Documentation for the sample itself doesn't discuss the
firmware update mechanism, but it's here for reference:
http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html
<http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html>

Then, there is a robust DFU (Device Firmware Update) subsystem to
help implement the image writing portion of a firmware update as
well as integrate with mcuboot (an MCU bootloader) which would
check an image for validity and then move it into the bootable
application slot. ? See:
https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu
<https://github.com/zephyrproject-rtos/zephyr/tree/master/subsys/dfu>
for sources.

Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and
other uses.? There are several pull requests in-progress which aim
to add these higher level services.? (The first PR will probably
be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development
www.corvi.com <http://www.corvi.com/>


Please consider the environment before printing this e-mail or
its attachments.

Disclaimer: The information contained herein (including any
accompanying documents) is confidential and is intended solely
for the addressee(s). If you have erroneously received this
message, please immediately delete it and notify the sender.
Also, if you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution or taking any
action in reliance on the contents of this message or any
accompanying document is strictly prohibited and is unlawful. The
organization is not responsible for any damage caused by a virus
or alteration of the e-mail by a third party or otherwise. The
contents of this message may not necessarily represent the views
or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Carles Cufi
 

Hi Jehudi,

Thanks for all the info, maybe you could post this in the NVS PR.

Additionally, you compared Config+FCB with NVS accessed directly. Since the first item we need to decide on is actually whether we "augment" FCB with some of the NVS code or we add NVS as a separate subsystem, a very interesting comparison would be to have a real-life RAM and Flash usage comparison of FCB and NVS both being accessed directly, if possible for BLE Mesh data since that is the main target of NVS right now. How much worse off would we be if we used FCB as it stands today instead of NVS?

Thanks,

Carles

-----Original Message-----
From: zephyr-devel-bounces@lists.zephyrproject.org <zephyr-devel-
bounces@lists.zephyrproject.org> On Behalf Of Laczen JMS
Sent: 22 March 2018 14:01
To: zephyr-devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB support
in 1.11.0

Hi,

As the author of NVS (PR6391
https://github.com/zephyrproject-rtos/zephyr/pull/6391) I am probably a
bit biased. NVS as is can be used directly for managing persistent data.
I am looking into adding NVS as a backend to the configuration system,
but the configuration system comes with a major drawback:
flash usage.

I did a little calculation to compare the configuration system with FCB
as backend to direct use of NVS. Lets make the following
assumptions: Device is nrf51822, flash erase size is 1k, flash erase
cycles is 20000. Flash available for storage is 2k (2 pages). Data
without the sequence number takes up 512 bytes so 512 bytes is available
to store the sequence number.

Now lets see what happens when storing the sequence number, we will only
store it after 8192 messages (this means we only need to store a
11 bit number). This number can be increased but will reduce the number
of reboots that are allowed (to keep the IV index from changing to
fast). The worst case scenario is 24.3 messages/s (see nordicsemi mesh-
sdk).

Calculation for the config system: each store of the sequence number is
a string looking like:
"bt/btmesh/seq=XXXX" which takes up 18 bytes. FCB overhead takes another
8 bytes, so in total 26 bytes for a write of the sequence number. After
19 (512/26) writes a sector is full and it will be erased (using a 1K
scratch sector). So with the worst case of 24.3
messages/s: after 19 * 8192 /24.3 s a sector will be erased. Or every
1.78 h. With the erase cycles of 20000 flash will be worn out after
about 4 years.

Calculation for NVS: the sequence number is given an id for storage.
This id is used to differentiate between other data that is stored (and
is part of the NVS overhead). The sequence number takes up only 2 bytes.
NVS overhead takes up 8 bytes, so in total 10 bytes for a write. After
51 (512/10) messages the first page (1k) is full and the second page is
taken into use. After 51 messages the second sector is full and the
first page is erased. So after 102 messages the same page is erased.
With the erase cycle of 20000 flash will be worn out after about 21.5
years.

Modifying the above assumptions should allow you to get an estimate of
the expected life of your specific application.

To summarize: the configuration system comes at a price, the use of
human readable strings as storage items takes up more space, and reduces
the life of the device.

I am currently working on a solution for the configuration system that
would use NVS as a backend but would also allow some data to be directly
written to NVS. This would allow combination of human readability for
some data with reduced flash wear for other. But the human readability
will always take up more space.

Kind regards,

Jehudi
Message: 1
Date: Wed, 21 Mar 2018 14:02:39 -0700
From: Michael Scott <michael@opensourcefoundries.com>
To: Vikrant More <vikrant8051@gmail.com>
Cc: zephyr-users@lists.zephyrproject.org,
zephyr-devel@lists.zephyrproject.org,
"ashish.shukla@corvi.com"
<ashish.shukla@corvi.com>
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB
support in 1.11.0
Message-ID:
<7d94daad-9272-4517-f236-50255c352f00@opensourcefoundries.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"



On 03/21/2018 10:09 AM, Vikrant More wrote:
Hi,
Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and other
uses.
There are several pull requests in-progress which aim to add these
higher level services.? (The first PR will probably be re-written to
use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on
flash which updates frequently ?
IIRC, the idea is to settle on 1 solution which would be appropriate
for most use-cases (including Mesh Sequence Numbers).?? You might want
to have 2 separate storage locations if you have data that only
updates occasionally *and* data that updates frequently.

The guys from Nordic (and possibly Johan) are looking at this issue as
well.

- Mike


Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott
<michael@opensourcefoundries.com
<mailto:michael@opensourcefoundries.com>> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@corvi.com
<mailto:ashish.shukla@corvi.com> wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now
when
it is supported, I cannot see any samples available or proper
documentation to use these features in my project.
Hi Ashish,

Your question is a bit open-ended, and might be difficult to
answer without some details regarding your paricular use-case
(BLE
update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for
receiving a firmware update in the LwM2M client, but the
implementation of where to store the incoming binary data is up
to
you.? See
https://github.com/zephyrproject-
rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208
<https://github.com/zephyrproject-
rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-
client.c#L208>
for a callback example that is triggered on each incoming block
of
data.? Documentation for the sample itself doesn't discuss the
firmware update mechanism, but it's here for reference:
http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html

<http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html>

Then, there is a robust DFU (Device Firmware Update) subsystem to
help implement the image writing portion of a firmware update as
well as integrate with mcuboot (an MCU bootloader) which would
check an image for validity and then move it into the bootable
application slot. ? See:
https://github.com/zephyrproject-
rtos/zephyr/tree/master/subsys/dfu
<https://github.com/zephyrproject-
rtos/zephyr/tree/master/subsys/dfu>
for sources.

Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and
other uses.? There are several pull requests in-progress which
aim
to add these higher level services.? (The first PR will probably
be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development
www.corvi.com <http://www.corvi.com/>


Please consider the environment before printing this e-mail or
its attachments.

Disclaimer: The information contained herein (including any
accompanying documents) is confidential and is intended solely
for the addressee(s). If you have erroneously received this
message, please immediately delete it and notify the sender.
Also, if you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution or taking
any
action in reliance on the contents of this message or any
accompanying document is strictly prohibited and is unlawful.
The
organization is not responsible for any damage caused by a virus
or alteration of the e-mail by a third party or otherwise. The
contents of this message may not necessarily represent the views
or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


laczenJMS
 

Hi Carles,

I will add the lifetime calculation to the documentation of NVS.

Regarding the comparison between FCB and NVS: FCB would need an
additional layer to be able to get this functionality (which is the
config layer). The following decisions are influencing flash wear:

1. Config system: store name-value pairs as strings: this makes the
difference between 10 bytes and 26 bytes (should really be 27 because
I forgot the zero termination).
2. FCB storage: use of scratch sector this reduces the lifetime to the
amount of scratch sector erases.

Decision 1 will reduce life with a factor 2.6, Decision 2 will reduce
life with at least a factor 2.

Kind regards,

Jehudi

Date: Thu, 22 Mar 2018 15:27:23 +0000
From: "Cufi, Carles" <Carles.Cufi@nordicsemi.no>
To: Laczen JMS <laczenjms@gmail.com>,
"zephyr-devel@lists.zephyrproject.org"
<zephyr-devel@lists.zephyrproject.org>
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB
support in 1.11.0
Message-ID: <26729c68a51f4a9087156a98f53f9eee@nordicsemi.no>
Content-Type: text/plain; charset="us-ascii"

Hi Jehudi,

Thanks for all the info, maybe you could post this in the NVS PR.

Additionally, you compared Config+FCB with NVS accessed directly. Since the first item we need to decide on is actually whether we "augment" FCB with some of the NVS code or we add NVS as a separate subsystem, a very interesting comparison would be to have a real-life RAM and Flash usage comparison of FCB and NVS both being accessed directly, if possible for BLE Mesh data since that is the main target of NVS right now. How much worse off would we be if we used FCB as it stands today instead of NVS?

Thanks,

Carles

-----Original Message-----
From: zephyr-devel-bounces@lists.zephyrproject.org <zephyr-devel-
bounces@lists.zephyrproject.org> On Behalf Of Laczen JMS
Sent: 22 March 2018 14:01
To: zephyr-devel@lists.zephyrproject.org
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB support
in 1.11.0

Hi,

As the author of NVS (PR6391
https://github.com/zephyrproject-rtos/zephyr/pull/6391) I am probably a
bit biased. NVS as is can be used directly for managing persistent data.
I am looking into adding NVS as a backend to the configuration system,
but the configuration system comes with a major drawback:
flash usage.

I did a little calculation to compare the configuration system with FCB
as backend to direct use of NVS. Lets make the following
assumptions: Device is nrf51822, flash erase size is 1k, flash erase
cycles is 20000. Flash available for storage is 2k (2 pages). Data
without the sequence number takes up 512 bytes so 512 bytes is available
to store the sequence number.

Now lets see what happens when storing the sequence number, we will only
store it after 8192 messages (this means we only need to store a
11 bit number). This number can be increased but will reduce the number
of reboots that are allowed (to keep the IV index from changing to
fast). The worst case scenario is 24.3 messages/s (see nordicsemi mesh-
sdk).

Calculation for the config system: each store of the sequence number is
a string looking like:
"bt/btmesh/seq=XXXX" which takes up 18 bytes. FCB overhead takes another
8 bytes, so in total 26 bytes for a write of the sequence number. After
19 (512/26) writes a sector is full and it will be erased (using a 1K
scratch sector). So with the worst case of 24.3
messages/s: after 19 * 8192 /24.3 s a sector will be erased. Or every
1.78 h. With the erase cycles of 20000 flash will be worn out after
about 4 years.

Calculation for NVS: the sequence number is given an id for storage.
This id is used to differentiate between other data that is stored (and
is part of the NVS overhead). The sequence number takes up only 2 bytes.
NVS overhead takes up 8 bytes, so in total 10 bytes for a write. After
51 (512/10) messages the first page (1k) is full and the second page is
taken into use. After 51 messages the second sector is full and the
first page is erased. So after 102 messages the same page is erased.
With the erase cycle of 20000 flash will be worn out after about 21.5
years.

Modifying the above assumptions should allow you to get an estimate of
the expected life of your specific application.

To summarize: the configuration system comes at a price, the use of
human readable strings as storage items takes up more space, and reduces
the life of the device.

I am currently working on a solution for the configuration system that
would use NVS as a backend but would also allow some data to be directly
written to NVS. This would allow combination of human readability for
some data with reduced flash wear for other. But the human readability
will always take up more space.

Kind regards,

Jehudi
Message: 1
Date: Wed, 21 Mar 2018 14:02:39 -0700
From: Michael Scott <michael@opensourcefoundries.com>
To: Vikrant More <vikrant8051@gmail.com>
Cc: zephyr-users@lists.zephyrproject.org,
zephyr-devel@lists.zephyrproject.org,
"ashish.shukla@corvi.com"
<ashish.shukla@corvi.com>
Subject: Re: [Zephyr-devel] Firmware over the air (FOTA) and FCB
support in 1.11.0
Message-ID:
<7d94daad-9272-4517-f236-50255c352f00@opensourcefoundries.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"



On 03/21/2018 10:09 AM, Vikrant More wrote:
Hi,
Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and other
uses.
There are several pull requests in-progress which aim to add these
higher level services.? (The first PR will probably be re-written to
use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Are these PRs, suitable to save #BluetoothMesh's Sequence Numbers on
flash which updates frequently ?
IIRC, the idea is to settle on 1 solution which would be appropriate
for most use-cases (including Mesh Sequence Numbers).?? You might want
to have 2 separate storage locations if you have data that only
updates occasionally *and* data that updates frequently.

The guys from Nordic (and possibly Johan) are looking at this issue as
well.

- Mike


Thanks,


On Wed, Mar 21, 2018 at 9:59 PM, Michael Scott
<michael@opensourcefoundries.com
<mailto:michael@opensourcefoundries.com>> wrote:



On 03/20/2018 09:14 PM, ashish.shukla@corvi.com
<mailto:ashish.shukla@corvi.com> wrote:
Hi all,

I've been waiting for FOTA and FCB support in zephyr and now
when
it is supported, I cannot see any samples available or proper
documentation to use these features in my project.
Hi Ashish,

Your question is a bit open-ended, and might be difficult to
answer without some details regarding your paricular use-case
(BLE
update, IP-based update, Mesh, etc)

For instance, the LwM2M subsystem provides a mechanism for
receiving a firmware update in the LwM2M client, but the
implementation of where to store the incoming binary data is up
to
you.? See
https://github.com/zephyrproject-
rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-client.c#L208
<https://github.com/zephyrproject-
rtos/zephyr/blob/master/samples/net/lwm2m_client/src/lwm2m-
client.c#L208>
for a callback example that is triggered on each incoming block
of
data.? Documentation for the sample itself doesn't discuss the
firmware update mechanism, but it's here for reference:
http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html

<http://docs.zephyrproject.org/samples/net/lwm2m_client/README.html>

Then, there is a robust DFU (Device Firmware Update) subsystem to
help implement the image writing portion of a firmware update as
well as integrate with mcuboot (an MCU bootloader) which would
check an image for validity and then move it into the bootable
application slot. ? See:
https://github.com/zephyrproject-
rtos/zephyr/tree/master/subsys/dfu
<https://github.com/zephyrproject-
rtos/zephyr/tree/master/subsys/dfu>
for sources.

Regarding FCB: the initial implementation is done in v1.11, but
the APIs are fairly complex and IMHO it's meant to be used as a
base layer for other higher level implementations to manage
persistent data such as device configuration, system logs and
other uses.? There are several pull requests in-progress which
aim
to add these higher level services.? (The first PR will probably
be re-written to use FCB as it's base layer):
https://github.com/zephyrproject-rtos/zephyr/pull/6391
<https://github.com/zephyrproject-rtos/zephyr/pull/6391>
https://github.com/zephyrproject-rtos/zephyr/pull/6408
<https://github.com/zephyrproject-rtos/zephyr/pull/6408>

Hopefully that helps get you started,

- Mike


Any help regarding the same would be of great help.

--
Warm regards,
Ashish Shukla
Jr. Embedded Engineer
Research & Development
www.corvi.com <http://www.corvi.com/>


Please consider the environment before printing this e-mail or
its attachments.

Disclaimer: The information contained herein (including any
accompanying documents) is confidential and is intended solely
for the addressee(s). If you have erroneously received this
message, please immediately delete it and notify the sender.
Also, if you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution or taking
any
action in reliance on the contents of this message or any
accompanying document is strictly prohibited and is unlawful.
The
organization is not responsible for any damage caused by a virus
or alteration of the e-mail by a third party or otherwise. The
contents of this message may not necessarily represent the views
or policies of Corvi



_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
<mailto:Zephyr-devel@lists.zephyrproject.org>
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel
<https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel>

_______________________________________________
Zephyr-devel mailing list
Zephyr-devel@lists.zephyrproject.org
https://lists.zephyrproject.org/mailman/listinfo/zephyr-devel


Carles Cufi
 

Hi Ashish,

 

Here is the published documentation:

 

http://docs.zephyrproject.org/samples/subsys/mgmt/mcumgr/smp_svr/README.html

 

Regards,

 

Carles

 

 

From: zephyr-devel-bounces@... <zephyr-devel-bounces@...> On Behalf Of ashish.shukla@...
Sent: 21 March 2018 05:15
To: zephyr-devel@...; zephyr-users@...
Subject: [Zephyr-devel] Firmware over the air (FOTA) and FCB support in 1.11.0

 

Hi all,

I've been waiting for FOTA and FCB support in zephyr and now when it is supported, I cannot see any samples available or proper documentation to use these features in my project.

Any help regarding the same would be of great help.

 

--

Warm regards,
Ashish Shukla

Jr. Embedded Engineer

Research & Development

 

Please consider the environment before printing this e-mail or its attachments.

 

Disclaimer: The information contained herein (including any accompanying documents) is confidential and is intended solely for the addressee(s). If you have erroneously received this message, please immediately delete it and notify the sender. Also, if you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this message or any accompanying document is strictly prohibited and is unlawful. The organization is not responsible for any damage caused by a virus or alteration of the e-mail by a third party or otherwise. The contents of this message may not necessarily represent the views or policies of Corvi