Hi guys,

Recently I start working on Zephyr to be compliant with MISRA-C. I'd
like to give you a brief overview about MISRA-C and why this is valuable
to Zephyr.

MISRA-C is code guideline that define a subset of C that aims to make
embedded software safer, more secure and especially more portable. It
does it defining a set of guidelines that try to fill the gaps between C
specification and different implementations, it means things with
undefined behavior, unspecified behavior and things that the
compiler implementation is free to choose the behavior.

That's been said, every MISRA-C guideline is classified as rule or
directive. These rules and directives are classified in mandatory,
required or advisory. Mandatory means that the code must be compliant
with the guideline, required the code also must be compliant but it's
possible creating a deviation, advisory are recommendations and don't
need a formal deviation if ignored.

So, what about Zephyr ? Zephyr currently contains ~8k violations and
approximately half are either required or mandatory (~10). I'm filling
issues on github
(https://github.com/zephyrproject-rtos/zephyr/labels/MISRA-C) that
contains (I hope) enough information about the problem and some
suggestions to approach them. I ask everybody to take a look on them and
give opinions. Some of them required collaboration of diverse code-owners
and developers.

To finish, why MISRA-C is valuable to Zephyr ?

- Following a standard helps with maintainability of the code.
- Process standards like ISO26262 and IEC61508 used to develop critical
products require that the code follows a code guideline (ISO26262
explicitly recommends MISRA-C).
- Other proprietary RTOs are MISRA-C compliant, if we want to be more
with them, is good too.
- It'll help us spot problems bugs and avoid them. I've worked this week
in two guidelines that are not that intrusive and I found at least
three real bugs.
- We'll have less problems with different compilers, the mandatory
violations are regarding using specific gcc builtin functions.

If you have any questions regarding MISRA-C lets discuss it :)

Regards,
Flavio Ceolin

Hi All,

Making Zephyr MISRA-C compliant would be a huge achievement and a major selling point.
It would also greatly help  contributors (who are not yet award of MISRA-C or equivalent standards) to highly increase the quality of their code.
Regards

Abderrezak

On Sep 14, 2018, at 1:46 PM, Flavio Ceolin <flavio.ceolin@...> wrote:

Hi guys,

Recently I start working on Zephyr to be compliant with MISRA-C. I'd
like to give you a brief overview about MISRA-C and why this is valuable
to Zephyr.

MISRA-C is code guideline that define a subset of C that aims to make
embedded software safer, more secure and especially more portable. It
does it defining a set of guidelines that try to fill the gaps between C
specification and different implementations, it means things with
undefined behavior, unspecified behavior and things that the
compiler implementation is free to choose the behavior.

That's been said, every MISRA-C guideline is classified as rule or
directive. These rules and directives are classified in mandatory,
required or advisory. Mandatory means that the code must be compliant
with the guideline, required the code also must be compliant but it's
possible creating a deviation, advisory are recommendations and don't
need a formal deviation if ignored.

So, what about Zephyr ? Zephyr currently contains ~8k violations and
approximately half are either required or mandatory (~10). I'm filling
issues on github
(https://github.com/zephyrproject-rtos/zephyr/labels/MISRA-C) that
contains (I hope) enough information about the problem and some
suggestions to approach them. I ask everybody to take a look on them and
give opinions. Some of them required collaboration of diverse code-owners
and developers.

To finish, why MISRA-C is valuable to Zephyr ?

- Following a standard helps with maintainability of the code.
- Process standards like ISO26262 and IEC61508 used to develop critical
products require that the code follows a code guideline (ISO26262
explicitly recommends MISRA-C).
- Other proprietary RTOs are MISRA-C compliant, if we want to be more
with them, is good too.
- It'll help us spot problems bugs and avoid them. I've worked this week
in two guidelines that are not that intrusive and I found at least
three real bugs.
- We'll have less problems with different compilers, the mandatory
violations are regarding using specific gcc builtin functions.

If you have any questions regarding MISRA-C lets discuss it :)

Regards,
Flavio Ceolin

Have you looked at various GitHub integration tools that might help check code for MISRA guidelines so we can automate such reviews in the future.

Tools like:

https://lgtm.com
https://www.codacy.com
https://www.codefactor.io/

I think we need some automated way to check new PRs for issues before we invest a large amount of time fixing the code base, otherwise its just going to bitrot back to being non-MISRA compliant.

- k

AFAIK none of the services you list below support MISRA, some do not even support C.
We are looking at https://sonarcloud.io which has an integration with GH.

Anas

Not support MISRA directly, but I assume we can implement the MISRA checks either by configuring specific rules of cppcheck or implementing our own. If there’s some MISRA checker out there, even better.

My point is we need something as part of the GitHub PR workflow that does a bunch of the checking otherwise this isn’t really going to work. Plus it would be nice to have something we can add our own style rules into to help automate code review.

- k

Hello Flavio,

On Fri, Sep 14, 2018 at 11:19:40PM -0700, Flavio ceolin wrote:

Agree, we have to automate it as much as possible. But we can't just wait
have all tools ready to start fixing the issues. Currently there are almost
8k violations on kernel. Any tool we setup will make a lot of noise.

Some rules we will be able to check with coccinelle, but ideally we need a
qualified tool with Misra c support.

Thanks for using Coccinelle for the memset cleanup!

Coccinelle can handle MISRA-C compliance at some extent and it really
depends on the what you wish to accomplish through a coccinelle script.

Some scripts are easy to write, but certainly we can't claim to write
scripts with 0% false positives. It depends upon the problem set and
how well you design a cocci script.

For instance: We planned to remove the unnecessary casts returned by
memory allocating functions *alloc since we know the conversion
from void* and other type is implicit.

https://github.com/torvalds/linux/blob/master/scripts/coccinelle/api/alloc/alloc_cast.cocci

But while working and testing the script, I found some strange code
where the cast was necessary and removing it would be a regression:

drivers/scsi/fnic/fnic_trace.c

<snip>
static unsigned long fnic_trace_buf_p;
...
fnic_trace_buf_p = (unsigned long)vmalloc((trace_max_pages * PAGE_SIZE));
</snip>

And some other case as well, where the cast was required...

Therefore, we removed those cases by embedding a python script into the
coccinelle script(yes, we can embedd python & Ocaml script if required)

What I'm trying to explain is that some cases are easy to handle while some
are disaster to get desired property.

Also, we classify accuracy of the Coccinelle by defining a property
called "Confidence" at the top of the cocci scripts.

Confindence: High --> very rare chances of false postives.
Moderarte --> some chances of false postivies.

We always try to improve the cocci script as much as possible.

If there is any MISRA-C compliant build system then its obviously great.
But if not, then my proposal would be add `coccicheck` to CI and write
cocci scripts to warn.

For now we have some ideas that we are going to inherit from the
mainline kernel like:

* ARRAY_SIZE: Lots of instances in Zephyr where it is hard-coded rather
than simply using the ARRAY_SIZE helper macro.

* GENMASK: again we could use the helper macro instead of
BIT(3) | BIT(2) | BIT(1) == GENMASK(3,1)

And this GENMASK defintion is defined in:

drivers/dma/dma_stm32f4x.c:#define GENMASK(h, l) (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))

So, we could probably, move the declaration to include/misc/util.h
and use it throroughly on the codebase.

...and few more.

Many of the rules we made for mainline is based on enforcing usage of
available standard APIs and thus avoiding developers to roll-out their
own wrappers. In this way we can enforce new incoming code to adhere to
the Zephyr APIs and prevent any subsequent violations/redundancy.


https://github.com/zephyrproject-rtos/zephyr/pull/9725


Thanks
--
Himanshu Jha
Undergraduate Student
Department of Electronics & Communication
Guru Tegh Bahadur Institute of Technology

On 09/15/2018 06:07 AM, Kumar Gala wrote:

Not support MISRA directly, but I assume we can implement the MISRA checks either by configuring specific rules of cppcheck or implementing our own. If there’s some MISRA checker out there, even better.

Cppcheck has some MISRAC C 2012 checks for a while now [1].

Also, it is quite simple to add such checks to clang-tidy [2]. I am no
longer working in a safety relevant area, so I did not push this pet
project in the past, but extending it for Zephyrs needs would probably
be a good reason to do so.

[1] http://cppcheck.net/misra.php
[2] https://github.com/rettichschnidi/clang-tidy-misra

Kind regards,
Reto

Hi Himanshu,

Hello Flavio,

On Fri, Sep 14, 2018 at 11:19:40PM -0700, Flavio ceolin wrote:

Agree, we have to automate it as much as possible. But we can't just wait
have all tools ready to start fixing the issues. Currently there are almost
8k violations on kernel. Any tool we setup will make a lot of noise.

Some rules we will be able to check with coccinelle, but ideally we need a
qualified tool with Misra c support.

Thanks for using Coccinelle for the memset cleanup!

Coccinelle can handle MISRA-C compliance at some extent and it really
depends on the what you wish to accomplish through a coccinelle script.

Some scripts are easy to write, but certainly we can't claim to write
scripts with 0% false positives. It depends upon the problem set and
how well you design a cocci script.

Agree, we'll never put any compliance claim based on Coccinelle, even
because to this *much likely* we have to use a qualified
tool. Nevertheless, there are rules that we can't even check with
Coccinelle. My idea is using Coccinelle as a best effort with other
tools to enforce as much as possible MISRA-C guidelines.

For instance: We planned to remove the unnecessary casts returned by
memory allocating functions *alloc since we know the conversion
from void* and other type is implicit.

https://github.com/torvalds/linux/blob/master/scripts/coccinelle/api/alloc/alloc_cast.cocci

But while working and testing the script, I found some strange code
where the cast was necessary and removing it would be a regression:

drivers/scsi/fnic/fnic_trace.c

<snip>
static unsigned long fnic_trace_buf_p;
...
fnic_trace_buf_p = (unsigned long)vmalloc((trace_max_pages * PAGE_SIZE));
</snip>

And some other case as well, where the cast was required...

Therefore, we removed those cases by embedding a python script into the
coccinelle script(yes, we can embedd python & Ocaml script if required)

What I'm trying to explain is that some cases are easy to handle while some
are disaster to get desired property.

Also, we classify accuracy of the Coccinelle by defining a property
called "Confidence" at the top of the cocci scripts.

Confindence: High --> very rare chances of false postives.
Moderarte --> some chances of false postivies.

We always try to improve the cocci script as much as possible.

If there is any MISRA-C compliant build system then its obviously great.
But if not, then my proposal would be add `coccicheck` to CI and write
cocci scripts to warn.

Yeah, that is really what I had in mind. Currently I'm using coccinelle
where I can but I'm still reviewing the output manually.

For now we have some ideas that we are going to inherit from the
mainline kernel like:

* ARRAY_SIZE: Lots of instances in Zephyr where it is hard-coded rather
than simply using the ARRAY_SIZE helper macro.

* GENMASK: again we could use the helper macro instead of
BIT(3) | BIT(2) | BIT(1) == GENMASK(3,1)

And this GENMASK defintion is defined in:

drivers/dma/dma_stm32f4x.c:#define GENMASK(h, l) (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))

So, we could probably, move the declaration to include/misc/util.h
and use it throroughly on the codebase.

...and few more.

Many of the rules we made for mainline is based on enforcing usage of
available standard APIs and thus avoiding developers to roll-out their
own wrappers. In this way we can enforce new incoming code to adhere to
the Zephyr APIs and prevent any subsequent violations/redundancy.

Good !

https://github.com/zephyrproject-rtos/zephyr/pull/9725

Thanks a lot for you work on coccinelle and Zephyr, I certainly will
ping you with more questions :)

Regards,
Flavio Ceolin

Hi Reto,

On 09/15/2018 06:07 AM, Kumar Gala wrote:

Not support MISRA directly, but I assume we can implement the MISRA checks either by configuring specific rules of cppcheck or implementing our own. If there’s some MISRA checker out there, even better.

Cppcheck has some MISRAC C 2012 checks for a while now [1].

Yeah, last time I've checked it was requiring the MISRA-C standard
converted to a pure text. The problem is that we can distribute
MISRA-C. Though, I'll check if the standard is only required for
pretty-print the errors.

Also, it is quite simple to add such checks to clang-tidy [2]. I am no
longer working in a safety relevant area, so I did not push this pet
project in the past, but extending it for Zephyrs needs would probably
be a good reason to do so.

Good to know :)

[1] http://cppcheck.net/misra.php
[2] https://github.com/rettichschnidi/clang-tidy-misra

Kind regards,
Reto

Regards,
Flavio Ceolin

--

On 9/17/18 10:48 AM, Jiří Kubias wrote:

I have one practical question regarding MISRA-C standard. Where I can
get the standard as a regular coder? I can buy one for £15 + VAT, but I
dont thing that every body want to buy it.

You have to buy it.

Is there any public description of this standard? There are many of us
who never met with MISRA standard.

This fairly new publication might help you: https://arxiv.org/abs/1809.00821

Kind regards,
Reto

--

Hi Szymon,

Hello,

I was wondering about where did void casting for functions like memcpy,
memset comes from.
Could commit messages for this point to specific rule?

Sure thing ! Next commits I'll add which rule that commits is fixing.
Thanks for pointint it out :)

On 14 September 2018 at 20:46, Flavio Ceolin <flavio.ceolin@...>
wrote:

Hi guys,

Recently I start working on Zephyr to be compliant with MISRA-C. I'd
like to give you a brief overview about MISRA-C and why this is valuable
to Zephyr.

MISRA-C is code guideline that define a subset of C that aims to make
embedded software safer, more secure and especially more portable. It
does it defining a set of guidelines that try to fill the gaps between C
specification and different implementations, it means things with
undefined behavior, unspecified behavior and things that the
compiler implementation is free to choose the behavior.

That's been said, every MISRA-C guideline is classified as rule or
directive. These rules and directives are classified in mandatory,
required or advisory. Mandatory means that the code must be compliant
with the guideline, required the code also must be compliant but it's
possible creating a deviation, advisory are recommendations and don't
need a formal deviation if ignored.

So, what about Zephyr ? Zephyr currently contains ~8k violations and
approximately half are either required or mandatory (~10). I'm filling
issues on github
(https://github.com/zephyrproject-rtos/zephyr/labels/MISRA-C) that
contains (I hope) enough information about the problem and some
suggestions to approach them. I ask everybody to take a look on them and
give opinions. Some of them required collaboration of diverse code-owners
and developers.

To finish, why MISRA-C is valuable to Zephyr ?

- Following a standard helps with maintainability of the code.
- Process standards like ISO26262 and IEC61508 used to develop critical
products require that the code follows a code guideline (ISO26262
explicitly recommends MISRA-C).
- Other proprietary RTOs are MISRA-C compliant, if we want to be more
with them, is good too.
- It'll help us spot problems bugs and avoid them. I've worked this week
in two guidelines that are not that intrusive and I found at least
three real bugs.
- We'll have less problems with different compilers, the mandatory
violations are regarding using specific gcc builtin functions.

If you have any questions regarding MISRA-C lets discuss it :)

Regards,
Flavio Ceolin

--
pozdrawiam
Szymon K. Janc

Regards,
Flavio Ceolin