[RFC] Zephyr Shell Enhancement


Avramovich, Yael <yael.avramovich@...>
 

Zephyr Shell Enhancement
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

========
Overview
========
Current shell implementation:
* Current Shell implementation is based on UART.
It uses one UART for console (UART number is determined by config file).

* Console device driver initialization priority:
Console has to be initialized after the UART driver it uses.

* When the shell is initialized, it starts the shell as a fiber and
initialize UART console.

* Current shell implementation supplies shell utilities only for a single
module.

=====
Goals
=====
1. Multiple and dynamic usage in modules level:
----------------------------------------------------------
Ability to add multiple modules to shell commands array.
Each module should be added dynamically by its config file.

2. Dynamic usage of the shell:
------------------------------------
Ability to enable / disable shell service.
If it is enabled (according to .Kconfig), basic framework shell commands
are also available for use.

3. Memory consumption:
------------------------------
Shell commands array should be allocated exactly according to the number of
entities that actually use the shell.

4. Shell format:
------------------
Use one of the following (Error message is shown in case of a wrong usage):
- ENTITY COMMAND
- help
- help ENTITY
- help ENTITY COMMAND
One of the entities is "FW", for the Framework module.

=============
Dynamic usage
=============

Infrastructure level:
-------------------------
File: misc/Kconfig

menu "Debugging Options"
......

config ENABLE_FW_SHELL
bool
prompt "Enable Shell service"
default y
help
Enabling shell services. If it is enabled, framework shell commands are
also available for use.


Module level:
------------------
Each module that wants to use shell service will add an appropriate config in
its kConfig file.
Example:
CONFIG_SAMPLE_MODULE_USE_SHELL=y
In the module's code, the shell usage is depend on the config parameter
(will be explained later on in the document).


===============================
FW shell service initialization
===============================
* Add a new call in _main() to "run_services".
* Kernel services:
Shell service initialization is done as part of a new module in kernel,
which is called - "kernel services".
This module initiates all kernel services.
Its functionality may be divided later on between microkernel and
nanokernel.
* Kernel services initialization is done before run main task
(nanokernel - background task, microkernel - idle task).
* Shell service initialization includes also adding basic framework
commands to shell commands.

Nanokernel initialization (nano_init.c)
----------------------------------------------
extern void run_services(void);
static void _main(void)
{
........
run_services();
extern void main(void);
main();
}

Microkernel initialization (k_init.c)
------------------------------------------
void _main(void)
{
...
task_group_start(EXE_GROUP);
run_services();
_k_kernel_idle();
}

Kernel services (kernel_services.c)
------------------------------------------
* Add a new file: kernel/nanokernel/kernel_services.c
* Responsibility: Initializing each kernel service, according to
configuration file.
* Kernel service definition: A module that should run after kernel and
devices initialization phase but before running main task.

Example to running kernel service:

extern void run_shell_service(void);

void run_services(void)
{
#ifdef CONFIG_ENABLE_FW_SHELL
run_shell_service();
#endif
}

The exact way of running kernel services will be redefined later.


Shell service (shell_service.c)
------------------------------------
* Add a new file: kernel/nanokernel/shell_service.c
* Responsibility: Initializes shell and add framework usage of the shell.
* Note: Memory for a specific module shell usage is not allocated if shell
is disabled in framework level, even if it is enabled in module's level.

#define SHELL_PROMPT "shell> "
#define SHELL_FW "FW"

struct shell_cmd fw_commands[] = {
{ "sample", shell_cmd_sample, "help sample info" },
{ NULL, NULL }
};

void run_shell_service(void)
{
shell_init(SHELL_PROMPT);
REGISTER_TO_SHELL(SHELL_FW, fw_commands);
}

====================
Module dynamic usage
====================
Each entity that wants to get shell services, should add its shell
identifier (as string) and register its call backs in shell database.

Example:
samples/shell/src/main.c
Put your code in main() or in another function, if the module is not an
exe task.

#define MY_SHELL_ENTITY "my_module"

struct shell_cmd commands[] = {
{ "ping", shell_cmd_ping, "print pong" },
{ "ticks", shell_cmd_ticks },
{ "highticks", shell_cmd_highticks },
{ NULL, NULL }
};

void main(void)
{
.....

#ifdef CONFIG_SAMPLE_MODULE_USE_SHELL
REGISTER_TO_SHELL(MY_SHELL_ENTITY, commands);
#endif
}

===================
Memory consumption
===================
File: include/misc/shell.h
This file is the shell header file.
The following code already exists in its current Zephyr version:
typedef int (*shell_cmd_function_t)(int argc, char *argv[]);

struct shell_cmd {
const char *cmd_name;
shell_cmd_function_t cb;
const char *help;
};

* Note: Currently, "help" string's length is not limited.
However, limitation on the length of each command's help string should be
added.

Shell initialization
-----------------------
File: include/misc/shell.h
* Current code:
/** init shell fiber and the shell commands.*/
void shell_init(const char *prompt, const struct shell_cmd *cmds);

* Suggested code:
/** Init shell fiber and register serial console handler. Commands are
initialized seperately*/
void shell_init(const char *prompt);

struct shell_entity {
const char *entity_name;
struct shell_cmd *commands;
};

#ifdef CONFIG_ENABLE_FW_SHELL
#define REGISTER_TO_SHELL(shell_name, shell_commands) \
\
static struct shell_entity (__shell_shell_name) __used \
__attribute__((__section__(".shell_"))) = { \
.entity_name = shell_name, \
.commands = shell_commands \
}

#else
#define REGISTER_TO_SHELL(shell_name, shell_commands)
#endif

Explanation:
- The shell entities and their commands need to be stored in a dedicated
code section whose boundaries are identified by its start and end
addresses.
- Memory for shell usage is not allocated if shell is disabled in
framework level, even if it is enabled in module level.

Shell Section
-----------------
1. Architecture dependency:
For all supported architectures - appropriate section prolog should be
declared.
Example- X86 arch:
File: include/arch/x86/linker-common-sections.h
SECTION_PROLOGUE(initshell, (OPTIONAL),)
{
SHELL_INIT_SECTIONS()
KEXEC_PGALIGN_PAD(MMU_PAGE_SIZE)
} GROUP_LINK_IN(RAM)

2. Architecture independence:
Identical for all architectures - shell section implementation:
File: include/linker-defs.h
#define SHELL_INIT_SECTIONS() \
__shell_cmd_start = .; \
KEEP(*(".shell_*")); \
__shell_cmd_end = .;

3. Section usage in code:
File: drivers/console/console_handler_shell.c
This file is the console handler implementation of shell.h API
* Current code:
static const struct shell_cmd *commands;
Each access to the commands array is done via commands array.

* Suggested code:
extern struct shell_entity __shell_cmd_start[];
extern struct shell_entity __shell_cmd_end[];
#define NUM_OF_SHELL_ENTITIES (__shell_cmd_end - __shell_cmd_start)
Each access to the commands array should be done via
__shell_cmd_start[entity].

============
Shell format
============
Specific command:
------------------------
* ENTITY COMMAND

Help commands:
---------------------
* help
Prints the string of all the entities.
* help ENTITY
Prints the available commands' names for the entity.
* help ENTITY COMMAND
Prints the help for the command of the entity (Depends on programmer
- the help should show function goal and required parameters).

=============================
Finding the correct call back
=============================
How do the shell service find the appropriate entity and command?
* Current implementation:
Search the string in commands array (strcmp)

* Suggested implementation:
First - search the entity.
Then - search the string in the specific's entity commands array.
The search is done by strcmp.

Note:
Relevant help is shown in case the search has failed.

======
Notes
======

* Non - preemption:
Due to the non-preemptive nature of the nanokernel's scheduler, a fiber
that performs lengthy computations may cause an unacceptable delay in the
scheduling of other fibers, including higher priority and equal priority
ones.
Therefore, shell callbacks should be short as possible.

* Shell fiber priority:
Should be the lowest among all fibers.
Currently it's not the lowest. For example, workqueue fibers has lower
priority. This should be changed.

===========
Open issues
===========
* Improve search:
Search (of the entity and then of the command) can be improved, for
example, by using hash tables.
This improvement has a conflict between Performance issue and Memory
consumption.
The improvement may be different between nanokernel and microkernel.
This improvement is currently out of this proposal scope.

---------------------------------------------------------------------
A member of the Intel Corporation group of companies

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Tomasz Bursztyka
 

Hi Yael,

Clear RFC.

I have tiny inputs below

Shell service (shell_service.c)

------------------------------------

• Add a new file: kernel/nanokernel/shell_service.c

• Responsibility: Initializes shell and add framework usage of the shell.

• Note: Memory for a specific module shell usage is not allocated if shell

is disabled in framework level, even if it is enabled in module’s level.

#define SHELL_PROMPT "shell> "

#define SHELL_FW "FW"

struct shell_cmd fw_commands[] = {

{ "sample", shell_cmd_sample, "help sample info" },

{ NULL, NULL }
As you know the size of fw_commands (which can be declared as const
btw) at built time,
you will be able to loop directly on that count, thus you could remove
this last entry.

===================

Memory consumption

===================

File: include/misc/shell.h

This file is the shell header file.

The following code already exists in its current Zephyr version:

typedef int (*shell_cmd_function_t)(int argc, char *argv[]);

struct shell_cmd {

const char *cmd_name;

shell_cmd_function_t cb;

const char *help;

};

• Note: Currently, “help” string’s length is not limited.

However, limitation on the length of each command’s help string
should be

added.
And supporting such help could be Kconfig based.
You may want to build the commands without their help string.

Shell Section

-----------------

1. Architecture dependency:

For all supported architectures – appropriate section prolog should be

declared.

Example- X86 arch:

File: include/arch/x86/linker-common-sections.h

SECTION_PROLOGUE(initshell, (OPTIONAL),)

{

SHELL_INIT_SECTIONS()

KEXEC_PGALIGN_PAD(MMU_PAGE_SIZE)

} GROUP_LINK_IN(RAM)
I don't see any blocker not to put it in romable region or?

===========

Open issues

===========

• Improve search:

Search (of the entity and then of the command) can be improved, for

example, by using hash tables.

This improvement has a conflict between Performance issue and Memory

consumption.

The improvement may be different between nanokernel and microkernel.

This improvement is currently out of this proposal scope.
Indeed there would be a lot to try here, from a very basic alphabet
index to an hash table
or search tree etc...

Tomasz


Luiz Augusto von Dentz
 

Hi Yael,

On Wed, Jul 13, 2016 at 2:05 PM, Avramovich, Yael
<yael.avramovich(a)intel.com> wrote:
Zephyr Shell Enhancement

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



========

Overview

========

Current shell implementation:

• Current Shell implementation is based on UART.

It uses one UART for console (UART number is determined by config
file).



• Console device driver initialization priority:

Console has to be initialized after the UART driver it uses.



• When the shell is initialized, it starts the shell as a fiber and

initialize UART console.



• Current shell implementation supplies shell utilities only for a
single

module.



=====

Goals

=====

1. Multiple and dynamic usage in modules level:

----------------------------------------------------------

Ability to add multiple modules to shell commands array.

Each module should be added dynamically by its config file.



2. Dynamic usage of the shell:

------------------------------------

Ability to enable / disable shell service.

If it is enabled (according to .Kconfig), basic framework shell commands

are also available for use.



3. Memory consumption:

------------------------------

Shell commands array should be allocated exactly according to the number
of

entities that actually use the shell.



4. Shell format:

------------------

Use one of the following (Error message is shown in case of a wrong
usage):

- ENTITY COMMAND

- help

- help ENTITY

- help ENTITY COMMAND

One of the entities is “FW”, for the Framework module.
Just have a command to select the module you want to operate, e.g.
select-module bluetooth, that way one don't have to keep entering the
entity name always. The prompt can also reflect the current module in
use.