Linux driver: How to enable dynamic debug at booting time for built-in driver.

 Dynamic debug is useful for debug driver, and can be enable by:

1. Mount debug fs

#>mount -t debugfs none /sys/kernel/debug

2. Enable dynamic debug for specifiy funcition

 echo 'file virtio_pci_modern.c +p' > /sys/kernel/debug/dynamic_debug/control

However somtimes we need to debug built-in driver and need enable dynamic debug at boot time. then we can enable by passing dyndbg to Kernel command line like

dyndbg="'file virtio_pci_modern.c +p"

with QEMU we can use -append to attach dyndbg

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -append "dyndbg="file virtio_pci_modern.c +p""

Ref:

  Kernel document/dynamic-debug-howto

 https://www.kernel.org/doc/html/v4.11/admin-guide/dynamic-debug-howto.html

Edge AI - Convert training speech data to micro feature.

In micro speech project we can use training data and convert it to micro feature so that we can test model accuracy in MCU without bothering MIC gain or signal chain issue.

Tensorflow provide a python script to generate feature from a input data.

Steps:

1. Download datasets.

2. Using tensorflow/tensorflow/examples/speech_commands/wav_to_features.py to convert input data. (!!Notice: Must use --preprocess=micro for tensorflow micro model comparability.

Here is my colab note book to generate 10 random feature with specific keyword.

Ref.

 1. https://stackoverflow.com/questions/701402/best-way-to-choose-a-random-file-from-a-directory

 2.https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/micro/examples/micro_speech/train/README.md

Hexdump: Covert file content to hex array.

 Hexdump can dump file content with hex format,  with parameter -e can format displaying data and it's useful in writing shell.

Example usage:

#> hexdump -v FILE -e '1/1 "0x%02X "'

Test result:

Ref:

  Hexdump man page

  https://man7.org/linux/man-pages/man1/hexdump.1.html

Edge AI - Train micro_speech model and deploy to NXP FRDM-K66F.

Tensorflow lite micro_speech example provide training method for speech recognize model to different key word detect.

In this article we use Google Colaboratory to train micro_speech model.

1. Open note book and change WANTED_WORDS
    I choose ON/OFF for my project.

# A comma-delimited list of the words you want to train for.
# The options are: yes,no,up,down,left,right,on,off,stop,go
# All the other words will be used to train an "unknown" label and silent
# audio data with no spoken words will be used to train a "silence" label.
WANTED_WORDS = "on,off"

2. Follow instruction and you can see training  progress(Model accuracy for training set and validation set) in tensorboard section.

3. Convert to TF-lite model and test TensorFlow Lite model's accuracy

5. Modify micro_speech/micro_features/micro_model_settings.cc kCategoryLabels to on/off

6. Replace micro_speech/micro_features/model.cc  to new trained model.

7. Build and flash 

8. Test result 

I use K66F onboard mic to test and the accuracy is lower than I expect. But the trained model has 80% more accuracy, so maybe need to use line mic to get better accuracy speech recognize result.

Edge AI - Modify micro speech example MIC gain on NXP FRDM-K66F for better accuracy.

 The example code recognize accuracy is lower than I expect. So I try to increase microphone gain from 0 dB to +6 dB and the result is better now.

1. Change DA7212 configuration in tensorflow/lite/micro/examples/micro_speech/nxp_k66f/audio_provider.cc

volatile uint8_t g_da7212_register_config[da7212ConfigurationSize][2] = {

...

   {0x34, 0x07},                              // Change MIXIN to 6 dB

    {0x35, 0x07},                         // Change MIXIN to 6 dB

....

2. Build and flash new binary to NXP_K66F board and test.

Ref 

DA7212 datasheet 

Edge AI - Run tensorflow lite micro speech expample on NXP FRDM-K66F.

 Step to build tf-lite micro speach example for nxp_k66f

1. Clone tensorflow code

  $>git clone https://github.com/tensorflow/tensorflow.git

  

2. Install mbed-cli 

     $>sudo apt install python3 python3-pip git mercurial

     $>python3 -m pip install mbed-cli

3. Install ARM cross compile toolchain.(In this project I use GNU Arm Embedded Toolchain)

https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads

4. Configure mbed cli

  $>mbed config -G  GCC_ARM_PATH=/YOUR_COMPILER_INSTALL_PATH/gcc-arm-none-eabi-9-2020-q2-update/bin

5. Generate mbed project for nxp_k66f

  $>cd tensorflow

  $>make -f tensorflow/lite/micro/tools/make/Makefile TARGET=mbed TAGS="nxp_k66f" generate_micro_speech_mbed_project

6.  Create mbed project 

  $>cd tensorflow/lite/micro/tools/make/gen/mbed_cortex-m4/prj/micro_speech/mbed

  $>mbed new  .

7. Configure porject usin C++ 11

python -c 'import fileinput, glob;

for filename in glob.glob("mbed-os/tools/profiles/*.json"):

  for line in fileinput.input(filename, inplace=True):

    print line.replace("\"-std=gnu++14\"","\"-std=c++11\", \"-fpermissive\"")'

 

8 Connect the USB cable to nxp_k66f

9. Compile and flash project, after flash complete reset the board.

mbed compile --target K66F --toolchain GCC_ARM --profile release --flash

10. Connect to nxp_k66f serial port with baudrate 14400.

11. Saying Yes and NO and will print repsonse word in serial port, since nxp_k66f has onboard microphone you can direct speak to it and see the result in terminal.

12. Test result 

I want using this sample code to build a IOT switch that can control with speach, So  in next article I will try to retrain model with 'ON' and 'OFF' key word and try to deploy to nxp_k66f

Ref

Mbed

https://os.mbed.com/docs/mbed-os/v6.9/build-tools/install-and-set-up.html

Tensorflow lite

https://github.com/tensorflow/tensorflow/tree/114b8ef31ac66155ec9b0590bc7115125f7fe61e/tensorflow/lite/micro/examples/micro_speech#deploy-to-nxp-frdm-k66f

NXP_K66F user guide

https://www.nxp.com/docs/en/user-guide/FRDMK66FUG.pdf

rcc: Simple C compiler

Motivation: 

By implement a simple C compiler to know better on modern compiler and low level hardware.

Git repo:

https://github.com/a46554/rcc/tree/master

Reference:

https://github.com/koshizuow

https://koshizuow.gitbook.io/compilerbook/calculator_level_language/

Docker useful command.

1. List running docker machine 

docker ps

2.  Attach to specific docker machine

docker exec -it docker_helloworld -u admin -p admin

3. Copy file from/to docker machine 

docker cp test.txt  docker_helloworld :test.txt  

Reference:

https://docs.docker.com/

Testing the tests: What should we do when the test case self is buggy?

Nowadays, software testing is widely using to ensure quality and prevent bug, like unit test/AB test…. etc, But increasingly test case will causing some unwanted problem like the test case is buggy so the testing will failed. And causing developer waste time chasing down problems that potentially didn’t really exist

Facebook provide a method to detect the test case bug:

1. Using ML technology  to predict what test case to run 

2. All end-to-end tests will have some degree of flakiness, So make a index about how reliable in these test case

3. Assert that a test is sufficiently reliable and provide a scale to illustrate which tests are less reliable than they should be. 

Software testing is a good tool to ensure the product quality, with more complicated code base and more complicated test case, it need carefully consider introduce method that test the test case to prevent waste time and increase testing quality.

Performance tuning- Leveraging modern CPU branch predict mechanism.

In modern CPU the branch predictor is complicated and the deeper pipeline causing the cost of miss branch prediction higher.

How can we leveraging the modern CPU branch predictor, In [1] example code, the branch miss rate can  be reduce by just sorting input data before exciting original algorithm. 

Conclusion 

1. Adding pattern to your algorithm(sorted data).

2. Using likely()/unlikely() marco to help branch prediction more accuracy.

Reference:

1. Performance analysis in Linux

2. Code DetectiveHow to investigate LinuxPerformance Issues

3. Linux likely() unlikely() MARCO.

Machine Learning Foundations NLP(2): Using the Sequencing APIs

Using the Sequencing APIs

Sequencing is use to format sentence array using token,

For example:

"I have a car"

"Car have a door"

 

Tokenize: [I:1] [have:2] [a:3] [car:4] [door:5]

 

then these two sentence can be represent to:

[1 2 3 4] 

[4 2 3 5]

Sequencing is useful to represent sentence data and take as input for a neuron network.

Code

Result:

Reference:

Machine Learning Foundations: Ep #9 - Using the Sequencing APIs

Machine Learning Foundations NLP(1): Tokenization for Natural Language Processing

Tokenization for Natural Language Processing

Tokenize is mean to break down a sentence to server work, for example:

I have a car. -> I / have / a/ car

Tensorflow provide a tokenize tool:Tokenizer, it can easily to use for tokenize input sentence.

Code:

Result:

{'have': 1, 'a': 2, 'i': 3, 'he': 4, 'apple': 5, 'bike': 6, 'pen': 7, 'car': 8}

Reference:

Machine Learning Foundations: Ep #8 - Tokenization for Natural Language Processing

Linux kernel module: Add entry to debugfs and a read/write file.

Goal

This module will create Ray_DBG directory in debugfs and create a file REG for read/write. it's similar to previous module(create /proc/Ray), but easier to use and need less line of code.

 

Code

Makefile

Usage

1. Insert module 

$>insmod test_module.ko
$>dmesg | tail
[7501255.251763] Create Ray_DBG !
[7501255.251776] Create REG debug !

2. Mount debugfs 

$>mount -t debugfs none /sys/kernel/debug

3. Read data

$>cat /sys/kernel/debug/Ray_DBG/REG
$>0x000000aa

4. Write data 

$>echo 0xff>  /sys/kernel/debug/Ray_DBG/REG 
$>cat /sys/kernel/debug/Ray_DBG/REG 
$>0x000000ff

Reference

1. Kernel document: debugfs
2. Debugfs

Linux kernel module: Add entry in /proc and passing args while insmod

Goal 

Implement a kernel module that can passing parameter when insert module, and add an entry to /proc that can write/read data. In embedded linux often using similar facility to help debug.

Code:

Makefile:

Usage:

1. Insert module and make /proc/Ray entry

 #>sudo insmod test_module.ko entry="Ray" mode=1238

 

2. Write data 

#>echo 123 > /proc/Ray

3. Read data

 #>cat /proc/Ray

 

Reference:

1. Linux Device Drivers Development: Develop customized drivers for embedded Linux
2. LINUX KERNEL DEVELOPMENT – CREATING A PROC FILE AND INTERFACING WITH USER SPACE

DeepSpeech: Speech to text AI model.

• Overview

"DeepSpeech is an open source Speech-To-Text engine, using a model trained by machine learning techniques based on Baidu’s Deep Speech research paper."

DeepSpeech provide lots of lauange api support, Python Javascript, c, and it's easily use to involve in application

• Install DeepSpeech

Follow user guide instruction.

• Demo

Using command line tool to inference sound data.

$> deepspeech --model deepspeech-0.7.0-models.pbmm --audio audio/2830-3980-0043.wav

Output:

Loading model from file deepspeech-0.7.0-models.pbmm
TensorFlow: v1.15.0-24-gceb46aa
DeepSpeech: v0.7.1-0-g2e9c281
Loaded model in 0.0093s.
Loading scorer from files deepspeech-0.7.0-models.scorer
Loaded scorer in 0.00023s.
Running inference.
experience proves this
Inference took 1.480s for 1.975s audio file.

 

The red color string is inference text data of input sound data.

Using Python API to inference sound data.

Output:

your power is sufficient i said sound data

Reference:

1. DeepSpeech 0.7.x Examples
2. DeepSpeech Document
3. Python library-wave

Machine Learning Foundations: Exercise 4 Happy and sad image classify model with 99.9% accuracy .

Machine Learning Foundations: Exercise 4 Happy and sad image classify model with 99.9% accuracy:code lab link

Build a mode to classify happy and sad model with convolution neuron network. with more 99.9% accuracy.

 

Note:

•  Model over-fitting: Near 100% accuracy on training data, but lower accuracy on testing data.
•  ImageDataGenerator:  Label data automatic using directory name.

Code: 

Conference note: Making C Less Dangerous in the Linux kernel - Kees Cook

In this conference discuss below topic about unsafely C usage, and how Linux kernel has remove or add  facility to detect such condition.

1. Variable Length array

• Using compiler option to Detect VLA: gcc -W vla
• Using guard page tor prevent stack overflow. VLA is needed lots of instruction compared to the fixed-size array.

2. Switch case break or non-break

• Mark all non-breaks with a “fall through” to whether programmer intent to fall through or it's a bug.
• Compiler support this feature: -Wimplicit-fallthrough

3. Arithmetic overflow detection

• Using compiler option to detection overflow in compile time
• Support different warning label: ignore or take as warring

4. Compare different API for string copy

• The safer string copy function: strscpy().

5. Safe stack - Shadow stack

• Separate the local variable stack and return address stack
• Support by hardware:
• ARM pointer authentication (Sign the return address for distinguish between a local variable and return address 

Reference :

1. Making C Less Dangerous in the Linux kernel - Kees Cook

Conference note: Safety vs Security: A Tale of Two Updates - Jeremy Rosen, Smile.fr

Safety and Security are different aspects of a system, briefly discuss the differences between them and challenge to build a system with both concerns.

• Safety: Proof and make it simple for reliable

1. Completely define spec.
2. Proven: Check every state of the system is meet the design.
3. Change as less as possible: If a bug can solve by period reboot machine, just period reboot machine rather than update the software.

• Security: Prevent system been using in unwanted way

1. Fasting iteration: Change the old encryption algorithm to a new one.
2. Preventive: Need update the spec to face the new challenge.

These two aspects often neglect in embedded system nowadays, especially in customer market. but with more and more device connected to the internet and responsible for critical tasks like healthcare. Designer need to take care of these two fields, Include them in the beginning of designing a system and try to make an optimized combination of them to meet the system standards.

Reference : 

        1. Safety vs Security: A Tale of Two Updates - Jeremy Rosen, Smile.fr

Machine Learning Foundations: Exercise 3 Improve accuracy of MNIST using Convolutions.

Exercise 3: Improve accuracy of MNIST using Convolutions:code lab link

Improve MNIST to 99.8% accuracy or more using only a single convolutional layer and a single MaxPooling 2D.

The filter amount will affect the accuracy and training time.

Code: 

GO: Fixed warning: go env -w GOPATH=... does not override conflicting OS environment variable

Because the GOPATH is OS level setting variable so it can't be set using go env -v cmd.

We need to use OS specify way to modify this PATH variable.

1. Using CMD to set GOPATH variable (Environment: WIN10)

    > setx GOPATH ##YOURPATH##

2. Check the GOPATH been setup 

   > go env 

Reference 

     1. go 1.13 go env -w 設置GOPROXY報錯

     2. GO/env_write.txt

     3.https://github.com/golang/go/wiki/SettingGOPATH#windows-10-gui