rcc: Simple C compiler

Motivation: 

By implement a simple C compiler to know better on modern compiler and low level hardware.

Git repo:

https://github.com/a46554/rcc/tree/master

Reference:

https://github.com/koshizuow

https://koshizuow.gitbook.io/compilerbook/calculator_level_language/

Docker useful command.

1. List running docker machine 

docker ps

2.  Attach to specific docker machine

docker exec -it docker_helloworld -u admin -p admin

3. Copy file from/to docker machine 

docker cp test.txt  docker_helloworld :test.txt  

Reference:

https://docs.docker.com/

Testing the tests: What should we do when the test case self is buggy?

Nowadays, software testing is widely using to ensure quality and prevent bug, like unit test/AB test…. etc, But increasingly test case will causing some unwanted problem like the test case is buggy so the testing will failed. And causing developer waste time chasing down problems that potentially didn’t really exist

Facebook provide a method to detect the test case bug:

1. Using ML technology  to predict what test case to run 

2. All end-to-end tests will have some degree of flakiness, So make a index about how reliable in these test case

3. Assert that a test is sufficiently reliable and provide a scale to illustrate which tests are less reliable than they should be. 

Software testing is a good tool to ensure the product quality, with more complicated code base and more complicated test case, it need carefully consider introduce method that test the test case to prevent waste time and increase testing quality.

Performance tuning- Leveraging modern CPU branch predict mechanism.

In modern CPU the branch predictor is complicated and the deeper pipeline causing the cost of miss branch prediction higher.

How can we leveraging the modern CPU branch predictor, In [1] example code, the branch miss rate can  be reduce by just sorting input data before exciting original algorithm. 

Conclusion 

1. Adding pattern to your algorithm(sorted data).

2. Using likely()/unlikely() marco to help branch prediction more accuracy.

Reference:

1. Performance analysis in Linux

2. Code DetectiveHow to investigate LinuxPerformance Issues

3. Linux likely() unlikely() MARCO.

Machine Learning Foundations NLP(2): Using the Sequencing APIs

Using the Sequencing APIs

Sequencing is use to format sentence array using token,

For example:

"I have a car"

"Car have a door"

 

Tokenize: [I:1] [have:2] [a:3] [car:4] [door:5]

 

then these two sentence can be represent to:

[1 2 3 4] 

[4 2 3 5]

Sequencing is useful to represent sentence data and take as input for a neuron network.

Code

Result:

Reference:

Machine Learning Foundations: Ep #9 - Using the Sequencing APIs

Machine Learning Foundations NLP(1): Tokenization for Natural Language Processing

Tokenization for Natural Language Processing

Tokenize is mean to break down a sentence to server work, for example:

I have a car. -> I / have / a/ car

Tensorflow provide a tokenize tool:Tokenizer, it can easily to use for tokenize input sentence.

Code:

Result:

{'have': 1, 'a': 2, 'i': 3, 'he': 4, 'apple': 5, 'bike': 6, 'pen': 7, 'car': 8}

Reference:

Machine Learning Foundations: Ep #8 - Tokenization for Natural Language Processing

Linux kernel module: Add entry to debugfs and a read/write file.

Goal

This module will create Ray_DBG directory in debugfs and create a file REG for read/write. it's similar to previous module(create /proc/Ray), but easier to use and need less line of code.

 

Code

Makefile

Usage

1. Insert module 

$>insmod test_module.ko
$>dmesg | tail
[7501255.251763] Create Ray_DBG !
[7501255.251776] Create REG debug !

2. Mount debugfs 

$>mount -t debugfs none /sys/kernel/debug

3. Read data

$>cat /sys/kernel/debug/Ray_DBG/REG
$>0x000000aa

4. Write data 

$>echo 0xff>  /sys/kernel/debug/Ray_DBG/REG 
$>cat /sys/kernel/debug/Ray_DBG/REG 
$>0x000000ff

Reference

1. Kernel document: debugfs
2. Debugfs