2014年4月8日星期二

Linux touchscreen with tslib

Linux touchscreen 的工作分為兩大部份

1.Touchscreen driver
  通常透過SPI interface與Touchscreen A/D controller取得座標及觸控事件/壓力.
  我手邊的device是TI TSC2046 A/D controller. 座標及觸控壓力資料為12bits.
  User space則透過input device讀取資料,例如 /dev/input/event0.

2.Touchscreen library
  User space讀取到touchscreen 的資料還需要經過處理才能使用,
  例如轉換為銀幕的座標位置,濾除錯誤的資料(noise)等. 
  tslib這個library已經有這些模組功能.
  透過tslib, Graphic user interface 可以取得相對應座標的touchscreen 動作完成UI反應.

tslib

取得tslib
git clone https://github.com/kergoth/tslib.git

編譯

cd tslib

./autogen.sh

CC=mipsel-linux-uclibc-gcc AR=mipsel-linux-uclibc-ar ./configure --host=mipsel-linux --target=mipsel-linux --prefix=/usr/local/MIPSEL_UCLIBC --cache-file=/dev/null

make
make install

tslib在target的安裝位置

/phone/lib/libts-1.0.so.0.0.0
/phone/lib/ts/dejitter.so  
/phone/lib/ts/input.so    
/phone/lib/ts/linear.so    
/phone/lib/ts/pthres.so    
/phone/lib/ts/variance.so
/phone/etc/ts.conf
/phone/bin/ts_calibrate
/phone/bin/ts_print

tslib 模組

1.Variance filter - 過濾隨機的噪音資料.
2.Dejitter - 平均化X及Y軸的資料.
3.Linear - 轉換touchscreen座標至銀幕座標.

tslib ts.conf 內容

# Uncomment if you wish to use the linux input layer event interface
module_raw input

# Uncomment if you're using a Sharp Zaurus SL-5500/SL-5000d
# module_raw collie

# Uncomment if you're using a Sharp Zaurus SL-C700/C750/C760/C860
# module_raw corgi

# Uncomment if you're using a device with a UCB1200/1300/1400 TS interface
# module_raw ucb1x00

# Uncomment if you're using an HP iPaq h3600 or similar
# module_raw h3600

# Uncomment if you're using a Hitachi Webpad
# module_raw mk712

# Uncomment if you're using an IBM Arctic II
# module_raw arctic2

module pthres pmin=1
module variance delta=30
module dejitter delta=100
module linear

tslib 環境變數

tslib根據以下的環境變數取得模組及設定檔案(ts.conf / pointercal), ㄧ定要正確設定.

export TSLIB_TSDEVICE=/dev/input/event0
export TSLIB_CONFFILE=/phone/etc/ts.conf
export TSLIB_PLUGINDIR=/phone/lib/ts
export TSLIB_FBDEVICE=/dev/fb0
export TSLIB_CALIBFILE=/phone/etc/pointercal 

tslib 校正(ts_calibrate)

執行 /phone/bin/ts_calibrate
依 左上, 右上, 右下, 左下, 中間 的順序觸控 touchscreen 就完成校正.
校正資料會儲存在/phone/etc/pointercal 

tslib 測試(ts_print)

執行 ts_print 會讀取/phone/etc/pointercal, 在 console 上顯示目前觸控的銀幕座標





2013年6月17日星期一

C++ program without libstdc++

在 Embedded system 上儲存空間是相當珍貴的,目前除了智慧型手機上有Giga等級的flash儲存空間,其他能有個8M flash已經是相當奢侈了.
在這麼小的flash中想用c++來program最大的問題就是libstdc++.so實在太肥了.

gigijoe@gigijoe-i5-2500k:/opt/arm-linux-uclibcgnueabi/lib$ ls libstdc++* -al
-rw-r--r-- 1 root root 7233624 8月 18 2010 libstdc++.a
-rwxr-xr-x 1 root root 1307 8月 18 2010 libstdc++.la
-rw-r--r-- 1 root root 7224318 8月 18 2010 libstdc++_pic.a
lrwxrwxrwx 1 root root 18 8月 18 2010 libstdc++.so -> libstdc++.so.6.0.8
lrwxrwxrwx 1 root root 18 8月 18 2010 libstdc++.so.6 -> libstdc++.so.6.0.8
-rwxr-xr-x 1 root root 3750191 8月 18 2010 libstdc++.so.6.0.8

這個library包含了c++ STL / dynamic_cast / exception等功能,往後c++11,c++14等規格再陸續實作後只會越來越大.那麼用c來program就好啦,幹嘛找麻煩用c++啊?
c++ 有許多feature能讓程式更加結構化並容易閱讀,例如 function overload.
物件導向所提供的 繼承 / 多型 更能讓整個程式有更多的彈性.

以下是在網路上找到的方式,我自己也驗證過可行.
限制是不能使用 STL / dynamic_cast / exception

http://ptspts.blogspot.tw/2010/12/how-to-write-c-program-without-libstdc.html

以g++ compile .cpp ( -fno-rtti -fno-exceptions ), 以 gcc link 所有的.o

gigijoe@gigijoe-i5-2500k:~/SourceCode/test$ ldd main
linux-gate.so.1 => (0xb773c000)
libpicowm.so (0xb7736000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7560000)
/lib/ld-linux.so.2 (0xb773d000)

libstdc++.so 已經不再需要囉 :)



2013年3月12日星期二

Fixed-Point Fast Fourier Transform (FFT)

Fast Fourier Transform is very important for signal processing.
This time I will demostrate Fixed-Point FFT running on embedded system.
There's NO FPU on most of embedded CPU and most of implement of FFT using floating point variable.
This means bad performance of FFT calculation, so I found an Fixed-Point FFT library Written by Tom Roberts which fit the requirement.

The purpose of using FFT is to detect DTMF in 16 bits / 8k sample rate raw sound data.
DTMF is multi-frequency signal which mixture of two difference tone.
This is the foundation of modern telephone system.

            1209 Hz 1336 Hz 1477 Hz
697 Hz    1             2            3
770 Hz    4             5            6
852 Hz    7             8            9
941 Hz    *             0            #

The procdure to detect DTMF is to pass raw sound data into FFT then check frequency domain to determinate which frequencies with higher energy.

1.Fixed-Point FFT
The prototype of fix_fft function

int fix_fft(short fr[], short fi[], short m, short inverse)

fr[] is the real part array, fi[] is the imaginary part array.
m is power of 2 of the array size, the array size must be power of 2.

For example the array size is 256 (2^8) then m is 8.

Pass input sound data into fr[] and leave all zero to fi[]
Return value of fix_fft is useless while forward FFT.
The result of FFT is storing in fr[] and fi[].

2.Phyical of FFT

It's not hard to realize more input data lead to more precision of result.
The input sound data is 16 bit / 8k sample rate. Assume number of input samples is 256 (2^8)
So we have 256 buckets divided over 8khz, each bucket represents 31.25 hz ( 8k hz / 256).
But, the maximum frequency we can measure is half of the sampling rate so if we sample at 8Khz our maximum is 4Khz

(this is called the Nyquist frequency).

0.000000 ~ 31.250000hz
31.250000 ~ 62.500000hz
62.500000 ~ 93.750000hz
93.750000 ~ 125.000000hz
125.000000 ~ 156.250000hz
156.250000 ~ 187.500000hz
187.500000 ~ 218.750000hz

...

...

...

3812.500000 ~ 3843.750000hz
3843.750000 ~ 3875.000000hz
3875.000000 ~ 3906.250000hz
3906.250000 ~ 3937.500000hz
3937.500000 ~ 3968.750000hz
3968.750000 ~ 4000.000000hz

So, the frequency domain is from 0 ~ 4k hz and how about the energy of each frequency range ?

energy = sqrt(fr[i]^2 + fi[i]^2), where i form 0 to 127

Then check the higher energy (Threshold) of frequency and see if the frequency fit the range of DTMF tone.

3.The source code and result
fix_fft.c
dtmf_detect.c
T159.snd

The target machine is MIPS 4KEc running 162Mhz. No FPU support.
Enable DEBUG flag to see what's going on, but terrible performance due to printf()
1.5 second sound data takes more than 4 seconds to process.

$ ./dtmf_detect T159.snd
Detect None ...
Detect None ...
Detect None ...
Detect None ...
687.500000 ~ 718.750000hz : 2838
718.750000 ~ 750.000000hz : 2271
1218.750000 ~ 1250.000000hz : 3761
Max freq : 1218.750000 ~ 1218.750000hz (Value : 3761)
dtmf is 1
687.500000 ~ 718.750000hz : 3915
718.750000 ~ 750.000000hz : 2246
1218.750000 ~ 1250.000000hz : 4853
Max freq : 1218.750000 ~ 1218.750000hz (Value : 4853)
687.500000 ~ 718.750000hz : 3762
718.750000 ~ 750.000000hz : 2243
1218.750000 ~ 1250.000000hz : 6525
Max freq : 1218.750000 ~ 1218.750000hz (Value : 6525)
687.500000 ~ 718.750000hz : 4066
718.750000 ~ 750.000000hz : 2288
1218.750000 ~ 1250.000000hz : 5760
Max freq : 1218.750000 ~ 1218.750000hz (Value : 5760)
687.500000 ~ 718.750000hz : 3601
718.750000 ~ 750.000000hz : 2086
1218.750000 ~ 1250.000000hz : 5968
Max freq : 1218.750000 ~ 1218.750000hz (Value : 5968)
687.500000 ~ 718.750000hz : 4108
718.750000 ~ 750.000000hz : 2375
1218.750000 ~ 1250.000000hz : 6532
Max freq : 1218.750000 ~ 1218.750000hz (Value : 6532)
687.500000 ~ 718.750000hz : 3330
718.750000 ~ 750.000000hz : 2034
1218.750000 ~ 1250.000000hz : 4783
Max freq : 1218.750000 ~ 1218.750000hz (Value : 4783)
687.500000 ~ 718.750000hz : 3717
718.750000 ~ 750.000000hz : 2401
1218.750000 ~ 1250.000000hz : 5406
Max freq : 1218.750000 ~ 1218.750000hz (Value : 5406)
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
750.000000 ~ 781.250000hz : 2635
781.250000 ~ 812.500000hz : 2451
1312.500000 ~ 1343.750000hz : 2238
1343.750000 ~ 1375.000000hz : 2872
Max freq : 1343.750000 ~ 1343.750000hz (Value : 2872)
dtmf is 5
750.000000 ~ 781.250000hz : 3027
781.250000 ~ 812.500000hz : 3084
1312.500000 ~ 1343.750000hz : 2692
1343.750000 ~ 1375.000000hz : 4415
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4415)
750.000000 ~ 781.250000hz : 3119
781.250000 ~ 812.500000hz : 3196
1312.500000 ~ 1343.750000hz : 3174
1343.750000 ~ 1375.000000hz : 4766
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4766)
750.000000 ~ 781.250000hz : 3127
781.250000 ~ 812.500000hz : 3333
1312.500000 ~ 1343.750000hz : 3198
1343.750000 ~ 1375.000000hz : 4750
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4750)
750.000000 ~ 781.250000hz : 3129
781.250000 ~ 812.500000hz : 3451
1312.500000 ~ 1343.750000hz : 2803
1343.750000 ~ 1375.000000hz : 4114
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4114)
750.000000 ~ 781.250000hz : 3143
781.250000 ~ 812.500000hz : 3534
1312.500000 ~ 1343.750000hz : 3393
1343.750000 ~ 1375.000000hz : 4978
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4978)
750.000000 ~ 781.250000hz : 3144
781.250000 ~ 812.500000hz : 3561
1312.500000 ~ 1343.750000hz : 2593
1343.750000 ~ 1375.000000hz : 3631
Max freq : 1343.750000 ~ 1343.750000hz (Value : 3631)
750.000000 ~ 781.250000hz : 3195
781.250000 ~ 812.500000hz : 3544
1312.500000 ~ 1343.750000hz : 3246
1343.750000 ~ 1375.000000hz : 4971
Max freq : 1343.750000 ~ 1343.750000hz (Value : 4971)
1312.500000 ~ 1343.750000hz : 2850
1343.750000 ~ 1375.000000hz : 2935
Max freq : 1343.750000 ~ 1343.750000hz (Value : 2935)
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Detect None ...
843.750000 ~ 875.000000hz : 3998
1468.750000 ~ 1500.000000hz : 5109
Max freq : 1468.750000 ~ 1468.750000hz (Value : 5109)
dtmf is 9
843.750000 ~ 875.000000hz : 4803
1468.750000 ~ 1500.000000hz : 6069
Max freq : 1468.750000 ~ 1468.750000hz (Value : 6069)
843.750000 ~ 875.000000hz : 4620
1468.750000 ~ 1500.000000hz : 5350
Max freq : 1468.750000 ~ 1468.750000hz (Value : 5350)
843.750000 ~ 875.000000hz : 5024
1468.750000 ~ 1500.000000hz : 6536
Max freq : 1468.750000 ~ 1468.750000hz (Value : 6536)
843.750000 ~ 875.000000hz : 4157
1468.750000 ~ 1500.000000hz : 5645
Max freq : 1468.750000 ~ 1468.750000hz (Value : 5645)
843.750000 ~ 875.000000hz : 5140
1468.750000 ~ 1500.000000hz : 5768
Max freq : 1468.750000 ~ 1468.750000hz (Value : 5768)
843.750000 ~ 875.000000hz : 3629
1468.750000 ~ 1500.000000hz : 6446
Max freq : 1468.750000 ~ 1468.750000hz (Value : 6446)
Detect None ...
Detect None ...
Detect None ...
Detect None ...
Number samples : 12544, Duration : 4 seconds and 84537 usecs

Disable DEBUG with good performance.
1.5 second sound data takes 31.346 ms to process.

$ ./dtmf_detect T159.snd
dtmf is 1
dtmf is 5
dtmf is 9

Number samples : 12544, Duration : 0 seconds and 31346 usecs

2012年8月6日星期一

OpenCV 2.4.2 V4L2 Camera resolution bug fix

When using camera with opencv api, the default camera resolution is ALWAYS 160x120
From source code modules/highgui/src/cap_libv4l.cpp the default resolution should be 640x480

/* Defaults - If your board can do better, set it here.  Set for the most common type inputs. */
#define DEFAULT_V4L_WIDTH  640
#define DEFAULT_V4L_HEIGHT 480

I have study a little bit of modules/highgui/src/cap_libv4l.cpp then I found the problem.

   CvCaptureCAM_V4L * capture = (CvCaptureCAM_V4L*)cvAlloc(sizeof(CvCaptureCAM_V4L));
   if (!capture) {
      fprintf( stderr, "HIGHGUI ERROR: V4L: Could not allocate memory for capture process.\n");
      return NULL;
   }

   /* set the default size */
   capture->width  = DEFAULT_V4L_WIDTH;
   capture->height = DEFAULT_V4L_HEIGHT;

...
...

   /* w/o memset some parts  arent initialized - AKA: Fill it with zeros so it is clean */
   memset(capture,0,sizeof(CvCaptureCAM_V4L));

   /* Present the routines needed for V4L funtionality.  They are inserted as part of
      the standard set of cv calls promoting transparency.  "Vector Table" insertion. */
   capture->FirstCapture = 1;

   if (_capture_V4L2 (capture, deviceName) == -1) {

The memset instruction is called after setting width & height. @@

So, just move up memset behind alloc memory.

--- modules/highgui/src/cap_libv4l.cpp.org    2012-08-06 19:28:58.746347834 +0800
+++ modules/highgui/src/cap_libv4l.cpp    2012-08-06 19:29:42.510348966 +0800
@@ -1007,6 +1007,9 @@
       return NULL;
    }
 
+   /* w/o memset some parts  arent initialized - AKA: Fill it with zeros so it is clean */
+   memset(capture,0,sizeof(CvCaptureCAM_V4L));
+
    /* set the default size */
    capture->width  = DEFAULT_V4L_WIDTH;
    capture->height = DEFAULT_V4L_HEIGHT;
@@ -1028,8 +1031,6 @@
    /* Print the CameraNumber at the end of the string with a width of one character */
    sprintf(deviceName, "/dev/video%1d", index);
 
-   /* w/o memset some parts  arent initialized - AKA: Fill it with zeros so it is clean */
-   memset(capture,0,sizeof(CvCaptureCAM_V4L));
    /* Present the routines needed for V4L funtionality.  They are inserted as part of
       the standard set of cv calls promoting transparency.  "Vector Table" insertion. */
    capture->FirstCapture = 1;



2012年7月22日星期日

Ubuntu 12.04 stutter sound

The stuttering sound while playing mp3 or video bother me for a while.
After some googling I found solution

1.Edit
/etc/pulse/client.conf

from

; extra-arguments = --log-target=syslog

to

; extra-arguments = --log-target=syslog --resample-method=ffmpeg

2.Edit
/etc/modprobe.d/alsa-base.conf

Add

options snd-hda-intel model=generic

3.reboot

2012年5月29日星期二

S3C6410 MFC H.264 mp4 player (Bug fix)

In the previous post, I implement an mp4 player.
There's a problem confuse me.
The start of video frame seems broken. Then after some time (maybe seconds) the video frame goes well.

Finally, I found a way to solve it.
Just to decode the first frame twice. Really strange but work. Also there's some minor fix included

https://docs.google.com/open?id=0Bx901QMLr9WlVE1ldHVCaERkRkk

Patch

--- /home/gigijoe/mp4player.c    2012-05-29 16:11:06.094007930 +0800
+++ mp4player.c    2012-05-29 16:17:55.073995127 +0800
@@ -176,8 +176,12 @@
     pp_param.dst_width              = pp_param.dst_full_width;
     pp_param.dst_height             = pp_param.dst_full_height;
     pp_param.dst_color_space        = FB0_COLOR_SPACE;
+#if 1
     pp_param.out_path               = DMA_ONESHOT;
-
+#else
+        pp_param.out_path           = FIFO_FREERUN;
+        pp_param.scan_mode            = PROGRESSIVE_MODE;
+#endif
     ioctl(pp_fd, S3C_PP_SET_PARAMS, &pp_param);
     printf("stream width %d, height %d\n", pp_param.src_full_width, pp_param.src_full_height);
 
@@ -193,7 +197,7 @@
       return -1;
     }
 
-    memset(fb_addr, 0, fb_size);
+    //memset(fb_addr, 0, fb_size);
 
     s3c_win_info_t osd_info_to_driver;
     osd_info_to_driver.Bpp          = FB0_BPP;      // RGB16
@@ -212,6 +216,13 @@
     initialize = 1;
 
     //return 0;
+/*
+* Decode first frame again to avoid problem ...
+*/
+    if(SsbSipH264DecodeExe(handle, len) != SSBSIP_H264_DEC_RET_OK)  {
+      printf("MFC Decoder Configuration Failed.\n");
+      return -1;
+    }
   }
 
   unsigned int pYUVBuf[2];
@@ -229,6 +240,12 @@
   ioctl(fb_fd, FBIOGET_FSCREENINFO, &lcd_info);
   pp_param.dst_buf_addr_phy = lcd_info.smem_start;                  // LCD frame buffer
   ioctl(pp_fd, S3C_PP_SET_DST_BUF_ADDR_PHY, &pp_param);
+
+  struct pollfd test_fd;
+  test_fd.fd = pp_fd;
+  test_fd.events = POLLOUT|POLLERR;
+  poll(&test_fd, 1, 3000);
+
   ioctl(pp_fd, S3C_PP_START);
 
   return 0;
@@ -342,12 +359,15 @@
   AVCodecContext *codecContext = is->codec;
 
   long long delay = (1000000 * is->r_frame_rate.den) / (is->r_frame_rate.num);
-
-#if 0
+#if 1
   i = 0;
 #endif
   while(1)  {
     r = av_read_frame(formatContext, &pkt);
+    if(r != 0)  { /*  No more frame, seek head and replay */
+      av_seek_frame(formatContext, 0, 1, AVSEEK_FLAG_BACKWARD);
+      r = av_read_frame(formatContext, &pkt);
+    }
     if(r != 0)
       break;  /*  No more frame */
 
@@ -367,11 +387,16 @@
         i, pkt.stream_index, pkt.dts, pkt.size, pkt.data);
     av_pkt_dump(stdout, &pkt, 0);
 #endif
+    if(pkt.flags & PKT_FLAG_KEY)
+      printf("Key frame %d\n", i);
 
     unsigned char *lp = 0;
     int len = 0;
-    filterContext->filter->filter(filterContext, codecContext, NULL, &lp, &len, pkt.data, pkt.size, (pkt.flags | PKT_FLAG_KEY));
-
+ //   filterContext->filter->filter(filterContext, codecContext, NULL, &lp, &len, pkt.data, pkt.size, (pkt.flags | PKT_FLAG_KEY));
+    r = filterContext->filter->filter(filterContext, codecContext, NULL, &lp, &len, pkt.data, pkt.size, (pkt.flags));
+    if(r != 1)
+      printf("Frame %d : Fail to filter ...\n", i);
+//printf("Frame %d: %d\n", pkt.stream_index, len);
     decode_mfc(lp, len);
 
     av_free_packet(&pkt);
@@ -393,7 +418,7 @@
       } else
         break;
     }
-#if 0
+#if 1
     i++;
 #endif
   }
@@ -403,6 +428,10 @@
 
   avcodec_close(is->codec);
 
+  munmap(fb_addr, fb_size);
+    close(pp_fd);
+    close(fb_fd);
+
   deinit_mfc();
 
   return 0;


2012年4月10日星期二

S3C6410 MFC H.264 mp4 player

S3C6410 can do hardware video decode including H.264.
There's an example code from Samsung shows how to hardware decode H.264 raw data, but in the real world application this is not so useful.
Decode & Playing H.264 mp4 file is more reality. So I would like to share the implementation.

I use ffmpeg library to decode mp4 mux in order to get H.264 raw data and then send it to S3C6410 MFC to decode and finally play on the LCD panel.

Requirement
ffmpeg-0.6.1
Samsung FIMV_MFC_V1.0

Source code
https://docs.google.com/open?id=0Bx901QMLr9WlTTR1a0ZKaXVobW8

The most important part is to handle data from

  av_read_frame(formatContext, &pkt);

if you pass pkt.data to decode you got fail to decode. Another filter is required.

  typedef struct H264BSFContext {
    uint8_t  length_size;
    uint8_t  first_idr;
    uint8_t *sps_pps_data;
    uint32_t size;
  } H264BSFContext;

  AVBitStreamFilterContext *filterContext = (AVBitStreamFilterContext *)malloc(sizeof(AVBitStreamFilterContext));
  filterContext->priv_data = (H264BSFContext *)malloc(sizeof(H264BSFContext));
  memset(filterContext->priv_data, 0, sizeof(H264BSFContext));
  filterContext->filter = &h264_mp4toannexb_bsf;
  filterContext->parser = 0;
  filterContext->next = 0;
...
  r = av_read_frame(formatContext, &pkt);
...
    unsigned char *lp = 0;
    int len = 0;
    filterContext->filter->filter(filterContext, codecContext, NULL, &lp, &len, pkt.data, pkt.size, (pkt.flags | PKT_FLAG_KEY)); -->> filter here

    decode_mfc(lp, len);