git checkout cf910e83ae23692fdeefc7e506e504c4c468d38a HEAD is now at cf910e8... x86, trace: Add irq vector tracepoints ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:cf910e83ae23692fdeefc7e506e504c4c468d38a:bisect-stable TEST FAILURE /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-kbuild-6225-20130622055129-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-lkp-nex04-2081-20130622152756-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-snb-14725-20130622052350-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-snb-19353-20130622052322-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-snb-25577-20130622060025-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-snb-27565-20130622052322-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-snb-32784-20130622052346-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-stoakley-5619-20130622055353-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-stoakley-5727-20130622055306-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-10265-20130622052122-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-10808-20130622052115-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-11007-20130622055514-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-13669-20130622152755-3.10.0-rc6-00004-gcf910e8-190 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-16166-20130622052137-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-21561-20130622052122-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-vp-5231-20130622052136-3.10.0-rc6-00004-gcf910e8-189 /kernel/i386-randconfig-r00-0621/cf910e83ae23692fdeefc7e506e504c4c468d38a/dmesg-kvm-xgwo-3219-20130622055559-3.10.0-rc6-00004-gcf910e8-190 bisect: bad commit cf910e83ae23692fdeefc7e506e504c4c468d38a git checkout v3.9 Previous HEAD position was cf910e8... x86, trace: Add irq vector tracepoints HEAD is now at c1be5a5... Linux 3.9 ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:c1be5a5b1b355d40e6cf79cc979eb66dafa24ad1:bisect-stable 2013-06-22-15:28:23 c1be5a5b1b355d40e6cf79cc979eb66dafa24ad1 compiling 429 real 1403 user 130 sys 357.52% cpu i386-randconfig-r00-0621 2013-06-22-15:37:47 detecting boot state 3.9.0........... 2. 3 5 8 10 11. 13. 15. 18 20 22 26. 29 36 39. 40 45 48 52 SUCCESS bisect: good commit v3.9 git bisect start cf910e83ae23692fdeefc7e506e504c4c468d38a v3.9 -- Previous HEAD position was c1be5a5... Linux 3.9 HEAD is now at 521cb40... Linux 2.6.38 Bisecting: 7205 revisions left to test after this (roughly 13 steps) [fbcd4836d20a33209843dcf84f83a33b97b74c9a] [SCSI] qla4xxx: Assign values using correct datatype git bisect run /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:fbcd4836d20a33209843dcf84f83a33b97b74c9a:bisect-stable 2013-06-22-15:56:25 fbcd4836d20a33209843dcf84f83a33b97b74c9a compiling 280 real 1511 user 136 sys 587.25% cpu i386-randconfig-r00-0621 2013-06-22-16:02:44 detecting boot state 3.9.0-07205-gfbcd483.......... 2 4 6 7 11 17 18 22 24 30. 32 33 37 46 49 50 52 SUCCESS Bisecting: 3570 revisions left to test after this (roughly 12 steps) [91f8575685e35f3bd021286bc82d26397458f5a9] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:91f8575685e35f3bd021286bc82d26397458f5a9:bisect-stable 2013-06-22-16:16:48 91f8575685e35f3bd021286bc82d26397458f5a9 compiling 175 real 1519 user 149 sys 953.32% cpu i386-randconfig-r00-0621 2013-06-22-16:20:52 detecting boot state 3.9.0-10803-g91f8575........... 2 4 6 9 14 18 21 22 23 26 27.. 32 36 39 40 42. 49 51................................................................. 52 SUCCESS Bisecting: 1786 revisions left to test after this (roughly 11 steps) [c4cc75c3321cad6f20d1e5325293890255c8a663] Merge git://git.infradead.org/users/eparis/audit running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:c4cc75c3321cad6f20d1e5325293890255c8a663:bisect-stable 2013-06-22-17:09:55 c4cc75c3321cad6f20d1e5325293890255c8a663 compiling 177 real 1512 user 137 sys 928.77% cpu i386-randconfig-r00-0621 2013-06-22-17:13:10 detecting boot state 3.9.0-12587-gc4cc75c.......... 1 4 6 9 16 17 23 24 26 30 35 40 44 47 50 51.............................................................................................................................................................................................................................................................................................................add_to_run_queue 1 ............ 52 SUCCESS Bisecting: 893 revisions left to test after this (roughly 10 steps) [3a5395b3d57b9e3836c755434c88f4590d5ea6f6] net: ethernet: xilinx_emaclite: set protocol selector bits when writing ANAR running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:3a5395b3d57b9e3836c755434c88f4590d5ea6f6:bisect-stable 2013-06-22-20:03:15 3a5395b3d57b9e3836c755434c88f4590d5ea6f6 compiling 214 real 1506 user 130 sys 762.22% cpu i386-randconfig-r00-0621 2013-06-22-20:07:02 detecting boot state 3.10.0-rc2-00400-g3a5395b......... 2 3 7 15 32 50.. 51. 52 SUCCESS Bisecting: 445 revisions left to test after this (roughly 9 steps) [e6395b68ad09a835f058da31bad0fe23d3882659] Merge tag 'for-linus-v3.10-rc5' of git://oss.sgi.com/xfs/xfs running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:e6395b68ad09a835f058da31bad0fe23d3882659:bisect-stable 2013-06-22-20:17:04 e6395b68ad09a835f058da31bad0fe23d3882659 compiling 163 real 1504 user 131 sys 1000.97% cpu i386-randconfig-r00-0621 2013-06-22-20:20:02 detecting boot state 3.10.0-rc4-00164-ge6395b6......... 1 4 19 21 26 50 52 SUCCESS Bisecting: 222 revisions left to test after this (roughly 8 steps) [5402b8047b0d286b6501f9097891cbf1e06daa3a] lib/mpi/mpicoder.c: looping issue, need stop when equal to zero, found by 'EXTRA_FLAGS=-W'. running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:5402b8047b0d286b6501f9097891cbf1e06daa3a:bisect-stable 2013-06-22-20:28:07 5402b8047b0d286b6501f9097891cbf1e06daa3a compiling 173 real 1503 user 132 sys 945.13% cpu i386-randconfig-r00-0621 2013-06-22-20:32:37 detecting boot state 3.10.0-rc5-00065-g5402b80.......... 8 13 14 32 46 52 SUCCESS Bisecting: 111 revisions left to test after this (roughly 7 steps) [2dc85bf323515e59e15dfa858d1472bb25cad0fe] packet: packet_getname_spkt: make sure string is always 0-terminated running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:2dc85bf323515e59e15dfa858d1472bb25cad0fe:bisect-stable 2013-06-22-20:40:38 2dc85bf323515e59e15dfa858d1472bb25cad0fe compiling 153 real 1514 user 134 sys 1071.80% cpu i386-randconfig-r00-0621 2013-06-22-20:43:22 detecting boot state 3.10.0-rc5-00176-g2dc85bf.......... 2 20 29 32 33 38 44 48 51. 52 SUCCESS Bisecting: 53 revisions left to test after this (roughly 6 steps) [3ad2e318a24124c53cc6390b5bfbd7613d9c2145] Merge tag 'usb-3.10-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:3ad2e318a24124c53cc6390b5bfbd7613d9c2145:bisect-stable 2013-06-22-20:53:53 3ad2e318a24124c53cc6390b5bfbd7613d9c2145 compiling 158 real 1517 user 148 sys 1051.36% cpu i386-randconfig-r00-0621 2013-06-22-20:56:41 detecting boot state 3.10.0-rc5-00225-g3ad2e31........ 1. 9 22 32 35 48 50. 51 52 SUCCESS Bisecting: 33 revisions left to test after this (roughly 5 steps) [5938930e71affa390c3fa33fa2fda52f58f850e8] Merge branch 'merge' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:5938930e71affa390c3fa33fa2fda52f58f850e8:bisect-stable 2013-06-22-21:06:12 5938930e71affa390c3fa33fa2fda52f58f850e8 compiling 152 real 1518 user 132 sys 1082.93% cpu i386-randconfig-r00-0621 2013-06-22-21:08:56 detecting boot state 3.10.0-rc5-00245-g5938930......... 1 7 11 16 26 38 46 52 SUCCESS Bisecting: 19 revisions left to test after this (roughly 4 steps) [323226bbb3d865af12644d66df2b7f161adf51c7] Merge tag 'fixes-3.10-4' of git://git.infradead.org/users/jcooper/linux into fixes running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:323226bbb3d865af12644d66df2b7f161adf51c7:bisect-stable 2013-06-22-21:17:28 323226bbb3d865af12644d66df2b7f161adf51c7 compiling 155 real 1528 user 141 sys 1077.72% cpu i386-randconfig-r00-0621 2013-06-22-21:20:17 detecting boot state 3.10.0-rc4-00032-g323226b.......... 4 21 30 32 42 51.... 52 SUCCESS Bisecting: 9 revisions left to test after this (roughly 3 steps) [dd019897358b815f7828dab90b51d51df4d3658d] net: sh_eth: fix incorrect RX length error if R8A7740 running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:dd019897358b815f7828dab90b51d51df4d3658d:bisect-stable 2013-06-22-21:30:48 dd019897358b815f7828dab90b51d51df4d3658d compiling 158 real 1508 user 132 sys 1035.48% cpu i386-randconfig-r00-0621 2013-06-22-21:33:42 detecting boot state 3.10.0-rc5-00186-gdd01989........ 2 10 26 44 52 SUCCESS Bisecting: 4 revisions left to test after this (roughly 2 steps) [e6694d984adbe8146d2f1e08d500befc1481835e] Merge tag 'fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:e6694d984adbe8146d2f1e08d500befc1481835e:bisect-stable 2013-06-22-21:40:14 e6694d984adbe8146d2f1e08d500befc1481835e compiling 158 real 1507 user 138 sys 1040.75% cpu i386-randconfig-r00-0621 2013-06-22-21:43:00 detecting boot state 3.10.0-rc5-00283-ge6694d9........ 2 6 16 22 33 48 49.. 51................... 52 SUCCESS Bisecting: 2 revisions left to test after this (roughly 1 step) [f5abaa1bfc3dbf26d19d3513f39279ca369f8d65] tracing: Add DEFINE_EVENT_FN() macro running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:f5abaa1bfc3dbf26d19d3513f39279ca369f8d65:bisect-stable 2013-06-22-22:02:04 f5abaa1bfc3dbf26d19d3513f39279ca369f8d65 compiling 140 real 1103 user 98 sys 856.03% cpu i386-randconfig-r00-0621 2013-06-22-22:06:53 detecting boot state 3.10.0-rc6-00001-gf5abaa1....... 1 27 51... 52 SUCCESS Bisecting: 0 revisions left to test after this (roughly 1 step) [629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb] x86: Rename variables for debugging running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/stable/obj-bisect ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb:bisect-stable 2013-06-22-22:13:56 629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb compiling 174 real 1502 user 148 sys 944.15% cpu i386-randconfig-r00-0621 2013-06-22-22:17:02 detecting boot state 3.10.0-rc6-00003-g629f4f9....... 1 12. 25 45. 52 SUCCESS cf910e83ae23692fdeefc7e506e504c4c468d38a is the first bad commit commit cf910e83ae23692fdeefc7e506e504c4c468d38a Author: Seiji Aguchi Date: Thu Jun 20 11:46:53 2013 -0400 x86, trace: Add irq vector tracepoints [Purpose of this patch] As Vaibhav explained in the thread below, tracepoints for irq vectors are useful. http://www.spinics.net/lists/mm-commits/msg85707.html The current interrupt traces from irq_handler_entry and irq_handler_exit provide when an interrupt is handled. They provide good data about when the system has switched to kernel space and how it affects the currently running processes. There are some IRQ vectors which trigger the system into kernel space, which are not handled in generic IRQ handlers. Tracing such events gives us the information about IRQ interaction with other system events. The trace also tells where the system is spending its time. We want to know which cores are handling interrupts and how they are affecting other processes in the system. Also, the trace provides information about when the cores are idle and which interrupts are changing that state. On the other hand, my usecase is tracing just local timer event and getting a value of instruction pointer. I suggested to add an argument local timer event to get instruction pointer before. But there is another way to get it with external module like systemtap. So, I don't need to add any argument to irq vector tracepoints now. [Patch Description] Vaibhav's patch shared a trace point ,irq_vector_entry/irq_vector_exit, in all events. But there is an above use case to trace specific irq_vector rather than tracing all events. In this case, we are concerned about overhead due to unwanted events. So, add following tracepoints instead of introducing irq_vector_entry/exit. so that we can enable them independently. - local_timer_vector - reschedule_vector - call_function_vector - call_function_single_vector - irq_work_entry_vector - error_apic_vector - thermal_apic_vector - threshold_apic_vector - spurious_apic_vector - x86_platform_ipi_vector Also, introduce a logic switching IDT at enabling/disabling time so that a time penalty makes a zero when tracepoints are disabled. Detailed explanations are as follows. - Create trace irq handlers with entering_irq()/exiting_irq(). - Create a new IDT, trace_idt_table, at boot time by adding a logic to _set_gate(). It is just a copy of original idt table. - Register the new handlers for tracpoints to the new IDT by introducing macros to alloc_intr_gate() called at registering time of irq_vector handlers. - Add checking, whether irq vector tracing is on/off, into load_current_idt(). This has to be done below debug checking for these reasons. - Switching to debug IDT may be kicked while tracing is enabled. - On the other hands, switching to trace IDT is kicked only when debugging is disabled. In addition, the new IDT is created only when CONFIG_TRACING is enabled to avoid being used for other purposes. Signed-off-by: Seiji Aguchi Link: http://lkml.kernel.org/r/51C323ED.5050708@hds.com Signed-off-by: H. Peter Anvin Cc: Steven Rostedt :040000 040000 23d5ef146158b4101016ed2d0eb11e4d28712044 ec33c7b639b136e1f161d41e6cbee46c41bd58d7 M arch :040000 040000 d192324e35d4142edb426f2489b86f75eb19e671 0d13b3daa75c1ee862e877409b72d8a45b297a9a M include bisect run success ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb:bisect-stable 2013-06-22-22:24:05 629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb reuse /kernel/i386-randconfig-r00-0621/629f4f9d59a27d8e58aa612e886e6a9a63ea7aeb/vmlinuz-3.10.0-rc6-00003-g629f4f9 2013-06-22-22:24:05 detecting boot state ....... 53 66 96 142 155. 156 SUCCESS ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:83ab85140bc1492f92de263a1c30ea04a0f465f7:bisect-stable TEST FAILURE /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-kbuild-11952-20130622050928-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-kbuild-57315-20130622050909-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-snb-27885-20130622051843-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-12896-20130622051534-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-22414-20130622051556-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-22982-20130622051534-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-5231-20130622051544-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-5492-20130622051556-3.10.0-rc6-00005-g83ab851-188 /kernel/i386-randconfig-r00-0621/83ab85140bc1492f92de263a1c30ea04a0f465f7/dmesg-kvm-vp-6344-20130622051612-3.10.0-rc6-00005-g83ab851-188 HEAD is now at 83ab851 trace,x86: Move creation of irq tracepoints from apic.c to irq.c ========= upstream ========= Fetching linus ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:f71194a7d47c1da787555d27aac63973ca72323b:bisect-stable 2013-06-22-22:31:17 f71194a7d47c1da787555d27aac63973ca72323b reuse /kernel/i386-randconfig-r00-0621/f71194a7d47c1da787555d27aac63973ca72323b/vmlinuz-3.10.0-rc6-00132-gf71194a 2013-06-22-22:31:17 detecting boot state ...... 10 11 70 148 156 SUCCESS ========= linux-next ========= Fetching next ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:e1a86578747376f08985627c84df088a5d0d1e92:bisect-stable 2013-06-22-23:06:55 e1a86578747376f08985627c84df088a5d0d1e92 compiling 2013-06-22-23:10:40 e1a86578747376f08985627c84df088a5d0d1e92 SKIP BROKEN BUILD Check erros in /cc/wfg/stable-bisect and /tmp/kernel/i386-randconfig-r00-0621/e1a86578747376f08985627c84df088a5d0d1e92 ========= tip/master ========= Fetching tip ls -a /kernel-tests/run-queue/kvm/i386-randconfig-r00-0621/tip:x86:trace:a92a990789849502897e3e19c2c0e5154888edf7:bisect-stable 2013-06-22-23:10:44 a92a990789849502897e3e19c2c0e5154888edf7 reuse /kernel/i386-randconfig-r00-0621/a92a990789849502897e3e19c2c0e5154888edf7/vmlinuz-3.10.0-rc6-01764-ga92a990 2013-06-22-23:10:44 detecting boot state ....... 17 50 66 90 138 144 147.. 148........ 150.............................. 154................................ 156 SUCCESS