mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mike Sullivan <mksully@us.ibm.com>
To: linux-kernel@vger.kernel.org
Subject: 
Date: Mon, 8 Sep 2003 13:58:36 -0500	[thread overview]
Message-ID: <OF0770DEDD.BEBEF8A2-ON85256D9B.00659B11@us.ibm.com> (raw)

In 2.6.0-test3 I am seeing a significant slowdown (~40%) during
heavy database workloads when using the default anticipatory
I/O scheduler. When the same kernel is booted with the
elevator=deadline argument the slowdown does not occur.
I've attached excerpts from the database test  for vmstat and
iostat output as well as readprofiles for each of the schedulers.
Is anyone else seeing similiar behaviors?

Hardware: IBM x440 8way with hypertheading enabled, 16GB RAM,
4 QLA2310 FC adapters attached to 2 FastT900 controllers
(112 disks total).

1a. vmstat for anticipatory scheduler
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
14 41      0 3489576   3840 168280    0    0    25     2   70    30  1  2 96  0
 4 103      0 3329760   3860 172160    0    0 233549    97 6606 21284 27  6  0 67
 7 91      0 3327496   3864 172156    0    0 137106    14 7364 32957 20  5  0 75
 2 84      0 3325416   3868 172412    0    0 138979    73 7096 30445 19  5  0 76
 6 78      0 3324528   3868 173192    0    0 143213   444 7328 24901 13  4  0 83
 0 105      0 3316408   3876 173964    0    0 142413   524 7531 26854 12  5  0 83
 7 94      0 3313664   3880 174220    0    0 134939   821 7336 29037 14  5  0 82
 1 110      0 3313560   3884 174216    0    0 114651   923 6990 27063 11  4  0 84
 5 109      0 3313008   3884 174216    0    0 110115   862 6937 26381 11  4  0 85
 3 111      0 3325400   3920 174180    0    0 108664  1030 6892 26721 11  4  0 85


1b. vmstat for deadline scheduler
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
19  0      0 3518364   3104 119096    0    0    43     1   78    62  1  2 97  1
70 23      0 3361380   3776 170684    0    0 351035   538 3521 19435 68 17  9  5
96 25      0 3321060   3816 171684    0    0 588377   349 4973 25000 91  9  0  0
79 41      0 3316492   3824 171676    0    0 621683   831 5037 26853 91  9  0  0
86 48      0 3318684   3836 172964    0    0 609092  3073 4882 27878 91  9  0  0
71 44      0 3310164   3848 173212    0    0 551741  3746 4291 28492 92  8  0  0
91 35      0 3297708   3852 174248    0    0 508096  1425 4410 29533 91  8  0  1


2a. iostat for anticipatory scheduler

Linux 2.6.0-test3lp
Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     1.92   0.00 363.48  0.88 18908.11   29.12  9454.06    14.56    51.98     4.80   13.14   1.22  44.57
/dev/sdi     1.97   0.00 364.65  0.61 18541.72   20.66  9270.86    10.33    50.82     4.86   13.27   1.23  44.75
/dev/sdh     1.95   0.00 361.35  0.85 18901.05   28.30  9450.53    14.15    52.26     4.79   13.20   1.23  44.72
/dev/sdg     1.66   0.00 357.14  0.84 18155.59   28.06  9077.80    14.03    50.79     4.74   13.23   1.25  44.61
/dev/sdf     3.20   0.00 328.57  0.87 17293.65   28.98  8646.82    14.49    52.58     4.73   14.34   1.36  44.74
/dev/sde     3.44   0.00 330.51  0.75 17180.30   18.55  8590.15     9.28    51.92     4.76   14.35   1.36  45.08
/dev/sdd     3.36   0.00 323.76  0.42 16911.58   14.56  8455.79     7.28    52.21     4.80   14.78   1.38  44.75
/dev/sdc     3.04   0.00 325.47  0.93 16665.03   31.01  8332.51    15.50    51.15     4.94   15.13   1.37  44.74

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     4.80   0.00 863.10  2.00 39945.60   64.00 19972.80    32.00    46.25    11.25   13.08   1.16 100.02
/dev/sdi     4.50   0.00 787.50 14.90 34438.40  483.20 17219.20   241.60    43.52    14.13   17.58   1.25 100.02
/dev/sdh     3.40   0.00 851.00  2.10 33180.80   70.40 16590.40    35.20    38.98    11.65   13.60   1.17 100.02
/dev/sdg     5.10   0.00 840.60  9.10 37772.80  288.00 18886.40   144.00    44.79    13.01   15.27   1.18 100.02
/dev/sdf     8.90   0.00 740.70  3.90 33075.20  128.00 16537.60    64.00    44.59    11.66   15.67   1.34 100.02
/dev/sde     8.30   0.00 760.40  2.80 36169.60   96.00 18084.80    48.00    47.52    11.70   15.23   1.31 100.02
/dev/sdd    12.40   0.00 769.70  0.00 30147.20    0.00 15073.60     0.00    39.17    11.05   14.29   1.30 100.02
/dev/sdc     9.20   0.00 746.00  3.00 31568.00   92.80 15784.00    46.40    42.27    11.65   15.59   1.34 100.03

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     5.01   0.00 895.80  0.00 38073.27    0.00 19036.64     0.00    42.50    11.06   12.29   1.12 100.12
/dev/sdi     5.91   0.00 818.02 12.61 32515.72  406.81 16257.86   203.40    39.64    13.84   16.71   1.21 100.12
/dev/sdh     7.41   0.00 862.16  1.80 34652.25   54.45 17326.13    27.23    40.17    11.46   13.31   1.16 100.12
/dev/sdg     4.40   0.00 866.67  2.20 33271.67   70.47 16635.84    35.24    38.37    11.67   13.43   1.15 100.13
/dev/sdf     8.01   0.00 784.28  2.80 33242.84   86.49 16621.42    43.24    42.35    11.52   14.59   1.27 100.13
/dev/sde     6.11   0.20 728.53 17.62 30488.09  576.58 15244.04   288.29    41.63    15.00   20.15   1.34 100.13
/dev/sdd     6.01   0.40 705.51 12.11 29690.49  413.21 14845.25   206.61    41.95    13.73   19.17   1.40 100.13
/dev/sdc     7.41   0.80 738.04  7.61 33338.94  281.88 16669.47   140.94    45.09    12.13   16.17   1.34 100.12

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     4.29   0.00 864.37  0.00 33411.58    0.00 16705.79     0.00    38.65    11.17   12.97   1.15  99.83
/dev/sdi     6.29   0.00 826.35  6.49 35002.00  198.00 17501.00    99.00    42.27    12.34   14.75   1.20  99.83
/dev/sdh     6.69   0.00 844.91  0.90 36563.67   28.74 18281.84    14.37    43.26    10.92   13.03   1.18  99.83
/dev/sdg     3.39   0.00 860.78  0.70 33705.39   22.36 16852.69    11.18    39.15    11.27   13.09   1.16  99.82
/dev/sdf     8.28   0.00 749.40  1.20 32661.08   38.32 16330.54    19.16    43.56    11.24   14.97   1.33  99.82
/dev/sde     5.79   0.00 731.44 13.47 34328.14  427.94 17164.07   213.97    46.66    13.86   18.67   1.34  99.82
/dev/sdd     4.59   0.00 713.77 17.37 31853.09  552.50 15926.55   276.25    44.32    14.61   20.09   1.37  99.82
/dev/sdc    11.88   0.00 699.00 16.07 30914.17  514.17 15457.09   257.09    43.95    14.46   20.26   1.40  99.82


2b. io for deadline
Linux 2.6.0-test3lp

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     0.04   0.00 103.39  1.02 30230.24   36.05 15115.12    18.02   289.87     0.68    6.51   1.60  16.72
/dev/sdi     0.01   0.00 64.93  0.82 18642.75   29.55  9321.38    14.77   283.99     0.50    7.57   1.90  12.49
/dev/sdh     0.02   0.00 109.26  1.02 30215.96   36.05 15107.98    18.02   274.30     0.70    6.37   1.55  17.14
/dev/sdg     0.03   0.00 62.97  0.21 18346.02   10.04  9173.01     5.02   290.50     0.48    7.53   1.92  12.11
/dev/sdf     0.02   0.00 98.78  0.92 27471.17   32.80 13735.58    16.40   275.87     0.75    7.47   1.64  16.37
/dev/sde     1.64   0.00 105.42  1.75 27301.07   41.84 13650.54    20.92   255.15     0.77    7.21   1.63  17.46
/dev/sdd     0.02   0.00 103.89  1.02 28228.90   36.05 14114.45    18.02   269.40     0.77    7.36   1.58  16.54
/dev/sdc     0.05   0.00 108.91  1.63 31749.13   55.56 15874.56    27.78   287.71     0.77    6.94   1.67  18.49

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     0.10   0.00 581.30  0.00 152032.00    0.00 76016.00     0.00   261.54     4.16    7.16   1.62  94.39
/dev/sdi     0.20   0.00 497.70  0.00 152601.60    0.00 76300.80     0.00   306.61     3.75    7.52   1.92  95.35
/dev/sdh     0.20   0.00 552.40  0.00 150166.40    0.00 75083.20     0.00   271.84     3.97    7.18   1.71  94.71
/dev/sdg     0.20   0.00 545.70  0.00 148086.40    0.00 74043.20     0.00   271.37     3.93    7.19   1.74  95.15
/dev/sdf     0.20   0.00 661.90  0.00 160950.40    0.00 80475.20     0.00   243.16     4.97    7.50   1.47  97.56
/dev/sde     0.80   0.00 725.50  0.00 148051.20    0.00 74025.60     0.00   204.07     5.39    7.42   1.31  95.00
/dev/sdd     0.90   0.00 688.60  0.00 158950.40    0.00 79475.20     0.00   230.83     5.12    7.43   1.40  96.40
/dev/sdc     0.70   0.00 644.30 13.80 147622.40  441.60 73811.20   220.80   224.99     4.99    7.58   1.41  92.52

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     0.00   0.00 448.10  0.00 132969.60    0.00 66484.80     0.00   296.74     3.03    6.77   1.88  84.40
/dev/sdi     0.00   0.00 514.20  0.00 141683.20    0.00 70841.60     0.00   275.54     3.53    6.88   1.80  92.70
/dev/sdh     0.20   0.00 537.90  0.00 145718.40    0.00 72859.20     0.00   270.90     3.93    7.31   1.71  91.92
/dev/sdg     0.60   0.00 764.00  0.00 149760.00    0.00 74880.00     0.00   196.02     5.25    6.87   1.30  99.41
/dev/sdf     0.70   0.00 803.10  0.00 138851.20    0.00 69425.60     0.00   172.89     5.94    7.40   1.20  96.11
/dev/sde     1.00   0.00 993.30  0.00 164857.60    0.00 82428.80     0.00   165.97     6.97    7.02   1.01  99.96
/dev/sdd     1.80   0.00 899.70  0.00 166409.60    0.00 83204.80     0.00   184.96     5.93    6.59   1.11  99.49
/dev/sdc     0.20   0.00 445.30  0.00 131558.40    0.00 65779.20     0.00   295.44     3.37    7.57   1.93  85.86

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
/dev/sdj     0.10   0.00 402.60  0.00 154393.60    0.00 77196.80     0.00   383.49     2.77    6.89   2.01  80.84
/dev/sdi     0.00   0.00 506.60  9.20 132547.20  294.40 66273.60   147.20   257.54     3.47    6.73   1.49  76.64
/dev/sdh     0.20   0.00 323.90  0.00 103331.20    0.00 51665.60     0.00   319.02     2.28    7.04   2.04  66.11
/dev/sdg     0.10   0.00 553.00  0.00 174540.80    0.00 87270.40     0.00   315.63     3.56    6.44   1.71  94.53
/dev/sdf     0.00   0.00 447.70  0.00 154179.20    0.00 77089.60     0.00   344.38     3.33    7.44   1.99  88.89
/dev/sde     0.20   0.00 426.20  4.60 155136.00  147.20 77568.00    73.60   360.45     3.21    7.47   2.06  88.80
/dev/sdd     0.10   0.00 448.90  0.00 159312.00    0.00 79656.00     0.00   354.89     3.28    7.30   1.98  88.70
/dev/sdc     0.00   0.00 410.50  0.00 146473.60    0.00 73236.80     0.00   356.82     3.04    7.41   2.16  88.65


3a. readprofile for anticipatory scheduler
1699600 total                                      0.8206
1609934 poll_idle                                27757.4828
 12233 schedule                                  10.1772
 11380 do_softirq                                54.9758
  9795 __make_request                             8.0485
  5708 scsi_request_fn                            8.3450
  5282 scsi_end_request                          28.2460
  3482 qla2x00_start_scsi                         3.4960
  3466 dio_bio_end_io                            28.8833
  2279 dio_await_one                             14.6090
  2065 cpu_idle                                  32.2656
  1706 qla2x00_queuecommand                       1.1685
  1647 add_timer                                 11.6809
  1572 put_io_context                            18.2791
  1503 unlock_page                               17.0795
  1433 follow_hugetlb_page                        3.0042
  1292 sys_msgsnd                                 1.7969
  1260 current_kernel_time                       18.2609
  1081 scsi_dispatch_cmd                          2.8078
   918 qla2x00_get_new_sp                         8.2703
   758 try_to_wake_up                             1.3512
   702 system_call                               15.9545
   691 del_timer_sync                             5.4841
   687 scsi_io_completion                         0.6314
   628 do_no_page                                 0.5004
   589 blk_recount_segments                       0.8951
   577 blk_run_queues                             3.6752
   560 __end_that_request_first                   1.0219
   533 get_io_context                             4.1318
   506 scsi_device_unbusy                         4.0480
   501 set_page_dirty_lock                        5.5055
   493 get_request                                0.7515
   451 direct_io_worker                           0.4275
   450 do_direct_IO                               0.4306
   441 dio_bio_complete                           1.6036
   409 worker_thread                              0.6059
   406 submit_page_section                        0.9442
   386 kmem_cache_alloc                           5.0130
   358 bio_alloc                                  0.8689
   344 add_disk_randomness                        6.6154
   339 kmem_cache_free                            3.8966
   319 dio_bio_add_page                           3.0673
   317 __generic_file_aio_read                    0.5671
   316 ipc_lock                                   4.3288
   296 kfree                                      2.2256
   281 add_timer_randomness                       1.2165
   268 as_work_handler                            3.3924
   236 fsync_buffers_list                         0.6396
   230 get_offset_cyclone                         4.1818
   223 huge_pte_offset                            5.4390
   213 qla2x00_next                               0.2359
   211 io_schedule_timeout                        4.1373
   208 scsi_put_command                           1.4545
   197 mempool_alloc                              0.5988
   186 blockdev_direct_IO                         0.5360
   180 finished_one_bio                           1.5517
   173 testmsg                                    2.4714
   173 batch_entropy_store                        1.1234
   171 sys_msgrcv                                 0.1913
   168 blk_run_queue                              2.3014
   166 mempool_free_slab                          7.9048
   159 dio_refill_pages                           0.5179
   157 generic_make_request                       0.3601
   149 bio_add_page                               0.4357
   140 dnotify_parent                             1.3592
   136 fget_light                                 1.0543
   135 vfs_read                                   0.4576
   122 do_page_fault                              0.1083
   118 update_atime                               0.5315
   116 get_jiffies_64                             2.0000
   113 sd_rw_intr                                 0.2770
   111 scsi_run_queue                             0.5663
   105 .text.lock.util                            4.3750
    97 blkdev_get_blocks                          0.8220
    92 __copy_from_user_ll                        0.7541
    90 generic_file_direct_IO                     0.5172
    88 add_wait_queue                             1.2571
    85 submit_bio                                 0.5556
    84 sys_semtimedop                             0.0635
    84 __copy_to_user_ll                          0.7000
    82 remove_wait_queue                          1.1081
    82 dio_bio_submit                             0.8119
    81 io_schedule                                1.6875
    78 do_gettimeofday                            0.5417
    78 dio_send_cur_page                          0.4561
    77 find_vma                                   0.9506
    75 page_add_rmap                              0.1974
    75 memcpy                                     1.3158
    75 bio_put                                    1.3889
    74 page_waitqueue                             1.5745
    72 scsi_init_cmd_errh                         0.3564
    72 generic_file_read                          0.4186
    71 max_block                                  0.6174
    70 blkdev_direct_IO                           0.6140
    66 sys_ipc                                    0.0965
    65 proc_pid_stat                              0.0543
    64 kmap_atomic                                0.3951
    63 scsi_softirq                               0.2864
    63 del_timer                                  0.6000
    62 default_idle                               1.3778
    60 mempool_free                               0.4878
    60 bio_destructor                             0.6383
    56 get_user_pages                             0.0556
    56 find_extend_vma                            0.4444
    55 task_mem                                   0.2148
    55 __d_lookup                                 0.2000
    52 number                                     0.0692
    51 syscall_exit                               4.6364
    51 dio_await_completion                       0.6220
    50 bio_get_nr_vecs                            0.9434
    49 as_set_request                             0.4375
    48 sys_pread64                                0.3780
    48 dio_get_page                               0.5517
    45 find_get_page                              0.5769
    41 do_anonymous_page                          0.0429
    40 do_getitimer                               0.1455
    38 dio_new_bio                                0.3016
    38 __set_page_dirty_buffers                   0.1279
    36 mempool_alloc_slab                         1.7143
    35 sys_setitimer                              0.1268
    35 do_clock_nanosleep                         0.0448
    35 __kmalloc                                  0.2869
    34 __copy_user_intel                          0.1977
    33 adjust_abs_time                            0.1158
    31 page_remove_rmap                           0.0571
    31 blkdev_writepage                           0.8857
    30 inode_times_differ                         0.4412
    30 get_more_blocks                            0.1911
    29 s_show                                     0.0458
    28 scsi_decide_disposition                    0.0875
    28 link_path_walk                             0.0133
    27 sys_nanosleep                              0.1034
    26 show_regs                                  0.0741
    25 wait_on_page_bit                           0.1196
    25 do_setitimer                               0.0529
    25 bio_endio                                  0.2049
    23 vsnprintf                                  0.0201
    23 queue_delayed_work                         0.1655
    23 ipcperms                                   0.1411
    22 do_readv_writev                            0.0316
    21 zap_pte_range                              0.0297
    21 scsi_finish_command                        0.1167
    20 scsi_add_timer                             0.1980
    20 generic_file_aio_write_nolock              0.0068
    19 update_queue                               0.1652
    19 dio_zero_block                             0.1131
    19 blk_queue_bounce                           0.2468
    18 syscall_call                               1.6364
    18 store_msg                                  0.0783
    17 try_atomic_semop                           0.0570
    17 do_wp_page                                 0.0130
    17 buffered_rmqueue                           0.0489
    16 sys_gettimeofday                           0.0812
    16 handle_mm_fault                            0.0329
    15 scsi_free_sgtable                          0.3333
    15 qla2x00_calc_iocbs_64                      0.2542
    15 free_hot_cold_page                         0.0600
    15 flush_tlb_mm                               0.1136
    15 elv_set_request                            0.2586
    14 kmap_atomic_to_page                        0.0921
    14 .text.lock.direct_io                       0.1386
    13 wake_up_process                            0.3095
    13 dio_bio_alloc                              0.1429
    13 default_llseek                             0.0491
    12 scsi_next_command                          0.2927
    12 pid_revalidate                             0.1121
    11 show_cpuinfo                               0.0162
    11 render_sigset_t                            0.0598
    11 dio_bio_reap                               0.0965
    11 __lookup                                   0.0539
    10 proc_pid_status                            0.0090
    10 path_lookup                                0.0299
    10 get_pid_list                               0.0855
     9 free_msg                                   0.1957
     9 clear_page_tables                          0.0201
     9 atomic_dec_and_lock                        0.1385
     9 __pagevec_lru_add_active                   0.0354
     8 local_bh_enable                            0.0597
     8 load_msg                                   0.0193
     8 __wake_up                                  0.1231
     7 vfs_write                                  0.0237
     7 release_pages                              0.0216
     7 pte_alloc_one                              0.0547
     7 pte_alloc_map                              0.0215
     7 kernel_to_ipc64_perm                       0.1296
     7 ipc_unlock                                 0.8750
     7 fput                                       0.3333
     7 copy_process                               0.0027
     6 vfs_readv                                  0.0600
     6 tstojiffie                                 0.0594
     6 sock_alloc_send_pskb                       0.0117
     6 generic_file_readv                         0.0385
     6 end_that_request_chunk                     1.2000
     6 dio_complete                               0.0800
     5 write_profile                              0.0276
     5 sock_wfree                                 0.0658
     5 radix_tree_lookup                          0.0676
     5 pgd_ctor                                   0.1316
     5 page_address                               0.0289
     5 lru_cache_add_active                       0.0602
     5 get_empty_filp                             0.0209
     5 generic_file_write_nolock                  0.0321
     5 filemap_nopage                             0.0054
     5 bad_range                                  0.0442
     4 zap_pmd_range                              0.0342
     4 vma_link                                   0.0244
     4 unix_stream_recvmsg                        0.0029
     4 unix_stream_data_wait                      0.0134
     4 unix_release_sock                          0.0060
     4 unix_create1                               0.0113
     4 tg3_poll                                   0.0137
     4 sync_mapping_buffers                       0.0889
     4 split_vma                                  0.0137
     4 skb_recv_datagram                          0.0169
     4 qla2x00_blink_led                          0.0214
     4 pte_chain_alloc                            0.0625
     4 get_wchan                                  0.0328
     4 generic_osync_inode                        0.0156
     4 follow_page                                0.0095
     4 follow_mount                               0.0286
     4 flush_scheduled_work                       0.4000
     4 elv_put_request                            0.1212
     4 copy_page_range                            0.0030
     4 __pte_chain_free                           0.0455
     4 __get_user_4                               0.1739
     4 __constant_c_and_count_memset              0.0312
     4 .text.lock.time                            0.1212
     3 unix_stream_sendmsg                        0.0030
     3 tcp_v4_rcv                                 0.0013
     3 tcp_sendmsg                                0.0006
     3 sys_llseek                                 0.0118
     3 sys_fstat64                                0.0526
     3 sys_close                                  0.0265
     3 strncpy_from_user                          0.0303
     3 sock_fasync                                0.0052
     3 scsi_init_io                               0.0095
     3 poll_freewait                              0.0469
     3 may_open                                   0.0068
     3 kmap_high                                  0.0065
     3 ipc_buildid                                0.2500
     3 generic_fillattr                           0.0176
     3 fget                                       0.0476
     3 exit_notify                                0.0016
     3 eth_type_trans                             0.0179
     3 dput                                       0.0074
     3 do_poll                                    0.0152
     3 do_mmap_pgoff                              0.0017
     3 dio_cleanup                                0.0195
     3 bdev_read_only                             0.0517
     3 alloc_ldt                                  0.0071
     3 access_process_vm                          0.0065
     3 __mark_inode_dirty                         0.0123
     3 __filemap_copy_from_user_iovec             0.0171
     3 __alloc_pages                              0.0038
     2 wake_up_process_kick                       0.0476
     2 vsprintf                                   0.0465
     2 vfs_writev                                 0.0200
     2 unmap_region                               0.0058
     2 unix_write_space                           0.0142
     2 unix_poll                                  0.0129
     2 unix_find_socket_byinode                   0.0233
     2 sys_stat64                                 0.0351
     2 sys_readv                                  0.0202
     2 sys_open                                   0.0144
     2 sys_clock_nanosleep                        0.0067
     2 ss_wakeup                                  0.0426
     2 sock_poll                                  0.0408
     2 sock_map_fd                                0.0061
     2 set_page_address                           0.0060
     2 remove_suid                                0.0182
     2 remove_shared_vm_struct                    0.0177
     2 release_task                               0.0049
     2 raw_file_aio_write                         0.0345
     2 prepare_to_wait                            0.0222
     2 pid_alive                                  0.0571
     2 normal_poll                                0.0056
     2 mmput                                      0.0120
     2 iput                                       0.0161
     2 ip_rcv                                     0.0017
     2 get_write_access                           0.0351
     2 fn_hash_lookup                             0.0085
     2 eventpoll_init_file                        0.0741
     2 end_that_request_first                     0.2000
     2 do_sync_read                               0.0109
     2 do_aic7xxx_isr                             0.0129
     2 d_alloc                                    0.0044
     2 cp_new_stat64                              0.0075
     2 copy_files                                 0.0030
     2 change_protection                          0.0035
     2 cache_grow                                 0.0025
     2 __block_prepare_write                      0.0019
     2 __blk_queue_bounce                         0.0034
     1 work_resched                               0.0455
     1 wait_for_packet                            0.0031
     1 vm_acct_memory                             0.0185
     1 vfs_permission                             0.0031
     1 vfs_follow_link                            0.0022
     1 unix_stream_connect                        0.0010
     1 unix_peer_get                              0.0238
     1 unix_accept                                0.0033
     1 tg3_rx                                     0.0011
     1 tcp_transmit_skb                           0.0007
     1 tcp_poll                                   0.0026
     1 sys_timer_settime                          0.0012
     1 sys_select                                 0.0007
     1 sys_pwrite64                               0.0079
     1 sys_munmap                                 0.0098
     1 sys_mprotect                               0.0017
     1 sys_getuid                                 0.0625
     1 sys_getdents64                             0.0048
     1 strnlen_user                               0.0147
     1 sock_wmalloc                               0.0106
     1 sock_destroy_inode                         0.0323
     1 sock_def_readable                          0.0079
     1 sock_create                                0.0027
     1 skip_atoi                                  0.0154
     1 skb_clone                                  0.0030
     1 show_interrupts                            0.0014
     1 shm_get_stat                               0.0064
     1 set_huge_pte                               0.0057
     1 scsi_setup_cmd_retry                       0.0093
     1 schedule_timeout                           0.0053
     1 schedule_delayed_work                      0.0714
     1 restore_fpu                                0.0323
     1 recalc_bh_state                            0.0076
     1 rb_insert_color                            0.0044
     1 qla2x00_build_scsi_iocbs_32                0.0018
     1 pty_write                                  0.0024
     1 pty_chars_in_buffer                        0.0137
     1 profile_exit_task                          0.0132
     1 proc_lookup                                0.0037
     1 proc_calc_metrics                          0.0123
     1 prepare_to_wait_exclusive                  0.0112
     1 prep_new_page                              0.0122
     1 pid_fd_revalidate                          0.0051
     1 path_release                               0.0159
     1 parse_table                                0.0034
     1 page_cache_readahead                       0.0026
     1 old_mmap                                   0.0032
     1 nr_running                                 0.0244
     1 notify_change                              0.0026
     1 next_thread                                0.0127
     1 net_rx_action                              0.0039
     1 monotonic_clock_cyclone                    0.0061
     1 mm_init                                    0.0040
     1 math_state_restore                         0.0152
     1 lru_add_drain                              0.0119
     1 locks_remove_posix                         0.0036
     1 load_elf_binary                            0.0003
     1 kobj_lookup                                0.0034
     1 kill_fasync                                0.0169
     1 kill_bdev                                  0.0147
     1 ipc_checkid                                0.0303
     1 invalidate_inode_buffers                   0.0111
     1 inode_init_once                            0.0027
     1 inet_sendmsg                               0.0112
     1 in_group_p                                 0.0222
     1 i_waitq_head                               0.0455
     1 handle_signal                              0.0032
     1 get_unused_fd                              0.0025
     1 get_signal_to_deliver                      0.0012
     1 generic_file_open                          0.0111
     1 flush_tlb_page                             0.0072
     1 find_lock_page                             0.0036
     1 filp_close                                 0.0083
     1 filldir64                                  0.0032
     1 fd_install                                 0.0179
     1 ext2_update_inode                          0.0013
     1 ext2_get_inode                             0.0033
     1 ext2_alloc_block                           0.0050
     1 dup_task_struct                            0.0050
     1 do_sys_settimeofday                        0.0051
     1 do_select                                  0.0014
     1 do_page_cache_readahead                    0.0027
     1 do_flush_tlb_all                           0.0108
     1 disk_round_stats                           0.0047
     1 device_not_available                       0.0244
     1 dev_seq_printf_stats                       0.0044
     1 detach_vmas_to_be_unmapped                 0.0101
     1 destroy_context                            0.0058
     1 dentry_open                                0.0027
     1 dcache_readdir                             0.0021
     1 d_rehash                                   0.0111
     1 d_lookup                                   0.0125
     1 d_instantiate                              0.0111
     1 csum_partial                               0.0035
     1 create_elf_tables                          0.0011
     1 count_semncnt                              0.0091
     1 copy_strings                               0.0016
     1 copy_mm                                    0.0011
     1 check_unthrottle                           0.0164
     1 can_vma_merge_before                       0.0119
     1 cache_init_objs                            0.0085
     1 c_start                                    0.0278
     1 bd_claim                                   0.0172
     1 batch_entropy_process                      0.0047
     1 alloc_skb                                  0.0045
     1 alloc_inode                                0.0030
     1 add_softcursor                             0.0042
     1 __vma_link                                 0.0063
     1 __unix_remove_socket                       0.0084
     1 __tcp_select_window                        0.0040
     1 __posix_lock_file                          0.0007
     1 __kfree_skb                                0.0046
     1 __fput                                     0.0050
     1 __copy_user_zeroing_intel                  0.0058
     1 __block_commit_write                       0.0068
     1 __alloc_percpu                             0.0047
     1 SHATransform                               0.0033
     1 .text.lock.tty_io                          0.0023
     1 .text.lock.read_write                      0.0192
     1 .text.lock.open                            0.0058
     1 .text.lock.dnotify                         0.0233


3b. readprofile for deadline scheduler
171653 total                                      0.0829
 19968 do_softirq                                96.4638
 18055 schedule                                  15.0208
 14925 scsi_request_fn                           21.8202
 11718 qla2x00_start_scsi                        11.7651
  9323 follow_hugetlb_page                       19.5451
  5626 __make_request                             4.6228
  4567 sys_msgsnd                                 6.3519
  4108 poll_idle                                 70.8276
  4070 dio_bio_end_io                            33.9167
  4044 unlock_page                               45.9545
  3882 do_direct_IO                               3.7148
  3690 scsi_end_request                          19.7326
  3577 submit_page_section                        8.3186
  3146 set_page_dirty_lock                       34.5714
  2502 dio_bio_complete                           9.0982
  2313 qla2x00_queuecommand                       1.5842
  2298 dio_bio_add_page                          22.0962
  2230 system_call                               50.6818
  2163 del_timer_sync                            17.1667
  1905 try_to_wake_up                             3.3957
  1782 get_offset_cyclone                        32.4000
  1721 fsync_buffers_list                         4.6640
  1677 get_request                                2.5564
  1658 bio_add_page                               4.8480
  1394 add_timer                                  9.8865
  1181 dio_await_one                              7.5705
  1159 dio_refill_pages                           3.7752
  1121 scsi_dispatch_cmd                          2.9117
  1056 current_kernel_time                       15.3043
   939 bio_alloc                                  2.2791
   884 qla2x00_get_new_sp                         7.9640
   863 direct_io_worker                           0.8180
   858 ipc_lock                                  11.7534
   812 do_no_page                                 0.6470
   802 put_io_context                             9.3256
   798 blkdev_get_blocks                          6.7627
   765 .text.lock.util                           31.8750
   751 max_block                                  6.5304
   717 __generic_file_aio_read                    1.2826
   691 scsi_io_completion                         0.6351
   683 kmem_cache_alloc                           8.8701
   677 scsi_device_unbusy                         5.4160
   658 kfree                                      4.9474
   644 sys_msgrcv                                 0.7204
   624 kmem_cache_free                            7.1724
   610 get_jiffies_64                            10.5172
   582 blk_run_queue                              7.9726
   581 __end_that_request_first                   1.0602
   563 page_add_rmap                              1.4816
   546 find_vma                                   6.7407
   541 dio_send_cur_page                          3.1637
   517 scsi_put_command                           3.6154
   502 generic_unplug_device                      4.7810
   483 blk_run_queues                             3.0764
   471 testmsg                                    6.7286
   471 page_waitqueue                            10.0213
   461 blk_recount_segments                       0.7006
   455 __copy_from_user_ll                        3.7295
   433 default_llseek                             1.6340
   405 get_user_pages                             0.4018
   379 huge_pte_offset                            9.2439
   360 fget_light                                 2.7907
   348 copy_page_range                            0.2632
   337 add_disk_randomness                        6.4808
   301 get_io_context                             2.3333
   301 do_readv_writev                            0.4325
   301 blockdev_direct_IO                         0.8674
   289 do_page_fault                              0.2564
   280 __copy_to_user_ll                          2.3333
   273 sd_rw_intr                                 0.6691
   266 add_timer_randomness                       1.1515
   258 qla2x00_next                               0.2857
   254 mempool_alloc                              0.7720
   239 generic_make_request                       0.5482
   236 update_atime                               1.0631
   235 batch_entropy_store                        1.5260
   231 __kmalloc                                  1.8934
   227 do_gettimeofday                            1.5764
   223 sys_semtimedop                             0.1687
   219 do_wp_page                                 0.1681
   211 get_more_blocks                            1.3439
   208 sys_ipc                                    0.3041
   198 io_schedule                                4.1250
   186 __set_page_dirty_buffers                   0.6263
   183 finished_one_bio                           1.5776
   183 del_timer                                  1.7429
   182 kmap_atomic                                1.1235
   181 find_extend_vma                            1.4365
   180 wait_on_page_bit                           0.8612
   180 generic_file_direct_IO                     1.0345
   180 dio_get_page                               2.0690
   179 dnotify_parent                             1.7379
   178 io_schedule_timeout                        3.4902
   172 generic_file_readv                         1.1026
   158 blkdev_direct_IO                           1.3860
   152 submit_bio                                 0.9935
   146 sys_llseek                                 0.5748
   144 do_anonymous_page                          0.1506
   137 scsi_run_queue                             0.6990
   137 ipcperms                                   0.8405
   134 bio_destructor                             1.4255
   126 do_getitimer                               0.4582
   120 find_get_page                              1.5385
   118 mempool_free                               0.9593
   116 syscall_exit                              10.5455
   113 do_clock_nanosleep                         0.1445
   112 do_setitimer                               0.2368
   108 scsi_softirq                               0.4909
   108 deadline_set_request                       1.1020
   102 store_msg                                  0.4435
   102 scsi_init_cmd_errh                         0.5050
   101 sys_setitimer                              0.3659
    97 proc_pid_stat                              0.0810
    97 dio_bio_submit                             0.9604
    95 page_remove_rmap                           0.1750
    92 bio_put                                    1.7037
    92 __d_lookup                                 0.3345
    88 dio_await_completion                       1.0732
    85 memcpy                                     1.4912
    80 sys_gettimeofday                           0.4061
    79 wake_up_process                            1.8810
    76 blk_queue_bounce                           0.9870
    73 vfs_read                                   0.2475
    72 pte_alloc_one                              0.5625
    69 load_msg                                   0.1663
    68 task_mem                                   0.2656
    68 scsi_decide_disposition                    0.2125
    65 bio_get_nr_vecs                            1.2264
    64 number                                     0.0852
    63 mempool_alloc_slab                         3.0000
    61 zap_pte_range                              0.0863
    60 handle_mm_fault                            0.1235
    60 dio_new_bio                                0.4762
    58 adjust_abs_time                            0.2035
    57 .text.lock.direct_io                       0.5644
    56 sys_readv                                  0.5657
    54 scsi_finish_command                        0.3000
    53 blkdev_writepage                           1.5143
    52 huge_page_release                          1.2381
    50 vsnprintf                                  0.0436
    49 scsi_next_command                          1.1951
    48 try_atomic_semop                           0.1611
    46 syscall_call                               4.1818
    46 scsi_add_timer                             0.4554
    44 vfs_readv                                  0.4400
    44 inode_times_differ                         0.6471
    43 sys_select                                 0.0318
    42 sys_nanosleep                              0.1609
    42 elv_set_request                            0.7241
    41 find_lock_page                             0.1475
    40 generic_file_read                          0.2326
    39 __copy_user_intel                          0.2267
    37 buffered_rmqueue                           0.1063
    36 update_queue                               0.3130
    36 sys_pread64                                0.2835
    36 radix_tree_lookup                          0.4865
    36 link_path_walk                             0.0171
    36 do_select                                  0.0495
    35 work_resched                               1.5909
    35 qla2x00_calc_iocbs_64                      0.5932
    35 device_not_available                       0.8537
    33 s_show                                     0.0521
    32 dio_bio_alloc                              0.3516
    30 pte_chain_alloc                            0.4688
    28 free_msg                                   0.6087
    27 kmap_atomic_to_page                        0.1776
    27 bio_endio                                  0.2213
    26 free_hot_cold_page                         0.1040
    26 __lookup                                   0.1275
    24 release_pages                              0.0741
    24 pte_alloc_map                              0.0738
    24 flush_tlb_mm                               0.1818
    24 __pte_chain_free                           0.2727
    23 mempool_free_slab                          1.0952
    23 generic_file_aio_write_nolock              0.0078
    23 filemap_nopage                             0.0249
    23 dio_zero_block                             0.1369
    22 ipc_unlock                                 2.7500
    22 dio_complete                               0.2933
    21 tstojiffie                                 0.2079
    20 fget                                       0.3175
    20 clear_page_tables                          0.0446
    20 __alloc_pages                              0.0255
    19 set_huge_pte                               0.1092
    18 dio_bio_reap                               0.1579
    18 bad_range                                  0.1593
    18 __wake_up                                  0.2769
    17 shmem_getpage                              0.0062
    17 fput                                       0.8095
    17 find_vma_prev                              0.2099
    16 local_bh_enable                            0.1194
    16 kstat_read_proc                            0.0175
    16 end_that_request_chunk                     3.2000
    16 copy_files                                 0.0237
    15 queue_delayed_work                         0.1079
    15 pid_revalidate                             0.1402
    15 may_open                                   0.0339
    14 tcp_poll                                   0.0365
    14 scsi_free_sgtable                          0.3111
    14 render_sigset_t                            0.0761
    14 cache_grow                                 0.0175
    14 __pagevec_lru_add_active                   0.0551
    13 shmem_nopage                               0.1204
    13 math_state_restore                         0.1970
    13 copy_mm                                    0.0140
    13 .text.lock.time                            0.3939
    12 pid_alive                                  0.3429
    12 hugetlb_prefault                           0.0327
    12 atomic_dec_and_lock                        0.1846
    11 get_unused_fd                              0.0279
    11 get_pid_list                               0.0940
    11 generic_file_write_nolock                  0.0705
    11 cpu_idle                                   0.1719
    11 __get_user_4                               0.4783
    10 show_interrupts                            0.0136
    10 remove_shared_vm_struct                    0.0885
    10 ipc_checkid                                0.3030
    10 end_that_request_first                     1.0000
     9 vfs_write                                  0.0305
     9 sock_poll                                  0.1837
     9 restore_fpu                                0.2903
     9 qla2x00_build_scsi_iocbs_32                0.0161
     9 generic_osync_inode                        0.0350
     9 d_alloc                                    0.0199
     9 __posix_lock_file                          0.0060
     9 __mark_inode_dirty                         0.0370
     8 sys_close                                  0.0708
     8 strncpy_from_user                          0.0808
     8 qla2x00_blink_led                          0.0428
     8 proc_pid_status                            0.0072
     8 kernel_to_ipc64_perm                       0.1481
     8 ipc_parse_version                          0.2759
     8 flush_scheduled_work                       0.8000
     7 unmap_hugepage_range                       0.0281
     7 schedule_delayed_work                      0.5000
     7 poll_freewait                              0.1094
     7 nr_blockdev_pages                          0.0700
     7 free_pages                                 0.0909
     7 do_sys_settimeofday                        0.0354
     7 copy_process                               0.0027
     7 bdev_read_only                             0.1207
     7 __alloc_percpu                             0.0327
     6 write_profile                              0.0331
     6 vfs_writev                                 0.0600
     6 unix_write_space                           0.0426
     6 sys_shmat                                  0.0083
     6 ss_wakeup                                  0.1277
     6 pgd_ctor                                   0.1579
     6 mark_page_accessed                         0.1154
     6 lru_cache_add_active                       0.0723
     6 ipc_buildid                                0.5000
     6 flush_tlb_page                             0.0432
     6 do_sigaction                               0.0095
     6 do_mmap_pgoff                              0.0034
     6 dio_cleanup                                0.0390
     6 cp_new_stat64                              0.0226
     5 unmap_vmas                                 0.0069
     5 unix_stream_recvmsg                        0.0036
     5 sys_open                                   0.0360
     5 schedule_timeout                           0.0267
     5 path_lookup                                0.0150
     5 locks_remove_posix                         0.0182
     5 do_aic7xxx_isr                             0.0323
     5 chrdev_open                                0.0121
     5 access_process_vm                          0.0108
     5 __pollwait                                 0.0254
     5 __free_pages                               0.0694
     4 unix_stream_data_wait                      0.0134
     4 unix_release_sock                          0.0060
     4 tg3_rx                                     0.0043
     4 show_cpuinfo                               0.0059
     4 setup_sigcontext                           0.0135
     4 setup_rt_frame                             0.0059
     4 remove_wait_queue                          0.0541
     4 proc_lookup                                0.0148
     4 prep_new_page                              0.0488
     4 page_cache_readahead                       0.0105
     4 notify_change                              0.0102
     4 new_inode                                  0.0345
     4 max_select_fd                              0.0177
     4 kill_bdev                                  0.0588
     4 ip_route_input_slow                        0.0018
     4 get_empty_filp                             0.0167
     4 eventpoll_init_file                        0.1481
     4 dput                                       0.0099
     4 do_generic_mapping_read                    0.0031
     4 alloc_skb                                  0.0179
     4 __constant_c_and_count_memset              0.0312
     3 wake_up_process_kick                       0.0714
     3 unix_stream_sendmsg                        0.0030
     3 unix_poll                                  0.0194
     3 tg3_poll                                   0.0103
     3 sys_pwrite64                               0.0236
     3 split_vma                                  0.0103
     3 sock_map_fd                                0.0091
     3 sock_alloc_send_pskb                       0.0059
     3 sigprocmask                                0.0142
     3 scsi_init_io                               0.0095
     3 page_address                               0.0173
     3 open_namei                                 0.0029
     3 lru_add_drain                              0.0357
     3 kmap_high                                  0.0065
     3 hugetlbfs_statfs                           0.0370
     3 generic_fillattr                           0.0176
     3 follow_mount                               0.0214
     3 find_vma_prepare                           0.0297
     3 file_move                                  0.0526
     3 file_kill                                  0.0682
     3 fd_install                                 0.0536
     3 exit_notify                                0.0016
     3 dup_task_struct                            0.0149
     3 do_settimeofday                            0.0099
     3 do_munmap                                  0.0071
     3 disk_stats_read                            0.0041
     3 count_semncnt                              0.0273
     3 calc_crc_errors                            0.0155
     3 alloc_inode                                0.0090
     3 __get_free_pages                           0.0303
     3 __fput                                     0.0151
     3 __copy_user_zeroing_intel                  0.0174
     3 __blk_queue_bounce                         0.0052
     3 .text.lock.read_write                      0.0577
     2 wait_for_packet                            0.0062
     2 vma_link                                   0.0122
     2 vm_acct_memory                             0.0370
     2 unmap_region                               0.0058
     2 unix_accept                                0.0066
     2 task_prio                                  0.1429
     2 sys_fstat64                                0.0351
     2 sys_connect                                0.0113
     2 sys_brk                                    0.0073
     2 sys_alarm                                  0.0241
     2 sys_accept                                 0.0063
     2 sync_mapping_buffers                       0.0444
     2 sock_def_readable                          0.0157
     2 skb_recv_datagram                          0.0084
     2 shmem_swp_alloc                            0.0032
     2 set_page_address                           0.0060
     2 seq_read                                   0.0027
     2 schedule_tail                              0.0130
     2 proc_pident_lookup                         0.0037
     2 old_mmap                                   0.0063
     2 nr_free_pages                              0.0290
     2 memcpy_fromiovec                           0.0118
     2 lookup_mnt                                 0.0192
     2 locks_remove_flock                         0.0103
     2 load_elf_binary                            0.0007
     2 kunmap_atomic                              0.1667
     2 kobj_lookup                                0.0069
     2 kfree_skbmem                               0.0455
     2 invalidate_vcache                          0.0139
     2 init_new_context                           0.0085
     2 get_signal_to_deliver                      0.0024
     2 free_pages_and_swap_cache                  0.0153
     2 fn_hash_lookup                             0.0085
     2 filp_close                                 0.0165
     2 file_read_actor                            0.0085
     2 ext2_get_inode                             0.0066
     2 eligible_child                             0.0114
     2 elf_map                                    0.0107
     2 do_sync_read                               0.0109
     2 do_lookup                                  0.0124
     2 do_irq_balance                             0.0019
     2 dnotify_flush                              0.0128
     2 dentry_open                                0.0053
     2 create_elf_tables                          0.0022
     2 copy_strings                               0.0031
     2 convert_fxsr_to_user                       0.0057
     2 change_protection                          0.0035
     2 bounce_copy_vec                            0.0175
     2 alloc_ldt                                  0.0047
     2 add_wait_queue                             0.0286
     2 __user_walk                                0.0213
     2 __bio_map_user                             0.0030
     1 yield                                      0.0455
     1 wake_up_inode                              0.0156
     1 vsscanf                                    0.0008
     1 vgacon_cursor                              0.0024
     1 vfs_stat                                   0.0110
     1 vfs_permission                             0.0031
     1 vfs_fstat                                  0.0125
     1 unix_stream_connect                        0.0010
     1 unix_sock_destructor                       0.0036
     1 unix_peer_get                              0.0238
     1 uart_wait_until_sent                       0.0045
     1 sys_write                                  0.0101
     1 sys_wait4                                  0.0016
     1 sys_socketcall                             0.0015
     1 sys_shmdt                                  0.0030
     1 sys_rt_sigreturn                           0.0032
     1 sys_read                                   0.0101
     1 sys_mmap2                                  0.0059
     1 sys_ioctl                                  0.0016
     1 sys_dup2                                   0.0044
     1 sys_clock_nanosleep                        0.0034
     1 sys_access                                 0.0030
     1 sync_sb_inodes                             0.0017
     1 supplemental_group_member                  0.0204
     1 submit_bh                                  0.0029
     1 sockfd_lookup                              0.0084
     1 sock_wfree                                 0.0132
     1 sock_init_data                             0.0030
     1 sock_destroy_inode                         0.0323
     1 sock_create                                0.0027
     1 sock_alloc_send_skb                        0.0196
     1 sock_alloc                                 0.0106
     1 skip_atoi                                  0.0154
     1 skb_release_data                           0.0045
     1 skb_free_datagram                          0.0256
     1 shm_open                                   0.0111
     1 shm_get_stat                               0.0064
     1 shm_close                                  0.0063
     1 serial_in                                  0.0156
     1 seq_printf                                 0.0116
     1 sched_migrate_task                         0.0127
     1 save_i387_fxsave                           0.0050
     1 save_i387                                  0.0055
     1 s_start                                    0.0024
     1 run_timer_softirq                          0.0026
     1 restore_i387                               0.0079
     1 reserve_blocks                             0.0055
     1 release_task                               0.0024
     1 rb_insert_color                            0.0044
     1 rb_erase                                   0.0044
     1 raw_release                                0.0069
     1 raw_open                                   0.0033
     1 proc_readdir                               0.0022
     1 proc_info_read                             0.0029
     1 proc_destroy_inode                         0.0323
     1 prepare_binprm                             0.0044
     1 prep_compound_page                         0.0185
     1 posix_locks_conflict                       0.0058
     1 pipe_read                                  0.0017
     1 path_release                               0.0159
     1 netif_receive_skb                          0.0022
     1 mpage_writepages                           0.0014
     1 move_addr_to_kernel                        0.0069
     1 monotonic_clock_cyclone                    0.0061
     1 mmput                                      0.0060
     1 mm_init                                    0.0040
     1 meminfo_read_proc                          0.0020
     1 locks_insert_lock                          0.0143
     1 locks_conflict                             0.0323
     1 load_elf_interp                            0.0017
     1 lease_alloc                                0.0044
     1 kmap                                       0.0455
     1 ipc_addid                                  0.0058
     1 inode_update_time                          0.0059
     1 huge_pte_alloc                             0.0244
     1 hash_vcache                                0.0400
     1 handle_signal                              0.0032
     1 handle_IRQ_event                           0.0100
     1 get_zeroed_page                            0.0082
     1 get_wchan                                  0.0082
     1 get_unmapped_area                          0.0029
     1 generic_forget_inode                       0.0033
     1 follow_page                                0.0024
     1 flush_signal_handlers                      0.0149
     1 flush_old_exec                             0.0005
     1 flush_all_zero_pkmaps                      0.0056
     1 finish_wait                                0.0111
     1 find_task_by_pid                           0.0130
     1 find_group_orlov                           0.0012
     1 find_busiest_node                          0.0015
     1 filldir64                                  0.0032
     1 file_ra_state_init                         0.0312
     1 fcntl_setlk                                0.0014
     1 fasync_helper                              0.0043
     1 ext2_new_block                             0.0007
     1 ext2_free_blocks                           0.0014
     1 ext2_find_entry                            0.0015
     1 ext2_block_to_path                         0.0031
     1 ext2_alloc_branch                          0.0019
     1 ext2_add_link                              0.0009
     1 exit_thread                                0.0185
     1 error_code                                 0.0179
     1 end_buffer_async_write                     0.0051
     1 elv_put_request                            0.0303
     1 do_sync_write                              0.0055
     1 do_signal                                  0.0038
     1 do_sigaltstack                             0.0028
     1 do_rw_proc                                 0.0062
     1 do_pollfd                                  0.0069
     1 do_poll                                    0.0051
     1 destroy_inode                              0.0125
     1 destroy_context                            0.0058
     1 deny_write_access                          0.0159
     1 default_idle                               0.0222
     1 dcache_readdir                             0.0021
     1 d_rehash                                   0.0111
     1 copy_siginfo_to_user                       0.0025
     1 con_write_room                             0.0370
     1 complete                                   0.0123
     1 cap_vm_enough_memory                       0.0053
     1 cap_task_post_setuid                       0.0034
     1 can_vma_merge_after                        0.0109
     1 can_share_swap_page                        0.0097
     1 call_rcu                                   0.0192
     1 batch_entropy_process                      0.0047
     1 assign_type                                0.0417
     1 alloc_buffer_head                          0.0135
     1 __vma_link                                 0.0063
     1 __unix_insert_socket                       0.0109
     1 __rb_erase_color                           0.0024
     1 __pagevec_lru_add                          0.0048
     1 __kfree_skb                                0.0046
     1 __find_get_block_slow                      0.0029
     1 __filemap_copy_from_user_iovec             0.0057
     1 SHATransform                               0.0033
     1 .text.lock.swap                            0.0103
     1 .text.lock.file_table                      0.0092



Mike Sullivan


             reply	other threads:[~2003-09-08 19:00 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2003-09-08 18:58 Mike Sullivan [this message]
2003-09-08 20:26 ` I/O degredation with AS on 2.6.0-test3 Dave Hansen
2003-09-09  2:44   ` Nick Piggin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=OF0770DEDD.BEBEF8A2-ON85256D9B.00659B11@us.ibm.com \
    --to=mksully@us.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®