Badari Pulavarty wrote:
>
> ...
> max_sectors = 512
> SG_SEGMENTS = 32
> [root]# vmstat 5
> procs memory swap io system cpu
> r b w swpd free buff cache si so bi bo in cs us sy id
> 0 20 0 0 3652472 7752 34464 0 0 170755 15 2354 3477 2 98 0
> 0 20 0 0 3652472 7756 34464 0 0 172771 3 2363 3512 2 98 0
> 0 20 0 0 3652472 7760 34464 0 0 174031 3 2372 3576 0 100 0
> 0 20 0 0 3652408 7764 34464 0 0 173074 7 2384 3538 10 87 3
> 0 20 0 0 3652408 7768 34464 0 0 172670 3 2364 3502 2 98 0
I don't know what happened to your bandwidth, but that context switch rate
is excessive. qlogic's small BIOs are hurting.
We don't actually need to perform a wakeup-per-BIO in there. It is sufficient
to deliver a single wakeup on the very last BIO.
This should drop your CPU load a bit. (vmstat numbers might not change - the
statistical process accounting may not be very accurate for this sort of
thing. Use cyclesoak: http://www.zip.com.au/~akpm/linux/#zc)
fs/direct-io.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
--- 25/fs/direct-io.c~dio-reduce-context-switch-rate Sat Nov 16 16:55:25 2002
+++ 25-akpm/fs/direct-io.c Sat Nov 16 16:55:25 2002
@@ -105,7 +105,8 @@ struct dio {
int page_errors; /* errno from get_user_pages() */
/* BIO completion state */
- atomic_t bio_count; /* nr bios in flight */
+ atomic_t bio_count; /* nr bios to be completed */
+ atomic_t bios_in_flight; /* nr bios in flight */
spinlock_t bio_list_lock; /* protects bio_list */
struct bio *bio_list; /* singly linked via bi_private */
struct task_struct *waiter; /* waiting task (NULL if none) */
@@ -238,7 +239,8 @@ static int dio_bio_end_io(struct bio *bi
spin_lock_irqsave(&dio->bio_list_lock, flags);
bio->bi_private = dio->bio_list;
dio->bio_list = bio;
- if (dio->waiter)
+ atomic_dec(&dio->bios_in_flight);
+ if (dio->waiter && atomic_read(&dio->bios_in_flight) == 0)
wake_up_process(dio->waiter);
spin_unlock_irqrestore(&dio->bio_list_lock, flags);
return 0;
@@ -271,6 +273,7 @@ static void dio_bio_submit(struct dio *d
bio->bi_private = dio;
atomic_inc(&dio->bio_count);
+ atomic_inc(&dio->bios_in_flight);
submit_bio(dio->rw, bio);
dio->bio = NULL;
@@ -852,6 +855,7 @@ direct_io_worker(int rw, struct kiocb *i
* still submittion BIOs.
*/
atomic_set(&dio->bio_count, 1);
+ atomic_set(&dio->bios_in_flight, 0);
spin_lock_init(&dio->bio_list_lock);
dio->bio_list = NULL;
dio->waiter = NULL;
_
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Sat Nov 23 2002 - 22:00:18 EST