Re: [PATCH 0/3]: blk-iopoll, a polled completion API for block devices

[Bart Van Assche]

On Tue, Aug 11, 2009 at 7:14 PM, Jens Axboe<jens.axboe@xxxxxxxxxx> wrote:
> On Tue, Aug 11 2009, Bart Van Assche wrote:
>> On Tue, Aug 11, 2009 at 4:39 PM, Jens Axboe<jens.axboe@xxxxxxxxxx> wrote:
>> > On Tue, Aug 11 2009, Bart Van Assche wrote:
>> >> On Thu, Aug 6, 2009 at 9:58 PM, Jens Axboe <jens.axboe@xxxxxxxxxx> wrote:
>> >> > Anyway, YMMV, I would appreciate some test results (and as usual, that
>> >> > even includes just saying that it boots and functions for you). If
>> >> > people feel adventurous, patches for other controllers will be happily
>> >> > queued up for testing. I may even be convinced to implement support
>> >> > for your controller of choice, if you have some fast storage hooked up
>> >> > and would like to experiment. Generally, adding support to a driver is
>> >> > not very hard and the two conversions included were also meant to serve
>> >> > as an inspiration.
>> >>
>> >> Sounds very interesting. Have you already considered patching the SRP
>> >> initiator ? During the SRP performance tests I ran CPU usage on the
>> >> initiator was more than 95% and on the target less than 10%.
>> >
>> > No I haven't, if you point me at which srp files, I can take a look.
>>
>> The relevant source files are:
>> include/scsi/srp.h
>> include/scsi/scsi_transport_srp.h
>> drivers/infiniband/ulp/srp/ib_srp.h
>> drivers/infiniband/ulp/srp/ib_srp.c
>> drivers/scsi/scsi_transport_srp.c
>> drivers/scsi/libsrp.c
>> drivers/scsi/scsi_transport_srp_internal.h
>
> I can find | grep too :-)

The command "find | grep srp" would have returned a few more source
files. The most relevant starting point is the source file
drivers/infiniband/ulp/srp/ib_srp.c, which contains the struct
scsi_host_template.

> Did you profile this? Where did it burn all the CPU time on the
> initiator side?

The test I ran involved a Linux SRP initiator and a Linux SRP target
(SCST) using a RAM disk as backstorage. Read throughput is about 1700
MB/s for block sizes of 8 MB and above. But with a block size of 4 KB,
the read throughput on the initiator drops to 100 MB/s. At this block
size there are about 50.000 interrupts per second generated by the
InfiniBand HCA in the initiator system. On the same setup the
ib_send_bw tool reports a throughput of 1850 MB/s for a block size of
4 KB. This last tool is not interrupt driven but uses polling.

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/