SCSI tape buffer allocation hitting memory allocation

Simon Kirby (sim@netnation.com)
Fri, 12 Feb 1999 13:18:39 -0800 (PST)


This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.

---1362361889-790945333-918854319=:27028
Content-Type: TEXT/PLAIN; charset=US-ASCII

We've got a Quantum DLT7000 35GB uncompressed tape drive and we're using
it to do massive amounts of data backup. We've found that the transfer
rate is much, much faster using 64KB blocks instead of smaller (4KB)
blocks, so we switched it over to use these. Unfortunately, it seems to
hit memory fragmentation occasionally and barf with this message:

Feb 12 02:56:50 alfie kernel: st: failed to enlarge buffer to 65536 bytes.

Looking at the code, it seems to do this when scsi_init_malloc() returns
NULL (fails). So, I hit alt-sysreq...Sure enough, there didn't appear to
be any large areas free. I remembered that I saw that the floppy driver
once spat out that it was resorting to virtual DMA, so I thought that
maybe I would be able to convert scsi_init_malloc() to fall back to use
vmalloc() if allocation in physical memory fails. I don't know if this
will work or not, though, recalling that DMA has some funky limitations
(has to be kept < 16MB?) I'm not sure what limitations apply to the
aic7xxx SCSI stuff, though.

I wrote a patch that at first appeared to work -- it reports that it fell
back to vmalloc(), and it seemed to get the data off the tape without
any problems. It seemed to then report that it freed as many allocations
as it had allocated with vmalloc() (good news):

Feb 12 12:03:28 alfie kernel: scsi_init_malloc(): Unable to allocate 32768 bytes of contiguous memory -- falling back on virtual allocation.
Feb 12 12:03:28 alfie last message repeated 10 times
Feb 12 12:03:28 alfie kernel: scsi_init_free(): Freeing virtual memory.
Feb 12 12:03:28 alfie last message repeated 10 times

And I was happy...But then I ran the exact same command again and the
machine froze hard (even interrupts were dead -- no sysreq). So, is this
not possible? Is there another workaround that could be done? Is there
just a dumb bug in my patch (attached)?

Simon-

| Simon Kirby | Systems Administration |
| mailto:sim@netnation.com | NetNation Communications |
| http://www.netnation.com/ | Tech: (604) 684-6892 |

---1362361889-790945333-918854319=:27028
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="linux-2.2.1+scsi_virtual_memory_fallback.patch"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.10.9902121318390.27028@peace.netnation.com>
Content-Description:
Content-Disposition: attachment; filename="linux-2.2.1+scsi_virtual_memory_fallback.patch"

ZGlmZiAtdXIgbGludXgub3JpZy9kcml2ZXJzL3Njc2kvc2NzaS5jIGxpbnV4
L2RyaXZlcnMvc2NzaS9zY3NpLmMNCi0tLSBsaW51eC5vcmlnL2RyaXZlcnMv
c2NzaS9zY3NpLmMJTW9uIEZlYiAgOCAwOTo0ODoxMCAxOTk5DQorKysgbGlu
dXgvZHJpdmVycy9zY3NpL3Njc2kuYwlGcmkgRmViIDEyIDA5OjMzOjM1IDE5
OTkNCkBAIC00OCw2ICs0OCw3IEBADQogI2luY2x1ZGUgPGxpbnV4L2ludGVy
cnVwdC5oPg0KICNpbmNsdWRlIDxsaW51eC9kZWxheS5oPg0KICNpbmNsdWRl
IDxsaW51eC9pbml0Lmg+DQorI2luY2x1ZGUgPGxpbnV4L21tLmg+CQkvKiAi
aGlnaF9tZW1vcnkiICovDQogDQogI2RlZmluZSBfX0tFUk5FTF9TWVNDQUxM
U19fDQogDQpAQCAtMTg4NCw2ICsxODg1LDE4IEBADQogICAgIH0gZWxzZQ0K
ICAgICAgICAgcmV0dmFsID0ga21hbGxvYyhzaXplLCBnZnBfbWFzayk7DQog
DQorICAgIC8qDQorICAgICAqIElmIGVpdGhlciBfX2dldF9mcmVlX3BhZ2Vz
KCkgb3Iga21hbGxvYygpIGZhaWxlZCwgZmFsbCBiYWNrIG9uDQorICAgICAq
IHZpcnR1YWwgYWxsb2NhdGlvbi4gLXNpbQ0KKyAgICAgKi8NCisgICAgaWYg
KCFyZXR2YWwgJiYgc2l6ZSA+IFBBR0VfU0laRSkgew0KKwlyZXR2YWwgPSB2
bWFsbG9jKHNpemUpOw0KKwlwcmludGsoInNjc2lfaW5pdF9tYWxsb2MoKTog
VW5hYmxlIHRvIGFsbG9jYXRlICV1IGJ5dGVzIG9mIGNvbnRpZ3VvdXMiDQor
CSAgICAgICAiIG1lbW9yeSAtLSBmYWxsaW5nIGJhY2sgb24gdmlydHVhbCBh
bGxvY2F0aW9uLlxuIixzaXplKTsNCisJaWYgKCFyZXR2YWwpDQorCSAgICBw
cmludGsoInNjc2lfaW5pdF9tYWxsb2MoKTogRWVrISB2bWFsbG9jKCkgYWxz
byBmYWlsZWQhXG4iKTsNCisgICAgfQ0KKw0KICAgICBpZiAocmV0dmFsKQ0K
IAltZW1zZXQocmV0dmFsLCAwLCBzaXplKTsNCiAgICAgcmV0dXJuIHJldHZh
bDsNCkBAIC0xODk3LDE1ICsxOTEwLDI0IEBADQogICAgICAqIHBhZ2UgYWxp
Z25lZCBkYXRhLiAgQmVzaWRlcywgaXQgaXMgd2FzdGVmdWwgdG8gYWxsb2Nh
dGUNCiAgICAgICogcGFnZSBzaXplZCBjaHVua3Mgd2l0aCBrbWFsbG9jLg0K
ICAgICAgKi8NCi0gICAgaWYgKChzaXplICUgUEFHRV9TSVpFKSA9PSAwKSB7
DQotICAgIAlpbnQgb3JkZXIsIGFfc2l6ZTsNCisgICAgLyoNCisgICAgICog
QWxzbywgd2UgbXVzdCBub3cgY2hlY2sgdG8gc2VlIGlmIHdlIGZlbGwgYmFj
ayBvbiB2aXJ0dWFsDQorICAgICAqIGFsbG9jYXRpb24gdG8gYWxsb3cgZm9y
IGxhcmdlIGFsbG9jYXRpb25zIGluIHRoZSBTQ1NJIGNvZGUNCisgICAgICog
d2l0aG91dCBydW5uaW5nIGludG8gbWVtb3J5IGZyYWdtZW50YXRpb24gcHJv
YmxlbXMuIC1zaW0NCisgICAgICovDQorICAgIGlmICgodW5zaWduZWQgaW50
KSBwdHIgPj0gKHVuc2lnbmVkIGludCkgaGlnaF9tZW1vcnkpIHsNCisJdmZy
ZWUoKHZvaWQgKilwdHIpOw0KKyAgICB9IGVsc2Ugew0KKwlpZiAoKHNpemUg
JSBQQUdFX1NJWkUpID09IDApIHsNCisJICAgIGludCBvcmRlciwgYV9zaXpl
Ow0KIA0KLQlmb3IgKG9yZGVyID0gMCwgYV9zaXplID0gUEFHRV9TSVpFOw0K
LQkgICAgIGFfc2l6ZSA8IHNpemU7IG9yZGVyKyssIGFfc2l6ZSA8PD0gMSkN
CisJICAgIGZvciAob3JkZXIgPSAwLCBhX3NpemUgPSBQQUdFX1NJWkU7DQor
CQlzaXplIDwgc2l6ZTsgb3JkZXIrKywgYV9zaXplIDw8PSAxKQ0KIAkgICAg
Ow0KLQlmcmVlX3BhZ2VzKCh1bnNpZ25lZCBsb25nKXB0ciwgb3JkZXIpOw0K
LSAgICB9IGVsc2UNCi0Ja2ZyZWUocHRyKTsNCisJICAgIGZyZWVfcGFnZXMo
KHVuc2lnbmVkIGxvbmcpcHRyLCBvcmRlcik7DQorCX0gZWxzZQ0KKwkgICAg
a2ZyZWUocHRyKTsNCisgICAgfQ0KIH0NCiANCiB2b2lkIHNjc2lfYnVpbGRf
Y29tbWFuZGJsb2NrcyhTY3NpX0RldmljZSAqIFNEcG50KQ0K
---1362361889-790945333-918854319=:27028--

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/