From: "Martin J. Bligh" <Martin.Bligh@us.ibm.com>
To: Linus Torvalds <torvalds@transmeta.com>
Cc: Benjamin LaHaise <bcrl@redhat.com>,
Andrea Arcangeli <andrea@suse.de>,
Alan Cox <alan@lxorguk.ukuu.org.uk>,
Chris Friesen <cfriesen@nortelnetworks.com>,
Pavel Machek <pavel@elf.ucw.cz>,
linux-kernel@vger.kernel.org, linux-aio@kvack.org
Subject: Re: lots of mem on 32 bit machines (was: aio-core why not using SuS?)
Date: Fri, 16 Aug 2002 22:24:09 -0700 [thread overview]
Message-ID: <2160605666.1029536648@[10.10.2.3]> (raw)
In-Reply-To: <Pine.LNX.4.44.0208162156090.2539-100000@home.transmeta.com>
> Btw, at this point I should say that that doesn't mean that I'm _against_
> a per-VM kmap table, I'm just pointing out that it's not a trivial issue.
>
> Andrea and I talked about exactly this at OLS, because Andrea would have
> liked to use it for handling the generic_file_write() kmap case without
> having to worry about running out of kmap's and the deadlocks we used to
> have in that space (before the atomic user copy approach).
I know ;-) We talked over that one a lot ... but I think we ended
up just avoiding the problem (every solution we came up with sucked).
I wasn't so worried about avoiding kmap pool exhaustion as reducing
TLB flushing - if we put 4096 entries in the pool, and can cope with
blocking if the silly thing does happen to fill up, that seems OK.
The *really* nice thing is that you can do all sorts of smart stuff
with the TLB at this point. As we now duplicated the pool per VM,
it just got much, much larger. Chances are by the time you come to
re-use an entry you've already TLB flushed all the CPUs from context
switching. Keep track of that and you *never* tlb flush for kmap.
But in the end the two main users of kmap seem to be copy_to/from_user
stuff and highpte - using atomic for the former and the per-VM space
for the latter seems to fix the problem far enough that nobody could
see it anymore, so nobody cares ;-)
> And the thing is, you _can_ use a per-VM kmap setup, but it really only
> moves the problem from a global kmap space ("everybody shares the same
> VM") into a slightly smaller subset of it, a global thread kmap ("all
> threads share the same VM").
Well, I disagree with the "slightly" in that statement, but otherwise
yes ;-) If you're not running heavy threading, it works brilliantly.
If you are, it's better, but not necessarily brilliant.
> There may be other cases where this is ok. Moving to a per-VM kmap space
> may not _fix_ some fundamental scalability problem, but it might move it
> further out and make it a non-issue under normal load. Which is why I
> don't think the idea is fundamentally flawed, I just wanted to point out
> some of the traps to people since we've already almost fallen into some of
> them..
Yup, I think we ended up pushing it out by other means, but it's
still interesting. Things like this only need to be pushed out
until the timely death of 32 bit machines ;-)
If I'm not insane about the per-task stuff in the previous email,
that could be used for a per-task kmap, which would be much nicer
all round. But the chances of me being insane are pretty good ;-)
M.
next prev parent reply other threads:[~2002-08-17 5:22 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2002-07-30 5:41 async-io API registration for 2.5.29 Andrea Arcangeli
2002-07-30 8:11 ` Christoph Hellwig
2002-07-30 13:40 ` Linus Torvalds
2002-07-30 13:52 ` Benjamin LaHaise
2002-07-30 16:43 ` Andrea Arcangeli
2002-07-30 16:59 ` Benjamin LaHaise
2002-07-30 19:10 ` Jeff Dike
2002-07-30 18:09 ` Benjamin LaHaise
2002-07-30 18:15 ` Linus Torvalds
2002-07-30 18:31 ` Benjamin LaHaise
2002-07-30 20:57 ` Jeff Dike
2002-07-30 20:47 ` Jeff Dike
2002-07-30 21:26 ` Andrea Arcangeli
2002-07-30 10:50 ` Rik van Riel
2002-07-30 12:49 ` Benjamin LaHaise
2002-07-30 13:29 ` Suparna Bhattacharya
2002-07-30 21:41 ` Andrea Arcangeli
2002-07-30 21:54 ` [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29) Benjamin LaHaise
2002-07-31 0:44 ` Andrea Arcangeli
2002-07-31 14:46 ` Benjamin LaHaise
2002-07-31 16:31 ` Charles 'Buck' Krasic
2002-08-01 10:30 ` Pavel Machek
2002-08-01 14:47 ` Benjamin LaHaise
2002-08-01 15:00 ` Chris Friesen
2002-08-01 16:09 ` Linus Torvalds
2002-08-01 17:30 ` Alan Cox
2002-08-01 16:30 ` Linus Torvalds
2002-08-01 16:41 ` [rfc] aio-core for 2.5.29 (Re: async-io API registration for2.5.29) Chris Friesen
2002-08-01 18:01 ` [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29) Benjamin LaHaise
2002-08-15 23:54 ` aio-core why not using SuS? [Re: [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29)] Andrea Arcangeli
2002-08-16 1:42 ` Benjamin LaHaise
2002-08-16 1:57 ` Andrea Arcangeli
2002-08-16 2:00 ` Benjamin LaHaise
2002-08-16 2:08 ` Linus Torvalds
2002-08-16 2:16 ` Benjamin LaHaise
2002-08-16 2:40 ` Andrea Arcangeli
2002-08-16 3:43 ` Linus Torvalds
2002-08-16 3:50 ` Linus Torvalds
2002-08-16 4:47 ` William Lee Irwin III
2002-08-17 3:46 ` Martin J. Bligh
2002-08-17 4:00 ` Linus Torvalds
2002-08-17 4:15 ` Martin J. Bligh
2002-08-17 4:46 ` Linus Torvalds
2001-11-02 5:12 ` Pavel Machek
2002-08-17 5:04 ` Linus Torvalds
2002-08-17 5:24 ` Martin J. Bligh [this message]
2002-08-17 5:12 ` Martin J. Bligh
2002-08-17 17:02 ` Linus Torvalds
2002-08-17 21:27 ` 32 bit arch with lots of RAM Martin J. Bligh
2002-08-22 16:30 ` Andrea Arcangeli
2002-08-22 16:36 ` Martin J. Bligh
2002-08-22 16:15 ` aio-core why not using SuS? [Re: [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29)] Andrea Arcangeli
2002-08-22 16:12 ` Andrea Arcangeli
2002-08-20 0:35 ` Ingo Molnar
2002-08-17 4:36 ` William Lee Irwin III
2002-08-16 2:32 ` Rik van Riel
2002-08-16 2:32 ` Andrea Arcangeli
2002-08-16 9:39 ` Suparna Bhattacharya
2002-08-16 10:03 ` Andrea Arcangeli
2002-08-16 11:23 ` Suparna Bhattacharya
2002-08-16 11:28 ` Suparna Bhattacharya
2002-08-16 13:49 ` Dan Kegel
2002-09-02 18:40 ` Andrea Arcangeli
2002-09-03 12:04 ` aio-core in 2.5 - io_queue_wait and io_getevents Suparna Bhattacharya
2002-09-05 5:21 ` aio-core why not using SuS? [Re: [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29)] Benjamin LaHaise
2002-08-16 13:43 ` Dan Kegel
2002-08-16 14:21 ` Jamie Lokier
2002-08-16 14:42 ` Benjamin LaHaise
2002-08-16 15:40 ` John Gardiner Myers
2002-08-23 16:11 ` aio-core why not using SuS? [Re: [rfc] aio-core for 2.5.29 (Re:async-io " Dan Kegel
2002-08-16 1:53 ` aio-core why not using SuS? [Re: [rfc] aio-core for 2.5.29 (Re: async-io " Dan Kegel
2002-08-01 19:18 ` [rfc] aio-core for 2.5.29 (Re: async-io API registration for 2.5.29) Chris Wedgwood
2002-08-01 19:25 ` Linus Torvalds
2002-08-01 19:31 ` Chris Wedgwood
2002-08-02 8:24 ` Pavel Machek
2002-08-02 11:59 ` Alan Cox
2002-08-02 15:56 ` Linus Torvalds
2002-07-31 1:20 ` async-io API registration for 2.5.29 Rik van Riel
2002-07-31 1:32 ` Andrea Arcangeli
2002-07-31 8:25 ` Christoph Hellwig
2002-07-31 13:19 ` Andrea Arcangeli
2002-07-30 13:34 ` Linus Torvalds
2002-07-30 16:49 ` Andrea Arcangeli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='2160605666.1029536648@[10.10.2.3]' \
--to=martin.bligh@us.ibm.com \
--cc=alan@lxorguk.ukuu.org.uk \
--cc=andrea@suse.de \
--cc=bcrl@redhat.com \
--cc=cfriesen@nortelnetworks.com \
--cc=linux-aio@kvack.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pavel@elf.ucw.cz \
--cc=torvalds@transmeta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®