LFS performance and kern.maxvnodes

[Blair Sadewitz]

Try cranking up kern.maxvnodes, and you'll notice that at some point,
you'll be able to untar pkgsrc without tar/LFS eating up all your CPU
time. This should probably be in the guide.

Regards,

--Blair

[Daniel Carosone]

Post by Blair Sadewitz
Try cranking up kern.maxvnodes, and you'll notice that at some point,
you'll be able to untar pkgsrc without tar/LFS eating up all your CPU
time. This should probably be in the guide.

Yeah, the same advice is good for getting the most of ufs+softdep,
too.

--
Dan.

[Blair Sadewitz]

I think there might be a problem with pool(9) or the amount of pools
that are allocated for bpppool ... I'm not really sure. What I have
noticed is that after increasing kern.maxvnodes, my system hasn't
crashed in 24 hours using an LFS root partition. I don't know enough
about LFS to say what's actually going on without looking at it more,
but this combined with what I outlined in the PR I filed recently
seems to suggest that we need to sanity-check things a lot more--or at
least add something under 'BUGS' or 'CAVEATS' in newfs_lfs(8).

--Blair

[Adam Hamsik]

On Aug 27, 2007, at 9:56 AM, Blair Sadewitz wrote:
hi

Post by Blair Sadewitz
Try cranking up kern.maxvnodes, and you'll notice that at some point,
you'll be able to untar pkgsrc without tar/LFS eating up all your CPU
time. This should probably be in the guide.
Regards,
--Blair

Today I found that is good idea to increase this values also if
system is under the bigger filesystem activity e.g. cvs update. With
running cvs update system is slow in fs operations like directory
list. After increas of kern.maxvnodes to 65000 (i know this is
probably too much;) system worked smoothly again.

We should increase default kern.maxvnodes to something bigger than
45692 or at least document this somewhere so people can tune their
netbsd better. Is tunning netbsd chapter in our guide correct place ?
if yes I will write a patch .



Regards
- -----------------------------------------
Adam Hamsik
jabber: ***@jabber.org
icq: 249727910

Proud NetBSD user.

We program to have fun.
Even when we program for money, we want to have fun as well.
~ Yukihiro Matsumoto

[Blair Sadewitz]

On 512MB RAM machines I tend to use 128k as a
minimum as well.

On my desktop system with 1GB RAM, 262144 seems to yield the best
results for me (I've mainly used untarring pkgsrc as a test thusfar).
At times going higher can help, but that's not that often for me.
Thus, 128k seems like a reasonably conservative default--at least for
machines with lots of memory.

Another thing I was going to propose was setting sysctl.conf defaults
from sysinst using basic heuristics, taking into account [at least]
the amount of system memory and anticipated workload.
We could then use this information to help suggest more intelligent
defaults for things such as base filesystem layout and bsize/fsize,
etc.

--Blair

[Allen Briggs]

Post by Blair Sadewitz

On 512MB RAM machines I tend to use 128k as a
minimum as well.

On my desktop system with 1GB RAM, 262144 seems to yield the best
results for me (I've mainly used untarring pkgsrc as a test thusfar).

I have found that if I crank it too high, my system hangs (amd64).
I don't think it's out of memory, but I haven't had a chance to
track it down--I just backed it down to 262144 (trying with 524288
at the moment, though). I think I had it at 2097152 when it wedged
last (the machine has 6GB of RAM running GENERIC.MP from 30 Aug 07).

Post by Blair Sadewitz
Another thing I was going to propose was setting sysctl.conf defaults
from sysinst using basic heuristics, taking into account [at least]
the amount of system memory and anticipated workload.
We could then use this information to help suggest more intelligent
defaults for things such as base filesystem layout and bsize/fsize,
etc.

This sounds to me like a good start.

-allen

[David Holland]

Post by Blair Sadewitz
Another thing I was going to propose was setting sysctl.conf defaults
from sysinst using basic heuristics, taking into account [at least]
the amount of system memory and anticipated workload.
We could then use this information to help suggest more intelligent
defaults for things such as base filesystem layout and bsize/fsize,
etc.

I'm not going to say this is a bad idea - but in this case wouldn't it
be better to make the kernel default smarter? Then you don't have
wired-down values in /etc that go out of date when the hardware gets
rearranged.

There's already code in init_main.c that checks the default
MAXUSERS-based value for being too low and substitutes "1% of memory"
(based on the size of struct vnode), resulting in 62.4 vnodes per
megabyte. With the default MAXUSERS of 32 this kicks in when you have
over 12 megabytes of RAM. It wouldn't be difficult to pump this up
more for larger memory sizes.

[Simon Burge]

Post by Adam Hamsik
Today I found that is good idea to increase this values also if
system is under the bigger filesystem activity e.g. cvs update. With
running cvs update system is slow in fs operations like directory
list. After increas of kern.maxvnodes to 65000 (i know this is
probably too much;) system worked smoothly again.
We should increase default kern.maxvnodes to something bigger than
45692 or at least document this somewhere so people can tune their
netbsd better. Is tunning netbsd chapter in our guide correct place ?
if yes I will write a patch .

I generally use "RAM MB * 128" for sizing maxvnodes, although on 512MB
RAM machines I tend to use 128k as a minimum as well.

Note that configuring extra vnodes will have an impact on memory usage.
For example, on my 1GB RAM laptop, "vmstat -mW" shows that the 4 pools
in use that are affected by maxvnodes (ncachepl, dino1pl, ffsinopl,
vnodepl - and this is dependent on which filesystems you use) are using
80MB of RAM (which is actually more than I thought it would be using!).
So there is some sort of trade-off in going too large...

Certainly I think it's worth an entry in the tuning chapter.

Cheers,
Simon.

[Blair Sadewitz]

I was about to propose something like this. I don't recall what's in
the guide, but I use pstat -T to check the number of available vnodes.

There are many different ways this could be set better by default.
I'm still formulating how I think this should be done, but IMHO
maxusers is a bit archaic. Is it so sacrosanct that we couldn't
implement something else to make tuning NetBSD more intuitive for the
"average" user? It's always seemed odd to me to base tuning of vital
system parameters upon some constant named something so abstruse that
our documentation must clarify it in order for anyone to ever hazard a
guess as to what it really does!

How about raising the default at least on platforms where memory is
usually plentiful, e.g. amd64? If someone's using the amd64 port in,
for example, an embedded application with little memory, presumably
this user would already be proactively tuning the system.

On the other hand, there are surely more elegant [and complex] ways to
go about this. We could have it scale by default along with other vm
parameters, etc.

Any proposals?

Regards,

--Blair