I've been involved in a project to migrate an E-Business Suite database tier from 32-bit to 64-bit RDBMS. Thus far it's been pretty straightforward. One thing had been bugging me, though. Our original test platform was 64-bit SUSE Linux Enterprise Server 9 (SLES9), but a decision was made to upgrade to SLES10 instead. Not a big deal in this case, except for one thing: the settings I'd used in SLES9 to allocate hugepages weren't doing a darn thing in SLES10:
sles10db:~ # grep -i nr_huge /etc/sysctl.conf vm.nr_hugepages = 8196 sles10db:~ # grep memlock /etc/security/limits.conf oracle soft memlock 17000000 oracle hard memlock 17000000
Okay so far...
sles10db:~ # grep -i huge /proc/meminfo HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 Hugepagesize: 2048 kB sles10db:~ # cat /proc/sys/vm/nr_hugepages 0
Whahuh? Why wasn't my
nr_hugepages setting getting picked up from /etc/sysctl.conf? Wrong question, of course. If other settings in that file were being picked up (they were), then clearly something else was clobbering the value I had set. Sure enough, after not much poking around, I found this:
sles10db:~ # grep -i nr_huge /etc/sysconfig/oracle # /proc/sys/vm/nr_hugepages indicates the current number of configured hugetlb NR_HUGE_PAGES=0
How embarrassing is that? It's even documented in a very official, check-here-first sort of place. Since /etc/init.d/oracle is run long after /etc/init.d/boot.sysctl, the values set in /etc/sysconfig/oracle clearly will clobber anything set in /etc/sysctl.conf. My only defense is that
NR_HUGE_PAGES was not included in /etc/sysconfig/oracle on our SLES9 test system, which is why I was setting it in /etc/sysctl.conf in the first place. Of course, we might've been using an old version of orarun.rpm on that server, so it's still a pretty weak excuse. At any rate, changing the setting in /etc/sysconfig/oracle and running
rcoracle restart did the job:
sles10db:~ # grep -i nr_huge /etc/sysconfig/oracle # /proc/sys/vm/nr_hugepages indicates the current number of configured hugetlb NR_HUGE_PAGES=8196 sles10db:~ # grep -i huge /proc/meminfo HugePages_Total: 4645 HugePages_Free: 4645 HugePages_Rsvd: 0 Hugepagesize: 2048 kB
Hope this helps someone out there; otherwise, I'll just sit here, alone in my hopefully-temporary idiocy.