A Customer has extended his Oracle Database Appliance X6-L with Oracle Database Appliance Memory Expansion Kit from 192GB to 384GB. After the first startup, the odacli tool has recognized the additional available memory and gave us the suggested value for the new HugePage settings.
The ODA has the newest available patchset applied, OS is Oracle Enterprise Linux 6.10.
Update the OS Configuration
Oracle suggested the new values for the HugePage settings to 135350MB.
[root@srv20 ~]# odacli list-osconfigurations Parameter User ConfiguredValue SuggestedValue --------------- --------------- ------------------------------ --------------- Memlock grid 386716274KB 386716168KB Memlock oracle 386716274KB 386716168KB HugePage default 67815MB 135350MB
According the Oracle documentation Oracle® Database Appliance X6-2S/X6-2M/X6-2L Deployment and User’s Guide – Section 13.14 – we updated the OS configuration by odacli. The job was finished after some seconds.
odacli update-osconfigurations (page 13-70)
Use the command odacli update-osconfigurations to update the HugePage and memlock values.
[root@srv20]# odacli update-osconfigurations
Verfication
The HugePage Settings were updated now.
[root@srv20 ~]# odacli list-osconfigurations Parameter User ConfiguredValue SuggestedValue --------------- --------------- ------------------------------ --------------- Memlock grid 386716168KB 386716168KB Memlock oracle 386716168KB 386716168KB HugePage default 135350MB 135350MB
Time for a hardware restart.
Old HugePage Values after ODA Restart
But, after the machine has started, the old HugePage values were back again. But who has overwritten the settings?
[root@srv20 ~]# odacli list-osconfigurations Parameter User ConfiguredValue SuggestedValue --------------- --------------- ------------------------------ --------------- Memlock grid 386716274KB 386716168KB Memlock oracle 386716274KB 386716168KB HugePage default 67815MB 135350MB
An SR has solved the problem, the are two files in the OS which are responsible for that behavior:
- /etc/sysctl.d/99-initial-sysctl.conf
- /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf
We take a look at the values:
[root@srv20 ~]# cat /etc/sysctl.d/99-initial-sysctl.conf | grep huge vm.nr_hugepages=67815 [root@srv20 ~]# cat /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf | grep huge vm.nr_hugepages=67815
Solution: Remove these two Files
First, we updated the OS Configuration again.
[root@srv20 ~]# odacli update-osconfigurations
Second, we removed the files.
[root@srv20 ~]# rm /etc/sysctl.d/99-initial-sysctl.conf [root@srv20 ~]# rm /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf
Third, we restarted the Oracle Database Appliance.
Finally
HugePage settings are persistent now. By the way, there is an My Oracle Support note for the OS Level 6.10 which describes such a case too:
Changes in sysctl.conf File are not Persistent after Reboot (Doc ID 2477376.1)