Huge Pages on Oracle Linux

Environment

Oracle Linux Server release 7.3
Oracle database 11.2, 12.1

Quick checking

# cat /proc/sys/vm/nr_hugepages

2054

or

# cat /proc/meminfo | grep Huge
AnonHugePages:         0 kB
HugePages_Total:    2054
HugePages_Free:      546
HugePages_Rsvd:      542
HugePages_Surp:        0
Hugepagesize:       2048 kB




checking if transparent huge pages are disabled

# cat /sys/kernel/mm/transparent_hugepage/enabled

always madvise [never]

Viewing database 12.1 alert log

**********************************************************************

Mon Oct 16 12:28:54 2017

Dump of system resources acquired for SHARED GLOBAL AREA (SGA)

Mon Oct 16 12:28:54 2017

 Per process system memlock (soft) limit = 128G

Mon Oct 16 12:28:54 2017

 Expected per process system memlock (soft) limit to lock

 SHARED GLOBAL AREA (SGA) into memory: 2050M

Mon Oct 16 12:28:54 2017

 Available system pagesizes:

  4K, 2048K

Mon Oct 16 12:28:54 2017

 Supported system pagesize(s):

Mon Oct 16 12:28:54 2017

  PAGESIZE  AVAILABLE_PAGES  EXPECTED_PAGES  ALLOCATED_PAGES  ERROR(s)

        4K       Configured               3               3        NONE

     2048K             2054            1025            1025        NONE

**********************************************************************




Viewing database 11.2 alert log

************************ Large Pages Information *******************

Per process system memlock (soft) limit = 128 GB




Total Shared Global Region in Large Pages = 2050 MB (100%)




Large Pages used by this instance: 1025 (2050 MB)

Large Pages unused system wide = 4 (8192 KB)

Large Pages configured system wide = 2054 (4108 MB)

Large Page size = 2048 KB

********************************************************************

If number of Huge pages is less than required, than SGA will use both page types. Database 12.1 aler.log:

  PAGESIZE  AVAILABLE_PAGES  EXPECTED_PAGES  ALLOCATED_PAGES  ERROR(s)

        4K       Configured               3          187704        NONE

     2048K              659            1025             658        NONE




Simplified setup

1. Have the memlock user limit set in /etc/security/limits.conf file. Set the value (in KB) slightly smaller than total RAM (90%) , at least it must be bigger than HugePages size.
*   soft   memlock    14680064
*   hard   memlock    14680064
Atentu! If you use systemd to auto startup databases, systemd ignores limits.conf. You need to add the following to a service unit file
[Service]
LimitMEMLOCK=infinity
LimitNOFILE=65535



2. Get the required number of pages from alert.log (EXPECTED_PAGES) or by running Oracle script hugepages_settings.sh (Doc ID 401749.1)




3. edit vm.nr_hugepages in /etc/sysctl.conf as root

4. to reload the parameters, reboot Linux or use sysctl -p




Proper setup and more info

HugePages on Oracle Linux 64-bit (Doc ID 361468.1)

HugePages on Linux: What It Is... and What It Is Not... (Doc ID 361323.1)

Oracle Linux: Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration (Doc ID 401749.1)

Oracle 12.2 Documentation: Administering Oracle Database on Linux










Automating Oracle database Shutdown and Startup by systemd

Environment

Oracle Linux Server release 7.3 (or Redhat 7.3)
Oracle database 12.1.0.2

Documentation

MOSC Oracle Doc ID 2229679.1, Doc ID 2049901.1
Overview of systemd for RHEL 7

Automating by systemd

alternatives are Oracle Restart or SysV init ( /etc/init.d in older Linux).
Service Unit file typically has extension .service and stored in
/usr/lib/systemd/system
/etc/systemd/system
/usr/lib/systemd/user
/etc/systemd/user
For more details read Chapter 9. Managing Services with systemd

Create or edit a service unit file:
cd /etc/systemd/system
vi oracle_database.service
[Unit]
Description=The Oracle Database Service
After=network.target

[Service]
Type=forking
RemainAfterExit=yes
KillMode=none
TimeoutStopSec=10min
# memlock limit is needed for SGA to use HugePages
LimitMEMLOCK=infinity
LimitNOFILE=65535

User=oracle
Group=oinstall
# Please use absolute path here
# ExecStart=$ORACLE_HOME/bin/dbstart $ORACLE_HOME &
# First argument of dbstart is used to bring up Listener
ExecStart=/opt/oracle/product/12.1.0/se2_1/bin/dbstart /opt/oracle/product/12.1.0/se2_1 &
ExecStop=/opt/oracle/product/12.1.0/se2_1/bin/dbshut /opt/oracle/product/12.1.0/se2_1
Restart=no

[Install]
# Puts wants directive for the other units in the relationship

WantedBy=default.target

First argument of dbstart/dbshut is used to bring up/shutdown Oracle Listener. This script will start all databases listed in the /etc/oratab file whose third field is a "Y". If you use ASM or cluster services, read more in dbstart description.

Huge Pages
systemd ignores /etc/security/limits.conf. If HugePages are configured, you need to use LimitMEMLOCK and LimitNOFILE, otherwise SGA will use small pages and database alert log will show:

 Increase per process memlock (soft) limit to at least 2050MB to lock 100% of SHARED GLOBAL AREA (SGA) pages into physical memory

TimeoutStopSec=   Configures the time to wait for stop. If a service is asked to stop, but does not terminate in the specified time, it will be terminated forcibly via SIGTERM, and after another timeout of equal duration with SIGKILL (see KillMode= in systemd.kill(5)). Takes a unit-less value in seconds, or a time span value such as "5min 20s". Pass "infinity" to disable the timeout logic. Defaults to DefaultTimeoutStopSec= from the manager configuration file (see systemd-system.conf(5)).
For more info use
man systemd.service

reload systemd and enable the service:
systemctl daemon-reload
systemctl enable oracle_database.service
list services:
systemctl list-unit-files --type service|grep oracle
Start the service and check its status
systemctl start oracle_database.service
systemctl status oracle_database.service