Solaris 11 – Zones for Oracle Database and RAC

im07t1-solaris-cloud-1-1354432 As simple as that!

ZONE CONFIGURATION

Steps:
1.Create the zone configuration;
2.Specify datasets and devices;
3.Configure network;
4.Configure resources and limits;
5.Verify and commit;

1.Create the zone configuration:
Zone configuration is created and maintained through zonecfg utility.

To create a new zone we shall use the following syntax:

root@Kovachev-Pro:~# zonecfg -z zora
Use 'create' to begin configuring a new zone.
zonecfg:zora>

This will open an interactive interface for zone configuration. As we can understand from the output we shall use create statement to create a new zone. The create statement by default loads the default template from /etc/zones/, which can be quite useful for many situations, but in our case we need a blank one. So we shall use create -b:

zonecfg:zora> create -b
zonecfg:zora> info
zonename: zora
zonepath:
brand: solaris
autoboot: false
bootargs:
file-mac-profile:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
hostid:
fs-allowed:
zonecfg:zora>

As we can see from the info output, we have a blank/empty zone configuration. Note that zonecfg can be used for creation and further configuration of already installed zones.

2.Specify datasets and devices:
Each zone needs a zonepath path which shall point to a ZFS dataset. This dataset will become the zone root – /.

zonecfg:zora> set zonepath=/zones/zora

If the child dataset /zora in our case/ does not exist, it will be automatically created during the installation process. If you need to specify some quotas or reservations on it, you can create the child dataset before the installation or edit the zfs properties after that. Keep in mind that the parent dataset shall be outside of /rpool mount point.

Running ASM requires disk devices. If we are creating production environment we shall use directly disk devices or slices from devices. If we are creating a test environment and we need to share few disk devices with many zones, we can use zfs volumes instead.

zonecfg:zora> add device
zonecfg:zora:device> set match=/dev/rdsk/c3t1d0s0
zonecfg:zora:device> set allow-raw-io=true
zonecfg:zora:device> end
zonecfg:zora> add device
zonecfg:zora:device> set match=/dev/zvol/rdsk/ssd2/tst
zonecfg:zora:device> set allow-raw-io=true
zonecfg:zora:device> end
zonecfg:zora>

For the example I add device slice and ZFS volume. The ZFS volume tst is already created with: zfs create -V 10g ssd2/tst. Additionally, for each device we can enable raw-io operations.
Once again, for prod environment use disk devices only.

3.Configure network:
In short, for RAC or DB you need zone with ip-type=exclusive /default/ and net or anet devices.

anet is used for vnics, because it’s managing the vnics automatically . Of course we can use net for manually created vnics, but then we shall take care for everything /doc notes below/.

net is used mostly for physical devices. For example in Oracle RAC you must install the Grid on physical devices or the installation fails. So for RAC we shall use net with physical devices. Of course if we are building test environment we can create 4 vnics with anet and on top of them we can create 2 IPMP interfaces which are accepted from the installer 🙂

net and anet example:

zonecfg:zora> add net
zonecfg:zora:net> set physical=net0
zonecfg:zora:net> end
zonecfg:zora> add anet
zonecfg:zora:anet> set lower-link=net1
zonecfg:zora:anet> set mac-address=auto
zonecfg:zora:anet> end
zonecfg:zora>

Oracle Docs notes – anet:

lower-link

Specify the link over which the VNIC will be created. This property has a default value of auto. The administrator may explicitly specify a value upon adding an anet resource. The link can be any link accepted as an argument to dladm create-vnic”s -l option (see dladm(1M)). If this property is set to a linkname (other than auto) and that link does not exist, then the zone will fail to boot. When set to auto, the zoneadmd(1M) daemon will automatically choose the link over which the VNIC will be created each time the zone boots. A link will be chosen using the following heuristic:

A link aggregation that has a link state of up.

Of the physical Ethernet links that have a link state of up, the one with the alphabetically smallest link name.

If none is up, the datalink named net0 is used if it exists.

If none of the above can be satisfied, the zone will fail to boot.

mac-address

Set the VNIC”s MAC address based on the specified value or keyword. If the value is not a keyword, it is interpreted as a unicast MAC address. The supported keywords are:

factory: Assign a factory MAC address to the VNIC. When a factory MAC address is requested, the mac-slot property can be used to specify the MAC address slot identifier. Otherwise, the next available factory MAC address will be used.

random: Assign a random MAC address to the VNIC. Use the mac-prefix property to specify a prefix. Otherwise, a default prefix consisting of a valid IEEE OUI with the local bit set will be used. This is the default value.

auto: Try to use a factory MAC address first. If none is available, assign a random MAC address.

If a random MAC address is selected, then the address generated will be preserved across zone boots and zone detach/attach. This will allow zones to retain their DHCP leases by maintaining stable client IDs, and otherwise take advantage of other benefits of having stable MAC addresses.

4.Configure resources and limits:
With Solaris Zones we can control the license fee for Oracle Database. In other words we can limit the CPU number per zone.

zonecfg:zora> add dedicated-cpu
zonecfg:zora:dedicated-cpu> set ncpus=2
zonecfg:zora:dedicated-cpu> end

Keep in mind that the zone will not boot if it’s unable to allocate exactly 2 cpu cores.

Of course we can control the memory too:

zonecfg:zora> add capped-memory
zonecfg:zora:capped-memory> set physical=4g
zonecfg:zora:capped-memory> set locked=4g
zonecfg:zora:capped-memory> set swap=4g
zonecfg:zora:capped-memory> end
zonecfg:zora>

locked means max memory locked /not be flushed on the disk/swap – aka lock_sga/

Additionally we have some RAC/DB requirements:

zonecfg:zora> set limitpriv="default,proc_priocntl,proc_clock_highres,sys_time"
zonecfg:zora>

5.Verify and commit:
As simple as:

zonecfg:zora> verify;
zonecfg:zora> commit;
zonecfg:zora> exit;

Putting all together:
Instead of writing step by step, we can create a command file and use it for the zone creation. It can be useful for scripting and automation.

root@Kovachev-Pro:~# cat zora.cfg

create -b
set zonepath=/zones/zora
add device
set match=/dev/rdsk/c3t1d0s0
set allow-raw-io=true
end
add device
set match=/dev/zvol/rdsk/ssd2/tst
set allow-raw-io=true
end
add net
set physical=ne1
end
add anet
set lower-link=net0
set mac-address=auto
end
add dedicated-cpu
set ncpus=2
end
add capped-memory
set physical=4g
set locked=4g
set swap=4g
end
set limitpriv="default,proc_priocntl,proc_clock_highres,sys_time"
verify;
commit;
exit;
root@Kovachev-Pro:~#
root@Kovachev-Pro:~# zonecfg -z zora -f zora.cfg
root@Kovachev-Pro:~#

ZONE INSTALLATION

straight process:
zoneadm -z zora install
zoneadm -z zora ready
zoneadm -z zora boot

and our zone is running:

root@Kovachev-Pro:~# zoneadm -z zora install
The following ZFS file system(s) have been created:
    rpool/zones/zora
Progress being logged to /var/log/zones/zoneadm.20140423T202210Z.zora.install
       Image: Preparing at /zones/zora/root.

 AI Manifest: /tmp/manifest.xml.UjaqIn
  SC Profile: /usr/share/auto_install/sc_profiles/enable_sci.xml
    Zonename: zora
Installation: Starting ...

              Creating IPS image
Startup linked: 1/1 done
              Installing packages from:
                  solaris
                      origin:  http://pkg.oracle.com/solaris/release/
DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            183/183   33556/33556  222.2/222.2 12.9M/s

PHASE                                          ITEMS
Installing new actions                   46825/46825
Updating package state database                 Done
Updating image state                            Done
Creating fast lookup database                   Done
Installation: Succeeded

        Note: Man pages can be obtained by installing pkg:/system/manual

 done.

        Done: Installation completed in 68.525 seconds.


  Next Steps: Boot the zone, then log into the zone console (zlogin -C)

              to complete the configuration process.

Log saved in non-global zone as /zones/zora/root/var/log/zones/zoneadm.20140423T202210Z.zora.install
root@Kovachev-Pro:~#
root@Kovachev-Pro:~# zoneadm -z zora ready
root@Kovachev-Pro:~# zoneadm -z zora boot
root@Kovachev-Pro:~#

68 seconds, not bad 🙂


ZONE CONFIGURATION
As you can suppose from the previous output, we can log in to the zone with zlogin -C, which will open a console connection to our zone:

root@Kovachev-Pro:~# zlogin -C zora

                           System Configuration Tool

     System Configuration Tool enables you to specify the following        
     configuration parameters for your newly-installed Oracle Solaris 11   
     system:
     - network, time zone, user and root accounts, name services

     System Configuration Tool produces an SMF profile file in
     /system/volatile/scit_profile.xml.

     How to navigate through this tool:                                    
     - Use the function keys listed at the bottom of each screen to move
       from screen to screen and to perform other operations.
     - Use the up/down arrow keys to change the selection or to move
       between input fields.
     - If your keyboard does not have function keys, or they do not        
       respond, press ESC; the legend at the bottom of the screen will
       change to show the ESC keys for navigation and other functions.




  F2_Continue  F6_Help  F9_Quit

And we can see that sysconfig is waiting us. From here we can use the standard approach for Oracle Database and RAC installation.
Keep in mind that we can use predefined sysconfig xml file which can be generated with the sysconfig command. This file is then defined at the installation and the zone is starting already configured 🙂 I’ll not go in details so I don’t lost in AI.


Someone may notice that I skip something interesting at dedicated-cpu configuration. Yes that’s true, I didn’t mention the importance parameter and the Dynamic Resource Pools configuration. The reason is that they are just too good to be presented between the rows. They deserve at least a post, which I’ll write, maybe soon or later 🙂

Kovachev

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s