Modifying the network configuration on Exadata x5-2

Modifying the network configuration on Exadata x5-2

Run the /opt/oracle.cellos/ipconf utility which is an interactive tool that will allow you to modify any and all of the network settings, including NTP or DNS. 

Check Script:

[root@exasc02-adm ~]# vi check_cellconf.sh

IPCONF_RAW_OUTPUT=$(/opt/oracle.cellos/ipconf -verify -semantic -at-runtime -check-consistency -verbose 2>/dev/nu

ll);

IPCONF_RESULT=$(echo "$IPCONF_RAW_OUTPUT" | egrep "Consistency check PASSED" | wc -l);

IPCONF_SUMMARY=$(echo "$IPCONF_RAW_OUTPUT" | tail -1);

if [ $IPCONF_RESULT = "1" ]

  then

    echo -e "SUCCESS: $IPCONF_SUMMARY"

  else

    echo -e "FAILURE: $IPCONF_SUMMARY\n"

    echo -e "`echo -e "$IPCONF_RAW_OUTPUT" | grep FAILED`"

fi;

Run Script: 

sh -x check_cellconf.sh

----

ILOM second NTP server time.cmcom.net must have valid syntax in /opt/oracle.cellos/cell.conf  : PASSED

ILOM timezone Australia/Perth must be defined in /opt/oracle.cellos/cell.conf                     : PASSED

ILOM timezone Australia/Perth must exist in /usr/share/zoneinfo                                   : PASSED

ILOM timezone Australia/Perth must match Australia/Perth in /opt/oracle.cellos/cell.conf          : PASSED

[Info]: Consistency check FAILED'

+ echo -e 'Checking DNS server on 10.63.46.20                                                                : FAILED

DNS server 10.63.46.20 exists only in /opt/oracle.cellos/cell.conf                                : FAILED

Checking DNS server on 10.63.46.32                                                                : FAILED

DNS server 10.63.46.32 exists only in /opt/oracle.cellos/cell.conf                                : FAILED

ILOM DNS server 10.771.272.40,10.222.122.40,10.771.272.41 must match 10.63.46.32 in /opt/oracle.cellos/cell.conf  : FAILED

[Info]: Consistency check FAILED'

Checking DNS server on 10.63.46.20                                                                : FAILED

DNS server 10.63.46.20 exists only in /opt/oracle.cellos/cell.conf                                : FAILED

Checking DNS server on 10.63.46.32                                                                : FAILED

DNS server 10.63.46.32 exists only in /opt/oracle.cellos/cell.conf                                : FAILED

ILOM DNS server 10.771.272.40,10.222.122.40,10.771.272.41 must match 10.63.46.32 in /opt/oracle.cellos/cell.conf  : FAILED

After fix:

[root@exasc01-adm ~]# cat /etc/resolv.conf

# 13714588 add timeout, rotate, attempts to mitigate issues with poor or misconfigured single DNS server

# timeout:n Initial timeout for a query to a nameserver. The default value is five seconds. The maximum value is 30 seconds.

# For the second and successive rounds of queries, the resolver doubles the initial timeout and is divided by the number

# of nameservers in the resolv.conf file.

options timeout:4

# attempts:n How many queries the resolver should send to each nameserver in the resolv.conf file before it stops execution.

# The default value is 2. The maximum value is 5.

options attempts:2

# rotate Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.

options rotate

# Search domain and name server

search apac.ent.cmcom.net

# Commented it out, because OUI complains about it

#domain apac.ent.cmcom.net

nameserver 10.771.272.40

nameserver 10.222.122.40

nameserver 10.771.272.41

PROBLEM ONE:

[root@exasc02-adm ~]# cat /etc/resolv.conf

# 13714588 add timeout, rotate, attempts to mitigate issues with poor or misconfigured single DNS server

# timeout:n Initial timeout for a query to a nameserver. The default value is five seconds. The maximum value is 30 seconds.

# For the second and successive rounds of queries, the resolver doubles the initial timeout and is divided by the number

# of nameservers in the resolv.conf file.

options timeout:4

# attempts:n How many queries the resolver should send to each nameserver in the resolv.conf file before it stops execution.

# The default value is 2. The maximum value is 5.

options attempts:2

# rotate Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.

options rotate

# Search domain and name server

search apac.ent.cmcom.net

# Commented it out, because OUI complains about it

#domain apac.ent.cmcom.net

nameserver 10.249.64.70

nameserver 10.251.64.70

nameserver 10.251.64.71

STEPS:

1. Check storage cell isn’t synchronizing with the other storage cells:

[root@exasc01-adm ~]# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

Do not attempt to reboot the storage cell if its disks are in SYNCING mode. If any disks are, re-run the cellcli command until they have finished.

2. Place the storage cell’s grid disks Offline and then check status [all inactive]:

[root@exasc01-adm ~]# cellcli -e alter griddisk all inactive

[root@exasc01-adm ~]# cellcli -e list griddisk

3. We have to make sure that the cell services are shutdown and that the ocrvottargetd service has been stopped before we invoke the ipconf tool.

First, shutdown the cell services:

[root@exasc01-adm ~]# cellcli -e alter cell shutdown services all

Next, stop the ocrvottargetd service (this may not exist on your storage cell)

[root@exasc01-adm ~]# service ocrvottargetd stop

[root@exasc01-adm ~]# chkconfig --list

4. Run the ipconf utility to update the NTP servers:

[root@exasc01-adm ~]# /usr/local/bin/ipconf

--->  LOG [DOWN BELOW]

restart the services and online the grid disks manually instead:

[root@exasc01-adm ~]# cellcli -e alter cell startup services all

[root@exasc01-adm ~]# cellcli -e alter griddisk all active

[root@exasc01-adm ~]# cellcli -e list griddisk

[root@exasc01-adm ~]# cellcli -e list griddisk attributes name, asmmodestatus

[root@exasc01-adm ~]# cat /etc/resolv.conf

[root@exasc01-adm ~]# ping -c3 10.222.122.40

more /etc/ntp.conf

more /opt/oracle.cellos/cell.conf

[root@exasc01-adm ~]# sh check_cellconf.sh

SUCCESS: [Info]: Consistency check PASSED

=++++++++++++++++++++++++++++++++++++++++++++++++++

LOG

2016-02-05 10:28:02 +0800 ipconf.pl: Logging started to /var/log/cellos/ipconf.log

2016-02-05 10:28:02 +0800 ipconf.pl: Command line: -nocodes

2016-02-05 10:28:04 +0800 ipconf.pl: Interface ib0 is Linked.  hca: mlx4_0

2016-02-05 10:28:04 +0800 ipconf.pl: Interface ib1 is Linked.  hca: mlx4_0

2016-02-05 10:28:04 +0800 ipconf.pl: Interface eth0 is Linked.  driver/mac: ixgbe/00:10:e0:70:46:66

2016-02-05 10:28:04 +0800 ipconf.pl: Interface eth1 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:67

2016-02-05 10:28:10 +0800 ipconf.pl: Interface eth2 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:68

2016-02-05 10:28:16 +0800 ipconf.pl: Interface eth3 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:69

Network interfaces

ib0      Linked

ib1      Linked

eth0     Linked

eth1     Unlinked

eth2     Unlinked

eth3     Unlinked

2016-02-05 10:28:22 +0800 ipconf.pl: [Warning]: Some network interface(s) are disconnected. Check cables and swicthes and retry

2016-02-05 10:28:22 +0800 ipconf.pl: Do you want to retry (y/n) [y]: n

2016-02-05 10:28:30 +0800 ipconf.pl:

2016-02-05 10:28:30 +0800 ipconf.pl: The current nameserver(s): 10.63.46.32 10.63.46.20 10.249.64.70(custom) 10.251.64.70(custom) 10.251.64.71(custom)

2016-02-05 10:28:30 +0800 ipconf.pl: Do you want to change it (y/n) [n]: y

2016-02-05 10:29:05 +0800 ipconf.pl: Nameserver: 10.771.272.40

2016-02-05 10:29:26 +0800 ipconf.pl: Add more nameservers (y/n) [n]: y

2016-02-05 10:29:29 +0800 ipconf.pl: Nameserver: 10.222.122.40

2016-02-05 10:29:30 +0800 ipconf.pl: Add more nameservers (y/n) [n]: y

2016-02-05 10:29:39 +0800 ipconf.pl: Nameserver: 10.771.272.41

2016-02-05 10:29:40 +0800 ipconf.pl: Add more nameservers (y/n) [n]: n

2016-02-05 10:29:41 +0800 ipconf.pl: The current timezone: Australia/Perth

2016-02-05 10:29:41 +0800 ipconf.pl: Do you want to change it (y/n) [n]: y

2016-02-05 10:29:44 +0800 ipconf.pl: Setting up local time...

1) Andorra

2) United Arab Emirates

3) Afghanistan

4) Antigua & Barbuda

5) Anguilla

6) Albania

7) Armenia

8) Angola

9) Antarctica

10) Argentina

11) Samoa (American)

12) Austria

13) Australia

14) Aruba

15) Aaland Islands

16) Azerbaijan

2016-02-05 10:29:44 +0800 ipconf.pl: Select country by number, [n]ext, [l]ast: 13

2016-02-05 10:29:57 +0800 ipconf.pl: Selected country: Australia (AU). Now choose a zone

1) Australia/Lord_Howe

2) Antarctica/Macquarie

3) Australia/Hobart

4) Australia/Currie

5) Australia/Melbourne

6) Australia/Sydney

7) Australia/Broken_Hill

8) Australia/Brisbane

9) Australia/Lindeman

10) Australia/Adelaide

11) Australia/Darwin

12) Australia/Perth

13) Australia/Eucla

2016-02-05 10:29:57 +0800 ipconf.pl: Select zone by number: 12

2016-02-05 10:30:02 +0800 ipconf.pl: Selected timezone: Australia/Perth

2016-02-05 10:30:02 +0800 ipconf.pl: Is this correct (y/n) [y]: y

2016-02-05 10:30:07 +0800 ipconf.pl: The current NTP server(s): ntp.cmcom.net time.cmcom.net

2016-02-05 10:30:07 +0800 ipconf.pl: Do you want to change it (y/n) [n]: n

Network interfaces

ib0      Linked     UP      172.16.110.25    255.255.252.0                   Private      exasc01-priv1.apac.ent.cmcom.net

ib1      Linked     UP      172.16.110.26    255.255.252.0                   Private      exasc01-priv2.apac.ent.cmcom.net

eth0     Linked     UP      10.131.114.23    255.255.255.0   10.131.114.1    Management   exasc01-adm.apac.ent.cmcom.net

eth1     Unlinked   UNCONF

eth2     Unlinked   UNCONF

eth3     Unlinked   UNCONF

2016-02-05 10:30:11 +0800 ipconf.pl: Select interface name to configure or press Enter to continue:

2016-02-05 10:30:20 +0800 ipconf.pl:

2016-02-05 10:30:20 +0800 ipconf.pl: Select canonical hostname from the list below

2016-02-05 10:30:20 +0800 ipconf.pl: 1: exasc01-priv1.apac.ent.cmcom.net

2016-02-05 10:30:20 +0800 ipconf.pl: 2: exasc01-priv2.apac.ent.cmcom.net

2016-02-05 10:30:20 +0800 ipconf.pl: 3: exasc01-adm.apac.ent.cmcom.net

2016-02-05 10:30:20 +0800 ipconf.pl: Canonical fully qualified domain name [3]:

2016-02-05 10:30:32 +0800 ipconf.pl:

2016-02-05 10:30:32 +0800 ipconf.pl: Select default gateway interface from the list below

2016-02-05 10:30:32 +0800 ipconf.pl: 1: eth0

2016-02-05 10:30:32 +0800 ipconf.pl: Default gateway interface [1]:

Canonical hostname: exasc01-adm.apac.ent.cmcom.net

Nameservers: 10.771.272.40 10.222.122.40 10.771.272.41

Timezone: Australia/Perth

NTP servers: ntp.cmcom.net time.cmcom.net

Default gateway device: eth0

Network interfaces

ib0      Linked     UP      172.16.110.25    255.255.252.0                   Private      exasc01-priv1.apac.ent.cmcom.net

ib1      Linked     UP      172.16.110.26    255.255.252.0                   Private      exasc01-priv2.apac.ent.cmcom.net

eth0     Linked     UP      10.131.114.23    255.255.255.0   10.131.114.1    Management   exasc01-adm.apac.ent.cmcom.net

eth1     Unlinked   UNCONF

eth2     Unlinked   UNCONF

eth3     Unlinked   UNCONF

2016-02-05 10:30:34 +0800 ipconf.pl: Is this correct (y/n) [y]:

2016-02-05 10:30:39 +0800 ipconf.pl:

2016-02-05 10:30:39 +0800 ipconf.pl: Do you want to configure basic ILOM settings (y/n) [y]:

2016-02-05 10:30:43 +0800 ipconf.pl: Loading basic configuration settings from ILOM ...

2016-02-05 10:31:03 +0800 ipconf.pl:

Basic ILOM configuration settings:

Hostname             : exasc01r.apac.ent.cmcom.net

IP Discovery         : static

IP Address           : 10.131.114.28

Netmask              : 255.255.255.0

Gateway              : 10.131.114.1

DNS servers          : 10.771.272.40,10.222.122.40,10.771.272.41

Use NTP servers      : enabled

First NTP server     : ntp.cmcom.net

Second NTP server    : time.cmcom.net

Timezone (read-only) : Australia/Perth

2016-02-05 10:31:03 +0800 ipconf.pl:

ILOM settings in cell configuration file /opt/oracle.cellos/cell.conf:

Hostname             : exasc01r.apac.ent.cmcom.net

IP Discovery         : static

IP Address           : 10.131.114.28

Netmask              : 255.255.255.0

Gateway              : 10.131.114.1

DNS servers          : 10.63.46.32

Use NTP servers      : enabled

First NTP server     : ntp.cmcom.net

Second NTP server    : time.cmcom.net

Timezone (read-only) : Australia/Perth

2016-02-05 10:31:03 +0800 ipconf.pl:

Difference detected between the current basic ILOM settings and

stored in cell configuration file /opt/oracle.cellos/cell.conf

2016-02-05 10:31:03 +0800 ipconf.pl: Select default values from the cell configuration file (y/n) [y]:

2016-02-05 10:31:24 +0800 ipconf.pl: ILOM Fully qualified hostname [exasc01r.apac.ent.cmcom.net]:

2016-02-05 10:31:30 +0800 ipconf.pl: ILOM IP discovery (static/dhcp) [static]:

2016-02-05 10:31:37 +0800 ipconf.pl: ILOM IP address [10.131.114.28]:

2016-02-05 10:31:38 +0800 ipconf.pl: ILOM Netmask [255.255.255.0]:

2016-02-05 10:31:40 +0800 ipconf.pl: ILOM Gateway or none [10.131.114.1]:

2016-02-05 10:31:43 +0800 ipconf.pl: ILOM Nameserver (multiple IPs separated by a comma) or none [10.63.46.32]: 10.771.272.40,10.222.122.40,10.771.272.41

2016-02-05 10:32:25 +0800 ipconf.pl: ILOM Use NTP Servers (enabled/disabled) [enabled]:

2016-02-05 10:32:29 +0800 ipconf.pl: ILOM First NTP server. Fully qualified hostname or ip address or none [ntp.cmcom.net]:

2016-02-05 10:32:33 +0800 ipconf.pl: ILOM Second NTP server. Fully qualified hostname or ip address or none [time.cmcom.net]:

2016-02-05 10:32:33 +0800 ipconf.pl:

Basic ILOM configuration settings:

Hostname             : exasc01r.apac.ent.cmcom.net

IP Discovery         : static

IP Address           : 10.131.114.28

Netmask              : 255.255.255.0

Gateway              : 10.131.114.1

DNS servers          : 10.771.272.40,10.222.122.40,10.771.272.41

Use NTP servers      : enabled

First NTP server     : ntp.cmcom.net

Second NTP server    : time.cmcom.net

Timezone (read-only) : Australia/Perth

2016-02-05 10:32:33 +0800 ipconf.pl: Is this correct (y/n) [y]: y

2016-02-05 10:32:36 +0800 ipconf.pl:

2016-02-05 10:32:37 +0800 ipconf.pl: [Info]: Run /opt/oracle.cellos/validations/init.d/saveconfig

root/.ssh/

root/.ssh/known_hosts

root/.ssh/authorized_keys

home/cellmonitor/.ssh/

home/cellmonitor/.ssh/authorized_keys

etc/passwd

etc/gshadow

etc/shadow

etc/group

etc/sysconfig/keyboard

lib/kbd/keymaps/i386/qwerty/us.map.gz

root/.ssh/

root/.ssh/known_hosts

root/.ssh/authorized_keys

home/cellmonitor/.ssh/

home/cellmonitor/.ssh/authorized_keys

etc/passwd

etc/gshadow

etc/shadow

etc/group

etc/sysconfig/keyboard

lib/kbd/keymaps/i386/qwerty/us.map.gz

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++