Monday, 10 September 2018

SCAN Internals

Single Client Access Name (SCAN) is a feature used in Oracle RAC environments that provides a single name for clients to access any Oracle Database running in a cluster.The main benefit is that the client’s connect information does not need to change if you add or remove nodes or databases in the cluster.
In versions prior to Oracle 11.2, an entry in the tnsnames.ora file for a n-node RAC database always referenced all
nodes in the ADDRESS_LIST section, as in the listing that follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
TEST =
(DESCRIPTION =
(ADDRESS_LIST =
(LOAD_BALANCE = ON)
(FAILOVER = ON)
(ADDRESS = (PROTOCOL = tcp)(HOST = rac1-vip.localdomain)(PORT =1521))
(ADDRESS = (PROTOCOL = tcp)(HOST = rac2-vip.localdomain)(PORT =1521))
(ADDRESS = (PROTOCOL = tcp)(HOST = rac3-vip.localdomain)(PORT =1521))
(ADDRESS = (PROTOCOL = tcp)(HOST = rac4-vip.localdomain)(PORT =1521))
)
(CONNECT_DATA =
(SERVICE_NAME = testing)
)
)
The use of a SCAN address removes this problem, instead of addressing every single node as before, the SCAN virtual IP addresses refer to the cluster. Using the SCAN, the preceding connection entry is greatly simplified. Instead of listing each node’s virtual IP address, all we need to do is enter the SCAN rac-scan.localdomain.com. Here’s the simplified version:
1
2
3
4
5
6
7
8
9
TEST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = tcp)(HOST = rac-scan.localdomain)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = testing)
)
)

SCAN
Now in the following post we’ll understand how this SCAN concept manages the connections internally, for that we first need to understand the set-up of SCAN.
SCAN configuration using DNS (Domain Name Service)
For this you need to have a DNS entry with one single name that resolves to 3 IP addresses using a round robin algorithm. The IP addresses must be on the same subnet as your default public network in the cluster (In 12c R1 SCAN would support multiple subnets).
SCAN configuration using GNS (Grid Name Service)
In this case Oracle assumes you use some form of dynamic IP assignment on your public network, so you just need to mention the SCAN name and 3 IPs will be acquired from either a DHCP service or SLAAC when using IPv6 based IP address.
Round Robin Algorithm
Round-robin on DNS level allows for a connection request load balancing across SCAN listeners floating in the cluster.  If your DNS is set up to provide round-robin access to the IPs resolved by the SCAN entry, then run the “nslookup” command at least twice to see the round-robin algorithm work. The result should be that each time, the “nslookup” would return a set of three IPs in a different order.
1) nslookup rac-scan
Server: 192.168.46.1
Address: 192.168.46.1#53
Non-authoritative answer:
Name: rac-scan.localdomain
Address: 192.168.46.30
Name: rac-scan.localdomain
Address: 192.168.46.31
Name: rac-scan.localdomain
Address: 192.168.46.32
2) nslookup rac-scan
Server: 192.168.46.1
Address: 192.168.46.1#53
Non-authoritative answer:
Name: rac-scan.localdomain
Address: 192.168.46.31
Name: rac-scan.localdomain
Address: 192.168.46.32
Name: rac-scan.localdomain
Address: 192.168.46.30
Three IP addresses are recommended considering load balancing and high availability requirements regardless of the number of servers in the cluster.
SCAN (Single Client Access Name)
The CRSd process starts orarootagent & oraagent processes,  orarootagent is responsible for starting Node VIP, SCAN VIP and oraagent process is responsible for starting SCAN Listener and Node Listener.
For each of the 3 IP addresses that the SCAN resolves to, a SCAN VIP resource is created and a SCAN Listener is created. The SCAN Listener is dependent on the SCAN VIP and the 3 SCAN VIPs (along with their associated listeners) will be dispersed across the cluster.REMOTE_LISTENER parameter is set to the SCAN by default this allows the instances to register with the SCAN Listeners as remote listeners to provide information on what services are being provided by the instance, the current load and a recommendation on how many incoming connections should be directed to the instance.
The LOCAL_LISTENER parameter is set to the node-VIP,  the node listener run out of the Oracle Grid Infrastructure home and listens on the node-VIP using the specified port (default port is 1521).
So up till now we understood, SCAN configuration, SCAN IPs, SCAN Listener & Node Listener and how the Instances provide relevant information to SCAN Listeners, now we’ll understand the Client Side Load balancing in RAC using SCAN which would further explain another major benefit of using SCAN.
Connection Load Balancing
Client Side Load Balancing
                                     Client Side Load Balancing
Client Side Load Balancing – For clients connecting using Oracle SQL*Net 11g Release 2 and above, 3 IP addresses will be received by the client by resolving the SCAN name through DNS. The client will then go through the list it receives from the DNS and try connecting through one of the IPs received. If the client receives an error, it will try the other addresses before returning an error to the user or application. This is similar to how client connection failover works in previous releases when an address list is provided in the client connection string.
Server Side Load Balancing – When a SCAN Listener receives a connection request, the SCAN Listener will check for the least loaded instance providing the requested service. It will then re-direct the connection request to the local listener on the node where the least loaded instance is running. Subsequently, the client will be given the address of the local listener. The local listener will finally create the connection to the database instance.
I hope the above information helps you in understanding the SCAN Concept, how it is configured and how it is used for client/server side connect time load balancing. Please comment below if you need more information on SCAN & Load balancing as in how to configure client/server side connect time load balancing, Transparent Application Failover concept, FAN Concept, etc.

OCR Internals

With Oracle Grid Infrastructure 11gr2, Oracle ASM and Oracle Clusterware are installed into a single home directory known as Grid Infrastructure Home.
Oracle Clusterware consists of 2 stacks:-
1) The Cluster Ready Service Stack ( CRS Stack ) :- It manages (start, stop, monitor, and failover operations) the cluster resources based on the configuration information stored in OCR for each resource.
2) The Oracle High Availability Service Stack:- It is responsible for monitoring and maintaining high availability of Oracle ASM and Clusterware itself.

The Oracle Cluster Registry (OCR) file is a key component of Oracle Clusterware. It maintains information about the high-availability components in your cluster, such as the voting file details, cluster database instance to node mapping, and CRS application resource profiles (such as services, Virtual Interconnect Protocol addresses, and so on),Above highlighted information and the part of dump is to just give you an idea about how the information is stored in OCR. In reality OCR uses a file based repository to store configuration information in a series of key-value pairs using a directory tree like structure. So now after scanning the whole dump I can say that the OCR contains information about the resources controlled by Oracle Clusterware, including the following:
RAC databases and instances
SCAN listeners and local listeners
SCAN VIPs and local VIPs
Nodes and node apps types
ASM disk groups, volumes, file systems, and instances
OCRs Automatic and Manual backups information
Now that we have understood what is inside OCR the next natural curiosity is who maintains OCR who updates all this information in OCR, the answer is CRSd Process (and CSSd only during cluster startup), so even if you modify OCR via SRVCTL, CRSCTL, EM etc they communicate via CRSd. To explain this we need to know that Oracle uses a distributed shared cache architecture during cluster management to optimize queries against the cluster repository i.e. each node maintains a copy of the OCR in memory. Only one CRSd process (designated as the master) in the cluster performs any disk read/write activity. This master CRSd process refreshes the OCR cache on all cluster nodes. Clients (SRVCTL, CRSCTL etc) communicate with the local CRSd process to access the local copy of the OCR and contact the master CRSd process via the local CRSd process for any updates on the physical OCR binary file.
Because of this concept it is said that one should not modify OCR while any node is down ( as CRSd is down for that node ), so if need arises that one has to change the configuration in OCR when any node is down then a manual repair on the stopped node should be performed.
OCR and voting disks can be stored in Oracle ASM because the ASM’s partnership and status table is replicated on multiple disks and is extended to store OCR, the OCR can tolerate the loss of the same number of disks as are in the underlying disk group and can be relocated in response to disk failures.

Saturday, 10 March 2018

ORA-28040: No matching authentication protocol

 12c Upgrade: ORA-28040: No matching authentication protocol

An application could not connect to the Oracle Database that was upgraded to 12c (12.1.0.2). Whenever it tried connecting, the database would return the error ORA-28040: No matching authentication protocol back to the application.

Bug 14575666 states:
In 12.1, the default value for the SQLNET.ALLOWED_LOGON_VERSION parameter (deprecated now) has been updated to 11. This means that database clients using pre-11g JDBC thin drivers cannot authenticate to 12.1 database servers unless the SQLNET.ALLOWED_LOGON_VERSION parameter is set to the old default of 8.

Solution:
As the SQLNET.ALLOWED_LOGON_VERSION is a deprecated parameter, although it will work but will keep flooding your ALERT LOG file, it's best to use the updated parameter(s):

Add the below line(s) in the SQLNET.ORA file on the database server:

SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8

Featured post

Restircted session due to sync filed with ora-65177

Application is unable to connect the database due to restricted session. sql> show pdbs; SQL> show con_name CON_NAME -----------------...