Showing posts with label Infiniband. Show all posts
Showing posts with label Infiniband. Show all posts

Oracle Exadata default passwords

When Exadata get deployed,  by default it configured with Oracle standard username and password. As per good practice you should always change default password once Exadata handed over to you from Oracle. Below is the list of default Users and Passwords for Exadata.

Database Server

root/welcome1
oracle/welcome1
grid/welcome1
grub/sos1Exadata

Database server ILOMs

root/welcome1

Exadata Storage Cell Server

root/welcome1
celladmin/welcome1
cellmonitor/welcome1
Exadata Storage Cell Server ILOMs

Exadata DMA Roles and Responsibility

Exadata administration can be done by single Exadata DMA (Database Machine Administration). There is no requirement of team for each domain.Only one specialist (DMA) is required to manage the Exadata.

Based on our experience we have given following list of tasks which need to be performed by DMA. We can add many tasks in this list but we have just covered most common tasks specially from Exadata perspective.





Roles and Responsibility of DMA

Hardware installation and maintenance: A database machine required periodic maintenance like firmware upgrade or replace any failure component of machine.

Network Administration: Exadata comes with CISCO ethernet switch for Management and ILOM network, infiniband swithces for cell and DB Node node connectivity so management of network is also includes in DMA's task. General knowledge of netwrok topology required to manage Infiniband switches.

Operating System Administration: Linux or Solaris OS administration. Also manage server availability during patching with high availability management. Setting up users,  applying patches, managing disk space, NFS mount, 

Exadata Storage Administration: Important component of Exadata. In depth knowledge is required to manage the cell servers as this is the main component of Exadata which also configured with Exadata Storage Software. It include monitoring of celldisk, griddisk, cell flash cache, flash disks, OS administration, infiniband connectivity  administration. Maintain storage availability during patching.

Grid Infrastructure Administration: Inlcudes clusterware, ASM disk group, mirrored cell group as Exadata managed mirroring on ASM level not cell level.

Database Administration: Includes single instance, single instance on ASM, one or more RAC node database. Exadata specific parameters.


Here DMA does not need to know full scale Linux , storage or network. Only few relevant concepts are required to manage the Exadata. 

Overall required skill for component wise in percentage as follows.



  • 60% Database Administration
  • 20% Cell Storage Administration
  • 15% Linux Administration 
  • 5% Miscellaneous (Infiniband, network, etc.)
You can follow our Exadata Certification Question Bank to achieve maximum success in your Exadata certification exam.

Cell Server Connectivity and Port Details

Following image shows back end layout of cell storage server.



Short description for each port :


ILOM -- Useful to manage the Hardware remotely

eth0 -- Use to manage the server from OS level

eth1 , eth2 & eth3 -- Not connected as there is no public network required for it.



IB0 & IB1 -- Infiniband network ports (Bonded)

KVM -- For kvm connectivity.

Database Node Connectivity and Port Details

Following image shows the back end connectivity of database server.





Short description for each port:

ILOM -- Useful to manage the Hardware remotely

eth0 -- Use to manage the server from OS level

eth1 & eth2 -- Useful for public network (Bonded)

eth3 -- This port is useful for additional network if required.

eth4 & eth5 -- 10Gig N/W port (Bonded)

IB0 & IB1 -- Infiniband network ports for storage connectivity (Bonded)

KVM -- For kvm connectivity.

Hot to Verify InfiniBand Fabric Topology ?





Verifying infiniband topology whether it is configured current or not. If it not gives the desired output as per below then it will cause the InfiniBand network to operate at degraded efficiency, intermittently, or fail to operate.




# /opt/oracle.SupportTools/ibdiagtools/verify-topology -t fattree

[ DB Machine InfiniBand Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch..........[SUCCESS]
Are any external switches connected to each other....................[SUCCESS]
Are any hosts connected to spine switch..............................[SUCCESS]
Check if all hosts have 2 CAs to different switches..................[SUCCESS]
Leaf switch check: cardinality and even distribution.................[SUCCESS]
Check if each rack has an valid internal ring........................[SUCCESS]


If anything other than "SUCCESS" is reported, investigate and correct the condition.

Exadata Networks

Management/ILOM:

This is Ethernet network which is used to manage the hardware components of Exadata machine. Using this network administrator can access the hardware through KVM or ssh to monitor the hardware components. Servers and storage contains ILOM also which is used to monitor the components remotely. All the devices contains management networks including infiniband and PDUs.


Infiniband:


This is high speed storage network between cell storage and database servers. This is also used as cluster interconnect. We can also configure high speed backup with the use of infiniband network. Each database server and cell storage connected to infiniband network with the use of bonded infiniband network to maintain redunduncy.This is the private network which does not required any network connection from client switch.
 
Client Access:

Exadata comes with both 1GbE and 10GbE network ports. Initially it configured single client access with single port (NET1) or with bonded ports (NET1 & NET2), depends upon the customer.This network will be used for public access so it will be different from the management network.As per the best practice management and client access network should not come from same network. Database servers also contains two 10GbE ports to configure 10gig network either for public or backup again depends on the customer as SFP (Interface between server and 10gig network) don't come with the Exadata set , if anyone want to use then order has to make separately.Also Customer must have required 10gig environment to use 10 gig ports.


         All the three network should be on different network to maintain the security.

Exadata Best Practices Meta link Notes

Documentation availability: 
For Exadata Customers:

Online Documentation

The following documentation can be found in the /opt/oracle/cell/doc
directory on the Oracle Exadata Storage Server (Exadata Cell):
¦ Oracle Exadata Storage Server Software User’s Guide
¦ Oracle Exadata Storage Server Software Release Notes

MAA Best Practice Papers for Exadata:

http://www.oracle.com/technetwork/database/features/availability/exadata-maa-best-practices-155385.html

Useful URL's and links

Latest Backup & Recovery Whitepaper:
Oracle Exadata Database Machine - Backup & Recovery Sizing: Tape Backups

http://www.oracle.com/technetwork/database/focus-areas/availability/maa-exadata-backup-methodology-495297.pdf

Exadata Whitepaper:

http://www.oracle.com/technetwork/database/exadata/exadata-technical-whitepaper-134575.pdf

Exadata and Database Machine Version 2 Demo Series

https://apex.oracle.com/pls/apex/f?p=44785:24:1352200682703903::::P24_CONTENT_ID,P24_PREV_PAGE:5118,24

Some important My Oracle Support Notes(Metalink):

Master Note for Oracle Database Machine and Exadata Storage Server
1187674.1

Patches & Updates, InfiniBand Firmware Updates
888828.1

Exadata Patching Overview
1262380.1

Oracle Sun Database Machine Application Best Practices for Data Warehousing
1094934.1

Best Practices for OLTP on the Sun Oracle Database Machine
1269706.1

DBFS:

11911441.1          Configuring Database for DBFS on Oracle Database Machine
1054431.1            Configuring DBFS on Oracle Database Machine

Monitoring Best Practices
1110675.1

ASM Diskgroup Imbalance Check
367445.1

Data Guard Related
1085687.1            Heterogeneous Data Guard Support

Infiniband RDS Support
745616.1

You can follow our Exadata Certification Question Bank to achieve maximum success in your Exadata certification exam.

X2-2 Exadata Machine Overview

Exadata Machine

-Database Servers, Storage and Network in a Single Enclosure
-High Bandwidth Network Connects Database and Storage Servers
-Intelligent Storage Filters Data and Minimizes Network Traffic
-Optimized for Both Data Warehouse and OLTP Activity
-Available in two variants, X2-2 and X2-8
-Available Full, half and Quarter Racks

Machine Configuration 

-8 Intel-based Dual-processor Database Servers (Sun Fire X4170 M2)
-96 cores (12 per server)
-768 GB memory (96GB per server)
-10 GigE connectivity to Data Center
-16 x 10GbE ports (2 per server)
-14 Exadata Storage Servers X2-2 All with High Performance 600GB SAS disks OR All with High Capacity 2 TB SAS disks
-3 Sun Datacenter InfiniBand Switch 36
-36-port Managed QDR (40Gb/s) switch
-1 “Admin” Cisco Ethernet switch
-Keyboard, Video, Mouse (KVM) hardware
-Redundant Power Distributions Units (PDUs)


Grid 


Flash