“This presentation is for informational purposes only and may not be incorporated into a contract or agreement.”
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Ara Shakian Principal Product Manager Oracle Server Technologies
Take the Guesswork out of Database I/O Tuning An Oracle Open World 2005 Presentation
Agenda • Storage Management Challenges • Application of S.A.M.E. Methodology • Optimizing I/O for Bandwidth, Access and Workload types
• • • •
Automatic Storage Management Overview Striping and Mirroring Discussions ASM Best Practices Conclusion
Storage Management Gap Avg. Managed Capacity per Administrator In Database Sites with >100 GB Capacity 60 50 40
Mgmt Gap
30
Capacity /DB Adm
20 10
Strategic Research Corporation
05 20
04 20
03 20
02 20
01 20
00 20
99 19
98 19
97
0
19
Capacity in Terabytes
Disk Capacity
Capacity /DB Admin With ASM (estimated)
What Guesswork? • Configuring storage for a DB depends on too many variables • Which data to put on which disk • LUN configurations • DB application workloads • OLTP, DSS, Batch vs. On-Line • Tradeoffs between available options
• Did I choose the right configuration? • On-going tuning
Typical Day for a DBA and System Admin • Design DB layout strategy • How many volumes, data files • Where data files reside • Inventory of files: • 2 control files, 2 log files, multiple datafiles per tablespace, backup files, temp files, etc… • 100’s or 1000’s of files to create, name, manage and not make mistakes! • Multiplied by n number of Databases • What about tuning, expanding/contracting your DB ?
Whiteboard Management
The Key Question How should I configure and tune my Oracle Database 10g storage environment? Conventional answer is: - Workloads - Storage configurations - RAC vs. single Instance - FS/VM or RAW
It Depends
Conventional FS/VM/RAW choices: - Too complicated - Too slow - Too Expensive
• Right answer is: Automatic Storage Management (ASM)
ASM is the Implementation of Oracle S.A.M.E. Methodology • The goal of S.A.M.E. is to optimize storage I/O utilization • Bandwidth • Sequential and random access • Workload types
Optimizing Sequential Access • The key to efficient sequential IO is to make positioning time small relative to transfer time • Positioning time is overhead • Make: Transfer time >= 5 * Positioning time • Implies disk access must be at least 1MB in size IO
S iz e
P o s itio n in g T im e
X fe r T im e
% T im e X fe r in g
6 4 K
1 0 m s
3 m s
2 3 %
2 5 6 K
1 0 m s
1 2 m s
5 5 %
1 M
1 0 m s
5 0 m s
8 3 %
2 M
1 0 m s
1 0 0 m s
9 1 %
* Oracle S.A.M.E. research data, 1999
Optimizing Transfer Time Create LUNs using ‘outside of disk’ • Disks hold more data towards outside edges than towards center • Transfer rate is faster for data near outside
Position on Disk (measured in capacity) Outer Edge
Tranfer Rate
22 MB/sec
Outer Quarter
21 MB/sec
Mid Point
19 MB/sec
Inner Quarter
16 MB/sec
Inner Edge
11 MB/sec
Flash Recovery Area (FRA)
* Oracle S.A.M.E. research data, 1999
Optimizing Random Access Key to optimizing random access is to limit length of seek • Place hot data on outside of disk to utilize increased density • Limiting access to outside half of disk gets most of seek benefit
* Oracle S.A.M.E. research data, 1999
% of Disk Avg Seek Used in Dist. distance from outer edge
% Disk Space Avg Read Used Time (seek + rotation)
25%
8%
33%
4.8ms
50%
17%
60%
5.7ms
75%
25%
85%
6.5ms
100%
33%
100%
7.3ms
Summary of Lessons Learned • Use ASM to stripe data at 1MB extents • Equalize the workload across disks and eliminate hot spots • Maximize disk drives in ASM • Optimum sequential bandwidth gained thru 1MB I/O
• Place frequently used data at outer half of disk • Fastest transfer rate • Minimize seek time
ASM
Law of Physics
How Do I Design for different Workload Types? Conventional answer is:
It Depends
• Right answer is: You don’t have to Oracle Database 10g & Automatic Storage Management (ASM)
Don’t Worry about Workloads Oracle DB and ASM are designed to reduce the complexity • Complexity stems from • Application types • OLTP, DW, batch, and mixed-workloads • Many operation types • Scan, lookup, load, insert, create index, join, LOB, sort, backup, recovery, batch write, etc. • Many file types • Data, log, archive, temp, undo, system, control, backup, etc.
Oracle DB Designed to Perform Large I/O When Possible Oracle performs 1MB I/O when possible • DBWR (the main server process) – Coalesces adjacent buffers to max size of 1MB • Kernel sequential file I/O – buffer size is set to 1MB by default • Archiver writes 1MB I/Os • Log Writer – Max I/O size is set to 1MB
Oracle Parallelizes I/O at the Highest Level • Sequential I/O is optimized by Oracle • Scans, loads, backups, log writes, sorts, etc. • Sequential operations issue large I/O, Random operations issue small I/O • Read-ahead for sequential I/O operations
• Oracle can parallelize many operations • Scan, sorts, joins, hash, load, create index, etc.
Workload Summary Database storage management does not have to be complex any more • Oracle Database 10g and ASM are designed to hide the complexity for all workload types • Large (1MB) I/O to optimize sequential performance • Parallelizes many operations • Distribute data evenly and widely for maximum performance
Automatic Storage Management A storage manager designed to manage Oracle Database 10g database files – offered at no additional cost • Volume Manager • File System • Clustering capabilities Reduce Cost and Complexity Without Compromising Performance or Availability Simplifies and Automates Database Storage Management
Increases Storage Utilization and Agility
Predictably Delivers on Performance & Availability SLA’s
ASM Key Features
ASM Disk Group LUN
LUN
LUN
1. Volume Management with 1MB or 128KB striping 2. Database File System with performance of RAW I/O 3. Supports clustering and single instance 4. Dynamic data distribution and rebalancing 5. Automatic file management 6. Flexible mirror protection
Let’s Talk About Storage Configurations & Best Practices
Migrating from 9i to 10g? Choice for Storage Management? 3 rd
Party
AS
M W A R
S CF O CF S VM
• ASM is Oracle’s strategic direction • Use ASM for database files • Highest value for Managing Oracle DB files
• Use OCFS for Oracle binaries (where available)
Which RAID Configuration for Best Performance and Availability? A. ASM mirroring B. Hardware RAID 1 (mirroring) C. Hardware RAID 5 (Parity Protection) D. Both ASM mirroring & hardware RAID Answer: Depends on business requirement and budget (cost, availability, performance & utilization) ASM leverages hardware RAID
Should I Use RAID 1 or RAID 5 RAID 1 (Mirroring)
RAID 5 (Parity)
- Recommended by Oracle - Most demanding applications
-Your mileage will vary - DSS and moderate OLTP
PRO • Best redundancy • Best performance • Low recovery overhead CON • Requires higher capacity
PRO • Requires less capacity CON • Less redundancy • Less performance • High recovery overhead
ASM Mirroring Protection • Best choice for low cost storage • Mirror across storage subsystems • Automatic recovery from block corruption ASM Disk Group
To Stripe or Not to Stripe: What Type of Striping Works Best? A. ASM only striping (no RAID) B. Hardware RAID 0 & ASM striping C. Use LVM D. No striping Answer: A & B ASM & RAID striping are complimentary!
ASM Striping Only Oracle DB size: 1TB (1TB Data File, 2TB Flash Recovery Area) Storage Configuration: 8 x Trays with 12 x 73GB disks per array (Total capacity: 7TB)
PRO • Drives evenly distributed for Data & FRA • Higher bandwidth • Allows small incremental growth (73GB) • No drive contention
Data Diskgroup (1TB) LUN 1
73GB LUN
CON • Not well balanced across ALL disks • LUN size limited to disk size
FRA Diskgroup (2TB) 16
LUN 17
48
73GB LUN
RAID 1
Hardware RAID Striped LUNs Oracle DB size: 1TB (1TB Data File, 2TB Flash Recovery Area) Storage Configuration: 8 x Trays with 12 x 73GB disks per array (Total capacity: 7TB)
PRO • Fastest region for Data DG • Balanced data distribution • Fewer LUNs to manage while max spindles
Data Diskgroup (1TB) LUN 1
250GB LUNs
2
3
4
CON • Large incremental growth • Data & FRA (friendly) contention
FRA Diskgroup (2TB) LUN 5
6
7
500GB LUNS
8
RAID 0+1
Customer Benchmark: Vanderbilt University
200-300% Increase In Performance!
Database Environment: • 9iRAC vs. 10gRAC Storage:
9i RAC/LVM/RAW vs. 10gR2 RAC/ASM 60:00:00
9i RAC and Legacy LVM Elapsed Time
EMC Symetrix- 16 drives 4 LUNs, each 4-way striped (RAID0) 16 LUNs, LUN=disk drive ( no RAID0)
72:00:00
48:00:00
10g R2 RAC+ASM 9i/LVM 36:00:00
ASM ASM+RAID0
10g R2 RAC ASM+RAID0
24:00:00
Benchmark Environment: Sqlbench – Most intensive sql queries Indexbench – collection of drops & re-builts in parallel
12:00:00
0:00:00 IndexBench1
IndexBench2
IndexBench3
IndexBench4
SQLBEnch1
SQLBEnch2
Benchmark type
SQLBEnch3
SQLBEnch4
RAID 5 (7+1) or RAID 1 LUNs
Linear scaling from 200MB/sec to 1600/MB/sec • Saturates FC HBAs – a good thing • RAID 5 (7+1) DS8000 IBM storage with 16 LUNs • Modified tcp-h DSS query (read only) • 2/4/6/8 nodes (AIX LPARS) • LUN configuration as shown
Database 1 Data DG
Database 2 FRA DG
Data DG
LUN 1
Data1 FRA1 Data2 FRA2
FRA DG
LUN 16
…….
How Many Disk Groups per Database? Data Disk Group Data, redo log, control file types
Flash Recovery Area Disk Group Archive log, backup, temp files types
• 2 Disk Groups are recommended • Backup for each other • Leverage maximum of disk per disk group • Exceptions • Additional Disk Groups for different capacity or performance characteristics • Different storage tiers
Should I Consolidate Multiple DBs in a Disk Group? LAN
RAC ASM
Data DG
A S M
10g ASM
FRA DG ERP Database
10g ASM
• •
•
YES Shared storage across several databases • RAC and Single Instance Benefits: • Higher storage utilization
• Lower cost
CRM Database HR Database
•
Higher performance
• More LUNs/DG •
Simplified and Centralized management
It’s Real Simple! You Don’t have to Guess Anymore • Use Oracle Database 10g ASM for volume and file management • External RAID protection when possible • Create LUNs using • Outside half of disk drives for highest performance • Small disk, high rpm (I.e. 73GB/15k rpm)
• Maximize number of spindles in your disk group
Oracle Database 10g and ASM do the rest!
Hundreds of ASM Customers In Production Amazon.com Advance America
Just Remember ASM is the realization of the S.A.M.E. methodology plus more… • • • • •
ASM automates to reduce human errors Simplifies configuration choices Eliminates workload type dependencies Optimizes IO bandwidth Reduces cost
• And, washes the dishes for you too!
ASM Featured in OOW 2005 • Exhibit Demos • Oracle and partner booths
• ASM specific sessions • 5 ASM specific presentations • 20+ RAC & ASM presentations
• X-treme Lab RAC, ASM & Grid Control
ASM Featured in OOW 2005 ID
Time
Room
Title
S1973
Tuesday 11:00 AM
304 – South
Amazon: Using Oracle Real Application Clusters and Oracle Database 10g Automatic Storage Management to Enable Scaling on Linux with LowCost Storage
S943
Tuesday 11:00 AM
104 – South
Database Storage Consolidation - 10g Release 2 Automatic Storage Management update
S027
Tuesday 3:00 PM
305 – South
Oracle 10g Release 2 Customer Panel: Experiences with RAC and ASM
S1955
Thursday 9:00 AM
305 – South
Oracle Real Application Clusters & Automatic Storage Management: Best Practices
S1949
Thursday 10:30 AM
103 – South
Take the Guesswork Out of Database I/O Tuning
ASM OTN Web Site: http://www.oracle.com/technology/products/database/asm/index.html
Take the Guesswork out of Database I/O Tuning An Oracle Open World 2005 Presentation