Sql Server 2008 Database Administration
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Sql Server 2008 Database Administration as PDF for free.
More details
- Words: 57,025
- Pages: 1,545
The History of SQL Server
Presented by Scott Whigham
It All Began in the 80s
• In 1989, Microsoft (“MSFT”) was a $200 million a year company with no database product • Ashton-Tate had the popular dBASE package • Sybase had DataServer, a new product
It All Began in the 80s
• MSFT signed deal to license Sybase’s DataServer on OS/2 and other MSFTbased operating systems • MSFT convinced Ashton-Tate to help them market the product • Ashton-Tate/Microsoft SQL Server – Version 1.0 shipping in May, 1989
It All Began in the 80s
• Microsoft SQL Server 1.1 shipped in 1990 – No more Ashton/Tate – Now for Windows 3.0
The 1990s
• Microsoft SQL Server 4.2 release timeline – There was no 2.x or 3.x – 16-bit originally for OS/2 released in 1992 – Windows NT 3.1 32-bit release in 1993
The 1990s
• Original license with Sybase was restrictive – MSFT could not add or change features without Sybase’s approval – Sybase and MSFT parted ways in 1994
The 1990s
• Next up: SQL95 – SQL95 was released in Spring, 1995 • Technically Microsoft SQL Server 6.0
– Replication is here! – MSSQL was now one of The Big Guys in the database world • Sybase, Informix, Oracle, and IBM still strong
The 1990s
• SQL Server 6.5 in 1996 – Short release cycle due to pressure from other companies – Can you say ”ANSI Standard Compliant”? – Still built on the “old” Sybase code
The 1990s
• SQL Server 7.0 in 1998 – Total rewrite – no more Sybase code! – Language is still Transact-SQL but it starts to fork – Tons of new features • Debut of Data Transformation Services • Row-Level locking • OLAP Services debuts
The Millenium
• SQL Server 2000 in 2000 – Originally planned as SQL Server 7.5 – Not that many new features but far more powerful – Built to work with Windows Server 2000 – OLAP Services becomes SQL Server Analysis Services – Reporting Services debuts a few years later as a ”free” add-on
The Millenium
• SQL Server 2005 in 2005 – Delays caused by integration with .NET team – Made to work with Windows Server 2003 – Many new features: • • • • • •
SQL CLR Integration Services Full integration of Reporting Services Database Mirroring Tighter XML integration Schemas!
The Millenium
• SQL Server 2008 in 2008 – Not a huge step but built for Windows Server 2008 • This allows for more memory, better 64-bit support
– Some new features but primarily enhancements • • • • •
Transparent data encryption Management policies Geo-spatial data and datatypes Filestream Backup compression
The Millennium
• SQL Server .NEXT – What’s next? – Business Intelligence (“BI”) continues to grow – Storing images is hot – Will we ever see an O.S. built on SQL Server?
Editions of SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Two Main Pain Points
• Microsoft SQL Server® 2008 comes in two flavors: – Server Editions • Enterprise, Standard
– Specialized Editions • Express, Developer, Workgroup, Web, Compact
Processor Architecture
• Microsoft SQL Server® 2008 is further segmented by processor architecture: – x86 • All
– x64 • All except Compact
– IA64 • Enterprise and Developer only
Server Editions • Enterprise (x86, x64, IA64) • Standard (x86, x64)
Specialized Editions • Developer (x86, x64, IA64) • Workgroup (x86, x64) • Web (x86, x64) • Express (x86, x64) • Compact (x86)
Compact Edition
• SQL Server 2008 Compact Edition – Compact is meant to run on phones/portable devices – Prior names include “SQL Server for Windows CE” and “SQL Server Mobile Edition”
• Targets the .NET Framework 3.5 • Free! • Not discussed further in this videobook
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Server Editions
• The Server Editions target companies running Windows Server who need: – Large databases – Lots of users – Scalability and availability
• Enterprise and Standard Editions are also the most expensive!
Enterprise Edition
• Enterprise Edition is the most powerful edition available – No hard limit for CPUs or Memory (OS maximum) • This allows for VLDBs (Very Large Databases)
– Expensive to buy – Used for mission-critical databases and data warehouses where uptime is paramount (even across distances)
Enterprise Edition
• Enterprise Edition is: – Expensive to buy • Averages around 5x Standard Edition cost
– Used for mission-critical databases and data warehouses where uptime is paramount (even across distances)
Standard Edition
• Standard Edition is the next most powerful edition – 4 CPU limit but no Memory limit (OS maximum) • This allows for KLDBs (Kinda Large Databases) – Okay, so that’s not technically an industry term…
– Used when needs are similar to Enterprise Edition but budget isn’t available • Even with just 4 CPUs you can potentially handle 100s of GBs of data
Standard Edition
• Standard Edition is the next most powerful edition – 4 CPU limit but no Memory limit (OS maximum) • This allows for KLDBs (Kinda Large Databases) – Okay, so that’s not technically an industry term…
– Used when needs are similar to Enterprise Edition but budget isn’t available • Even with just 4 CPUs you can potentially handle 100s of GBs of data
Comparing the Server Editions
Enterprise Edition Business Critical Large scale online transaction processing (OLTP) Large scale reporting Data warehousing
Advanced analytics Server consolidation Departmental Small-to-medium scale OLTP Reporting and analytics
Standard Edition
Comparing the Server Editions
Enterprise Edition
Standard Edition
O.S. maximum
4
50
16
Number of clustered nodes
16
2
Database mirroring
Full
Single-thread
Number of CPUs Number of Instances Hypervisor support Database snapshots Parallelism, index, and DBCC enhancements
Hot-add CPU, memory Fast recovery
In the Next Video…
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Editions of SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Server Editions • Enterprise (x86, x64, IA64) • Standard (x86, x64)
Specialized Editions
• Developer (x86, x64, IA64) • Workgroup (x86, x64) • Web (x86, x64) • Express (x86, x64) • Compact (x86)
Specialized Editions
• The Specialized Editions are for specific situations – Developer Edition: for testing and development • Same feature-set as Enterprise Edition but different license, O.S. & hardware requirements/limits
– Workgroup: for branches, small depts • Great for replication to remote sites
Specialized Editions
• Web: ideal for web hosts – Licensing allows for low Total Cost of Ownership (TCO)
• Compact: for mobile devices – Usually sync up with Enterprise/Standard to upload/download new data
Specialized Editions
• Express: free; great for learning or redistributing – Perfect for very small databases with small numbers of users – The most confusing edition since there are three “versions” of Express
The Express Editions
• There are three “flavors” of Express: – SQL Server Express • The SQL Server database but without tools, reporting
– SQL Server Express with Tools • Same as above but with feature-limited versions of management and development tools
– SQL Server Express with Advanced Services • Same as SQL Server Express with Tools but with reporting, more admin
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Comparing the Editions
Enterprise/ Developer
Standard
Workgroup
Web
Express
O.S. maximum
4
2
4
1
50
16
16
O.S. maximum
O.S. maximum
4GB
O.S. maximum
1GB
Number of clustered nodes
16
2
Database mirroring
Full
Single-thread
Witness only
Witness only
Witness only
Replication Support
Full
Full
Subscriber only
Subscriber only
Subscriber only
Number of CPUs
Number of Instances Memory Database snapshots Log shipping
Data Compression Transparent Data Encryption
Pricing and Licensing SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Two Ways to License
• When you purchase Microsoft SQL Server® 2008, you purchase: – A license to run SQL Server on the server and one of the following: • CPU license • Client Access License (“CAL”) for a user • Client Access License (“CAL”) for a device
CAL Licensing
• CAL Licensing Model Explained – User model: Requires single CAL “… for each user who accesses a system running SQL Server.” • One user can use the same CAL to access different SQL Servers separately • Source: “Database Pricing” whitepaper, MSFT 2008
CAL Licensing
• CAL Licensing Model Explained – Device model: Requires single CAL “… for each device that accesses a system running SQL Server.” • Devices can be PC, workstations, terminal, PDA, mobile phone, etc
CAL Licensing
• User model or Device model? – First question: Should you use CPU licensing instead? – If you cannot/should not use CPU licensing: • Use device model when there are multiple users per device • Use user model when there are multiple devices per user • When in doubt, consider user model
CAL Licensing
• Example scenario: Your CRM application has 50 users worldwide but only 10 use at the same time all using their individual workstations – Need a minimum of 10 user CALs
CRM: Customer Relationship Management
CPU Licensing
• CPU Licensing Model Explained – Unlimited number of connections, users for a single CPU – License only needed for physical CPU • A server with one quad-core CPU would require only a single CPU license
How MSFT Determines Usage
• Consider this scenario: – DBA installs SQL Server Enterprise Edition on web server – One web application makes a single connection to SQL Server – Web application is used by 500 people worldwide
• How many licenses do you need?
How MSFT Determines Usage
• Answer: “Not enough information to answer!” – Many people think the answer is “One CAL” since there is only one app accessing the database server! – You need to ask a follow-up question • What is the follow-up question you should ask?
How MSFT Determines Usage
• “How many of those 500 users will be using the system at the same time (concurrently)?” – Answer: 10 • Then you need 10 user CALs
– Answer: 100 • Then you need 100 CALs or 1 CPU license
CPU or CAL?
• At some point it makes sense to switch from CAL to CPU – 10 CALs or 1 CPU? – Depends on cost of each – Likely to be somewhere between 60 and 90 concurrent users that you save money by using CPU – Don’t forget to add in growth
In the Next Video
• Licensing Model • Virtualization
• Cost per edition
Pricing and Licensing SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Virtualization
• What about licensing when you install SQL Server inside VMware or another virtualized environment? – Same basic rules – For Enterprise Edition, if all processors are licensed, customers can run “unlimited instances of SQL Server on an unlimited number of virtual operating environments on that same machine” • Source: “Database Pricing” whitepaper, MSFT 2008
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Costs (Retail only)
CPU License
CAL
Enterprise
Standard
Workgroup
Web
$24,999
$5,999
$3,899
$15/CPU/month
$8,487 for server license plus $162 per CAL
$885 for server license plus $162 per CAL
$730 for server license plus $162 per CAL
Not available
Developer Edition: $50 total cost
Prices are in U.S. dollars Source: http://www.microsoft.com/sqlserver/2008/en/us/pricing.aspx, Nov, 2008
What’s New in SQL Server 2008
Presented by Scott Whigham
What to Expect
• This is an overview, not a detailed discussion • Expect to see “Here’s what it is and here’s 1 or 2 sentences about it” • Many if not all will be covered later in detail • Not all new features are covered
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
What We’re Going to Cover
• SSIS • SSRS • SSAS
SQL 2008 Is About…
• SQL 2008 is likely to be interesting to most people because of it’s ability to scale rather than because of new features – While there are lots and lots of new features, only a few of them are major new features
SQL 2008 Is About…
• Many of the new features target “5%ers” – Geospatial support is one such example
• Some of the new features are simply “better ways” – Geospatial support and FILESTREAM are examples
SQL 2008 Is About…
• If you were creating a first-time app for storing/working with geospatial data, SQL Server 2008 would be ideal – What if your app already uses a technique for storing/working with geospatial data that was different from the way SQL Server 2008’s internal datatypes and use?
SQL 2008 Is About…
• A SQL Server 2005 DBA can administer a SQL 2008 box without learning anything new – Obviously wouldn’t be aware of new features and enhancements but it is very possible (and likely) – SQL Server Management Studio (“SSMS”) has been augmented but not drastically changed
SQL 2008 Is About…
• SQL Server 2008 new features further segment “The Talent” – It’s not possible for one person to know it all • This was possible as recently as SQL Server 7.0 or even SQL Server 2000
SQL Server 2005 Changed Everything
• In SQL Server 2005, The Business Intelligence side of SQL Server matured thus adding (needed) complexity to existing products – SQL Server – Integration Services (“SSIS”) – Reporting Services (“SSRS”) – Analysis Services (“SSAS”)
Changing Roles
• Today, it’s easier to be a “Specialist” in one area and a “Generalist” or even a “Deep Generalist” in others – You might specialize in SQL Server DBA but have deep, general knowledge of SSIS, a general knowledge of SSRS and scant knowledge of SSAS
SQL Server 2008
• In SQL Server 2008, the DBA must work harder than ever to be a Specialist – Fairly easy to be a generalist – Tough to be a deep generalist – Very hard to be a specialist • What size databases do you specialize in? • At your last job, did you implement encryption? • Did you do database mirroring and use snapshots at your last job? • How many active/active clusters have you built? • What’s the best way to do document/file storage?
Why is this discussion here?
• Why have this discussion now? – Don’t get intimidated into thinking you have to know it all – Don’t be afraid to say, “That’s not my specialty” – Pick the areas you like and will need to continue employment and master those
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Performance Improvements
• Compression Enhancements: – Backup compression • Are your backups taking up too much space?
– Log Stream compression • Are you using database mirroring? If so, log stream compression can offer significantly better performance
– Data compression • Do you have a lot of data that is similar or bloated? • Easy to use: BACKUP DATABASE … WITH COMPRESSION
Performance Improvements
• Enhancements and improvements for databases of all sizes – Enhanced/Improved Plan Guide Support • Freezing plan guides, pull plans directly from plan cache, SSMS integration
Performance Improvements
• Enhancements and improvements for databases of all sizes – Resource Governor • Allows you to manage workload and resources by setting limits to incoming requests • Can limit certain users/groups/roles to fewer resources
Performance Improvements
• Enhancements and improvements for databases of all sizes – Performance Data Collection • Includes a central repository for storing performance data as well as tools and reports
Performance Improvements
• Enhancements and improvements for larger databases: – Partition-aligned indexed views • Are you using indexed views or partitions already? If so, this is great. If not, probably unnecessary for you at the moment
– Partitioned Table Parallelism • In SQL Server 2005, each partition was limited to a single thread
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Security Improvements
• Lots of new features that are potentially “better ways” – Transparent Data Encryption • Extends SQL Server 2005 encryption • Easier to turn on/off encryption for the entire database without having to modify calling applications
Security Improvements
• Lots of new features that are potentially “better ways” – Extensible Key Management (EKM) • Allows the keys that protect the database files to be stored externally
Security Improvements
• Some are just changes… – New database roles • MSDB database: – db_dts… roles have been renamed to be db_ssis… – Server group roles: ServerGroupAdministratorRole, ServerGroupReaderRole – Policy-Based Management role: PolicyAdministratorRole – Data Collector: dc_admin, dc_operator, dc_proxy
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Management Improvements
• New Feature: – Policy-Based Management • DBAs can set policies that cover everything from naming conventions to CLT integration • One of the most exciting new features for DBAs
Management Improvements
• New Feature: – SQL Server Audit • Allows customizing what you want audited • Allows for auditing at server and database level – – – –
SELECT, INSERT, UPDATE, DELETE, EXEC DBCC Login/logout attempts State changes
• Scripts can be generated from audits
Management Improvements
• New Feature: – Change Data Capture (CDC) • Allows you to record changes to data into another table without triggers – INSERT, UPDATE and DELETE
• Perfect for applications that update external databases • Example: you need to synchronize two 200GB databases nightly in a one-hour window
Management Improvements
• New Feature: – Data Profiling Task • New SSIS Task allows DBAs to profile for things like: – – – –
Number of rows Number of unique values Percentage of NULL values in column Duplicate values in potentially key columns
Management Improvements
• New Feature: – Extended Events (Xevents) • Capture, filter, and act upon events generated both by SQL Server, the operating system, and other applications • Can trace events at any level • Can choose aggregated data or event details
Management Improvements
• Enhancements – Central Management Servers & Server Groups • DBAs can now designate one SQL Server a Central Management Server and create a Server Group – – – –
Queries can be executed on the servers in the server group Policies can be evaluated against the group Changes made in Object Explorer can be propagated Windows authentication required
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Availability Enhancements
• Hot pluggable CPUs • Automatic recovery of suspect pages • Add nodes to peer-to-peer replication without stopping system activity • Enhanced database mirroring
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
T-SQL Improvements
• Intellisense is finally here! • Table-valued parameters are here! – Goofy syntax and requirements but you can now pass table results to stored procedures • Not to user-defined functions though
T-SQL Improvements
• Compound operators are finally here! SET @i += 1
• Declaration and assignment in one statement is finally here! DECLARE @i INT = 1
T-SQL Improvements
• XML enhancements – Better and easier schema validation – Better XQuery support • Use of “let” clause to assign values
– Full xs:dateTime support
T-SQL Improvements
• FILESTREAM – Allows DBAs and Developers to store documents as part of the operating system yet to be able to access those as part of the database • COMMIT/ROLLBACK aware • Uses new VARBINARY(MAX) FILESTREAM storage type • Users can use regular SELECT, INSERT, UPDATE, and DELETE statements to query and modify FILESTREAM data
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Datatypes and Geospatial
• New datatypes for working with date/time data Datatype
Accuracy
Used when…
DATETIME
3/1000th of a second
Need to store milliseconds
SMALLDATETIME
Minute
Need to store minutes
DATE
Day
Need to store only date
TIME
100 nanoseconds
Need to store only time
DATETIME2
100 nanoseconds
Need more accuracy than 3/1000th of a second
DATETIMEOFFSET
100 nanoseconds
Same as DATETIME2 with TimeZoneOffset capability
New datatypes
• HierarchyId – Used to represent the position in a hierarchy – Has it’s own functions to work with hierarchical data • GetAncestor() • IsDescendant()
Geospatial Datatypes
• SQL Server 2008 adds native support for storing and working with geospatial data • New datatypes: Datatype
Used when…
More…
GEOMETRY
Represents a flat 2D surface with X and Y coordinates for points
GEOGRAPHY
Same as GEOMETRY but for Earth
The GEOMETRY data type doesn’t really work as expected when you pass in latitude and longitude pairs. For GEOMETRY, the distance between the POINTS (90 0) and (90 180) is 180 but in GEOGRAPHY, since both points refer to the same location (the North Pole), the distance is 0.
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
The Debugger is finally in SSMS
• In SQL Server 2005, the T-SQL Debugger was not in the SQL Programmer’s native environment – SQL Server 2000 included it in the Query Analyzer – SQL Server 2005 forced us to Visual Studio
• SQL Server 2008 features the T-SQL Debugger in SSMS
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
SSIS Enhancements
• New SQL Server Integration Services (SSIS) features and enhancements include: – C# is finally here! – Scripting is VSTA (Visual Studio Tools for Applications) instead of old SQL Server 2005 way: VSA (Visual Studio for Applications) • Easier to use custom assemblies
– Performance improvements such as better threading and pipeline scalability
SSIS Enhancements
• More (SSIS) features and enhancements: – Lookups are way better • SQL 2005 had many restrictions and inefficiencies – Limited to ~3GB of memory – OLEDB only – Binary matching mean case+accent sensitivity (with workarounds)
• SQL 2008 makes life easier – Persistent Lookups store a cached copy locally – OLEDB, ADO.NET, and flat file sources – More memory
SSIS Enhancements
• New and Improved – Data Profiling Task (already covered) – Change Data Capture makes ETL (Extract, Transform, Load) easier – New SSIS datatypes for working with new SQL date/time datatypes
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Reporting Services Improvements
• IIS no longer required to be on same machine to host the Reporting Services web service – Http.sys now handles HTTP requests just like IIS
• Better caching and better memory management
Reporting Services Improvements
• Tablix data regions for free-form reporting • Reports now can be rendered in Microsoft Word® • Able to respond to “3-4 times the number of users and their requires on the same hardware” compared to SQL Server 2005 Reporting Services1 1 – Source: Carl Rabeler, Program Manager on the SQL Server Customer Advisory Team
Reporting Services Improvements
• Prettier, nicer, better – Microsoft purchased Dundas Data Visualization technology to use inside SSRS and we now have • Gauges • Better calendar controls • Easier to manage charts
– Report Builder 2.0 – Report Designer
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Analysis Services
• Star JOINs • MERGE statement • Various enhancements
Deprecated and Discontinued in SQL Server 2008
Presented by Scott Whigham
What this video covers
• This is an overview, not a complete list – Check Books Online for complete list
Deprecated
• “Deprecated” items are slated for removal in a “future version, hot fix, or service release” • “Discontinued” items are “things that don’t work that way anymore” – Usually Discontinued items were once Deprecated
Discontinued Items that may affect you
• Some Discontinued items that may affect you are: Discontinued item
Replacement
BACKUP LOG WITH NO_LOG
None. Switch to SIMPLE recovery model instead
BACKUP LOG WITH TRUNCATE_ONLY
None. Switch to SIMPLE recovery model instead
60, 65, and 70 compatibility models
Databases must be at least 80 (SQL Server 2000)
Northwind and Pubs
Gone – use AdventureWorks instead
Surface Area Configuration Tool
Lasted only one version – SQL Server 2005
Deprecated Items that may affect you
• Some Deprecated items that may affect you are: Deprecated item
Replacement
BACKUP WITH PASSWORD =
None
DATABASEPROPERTY
DATABASEPROPERTYEX
sp_dboption
ALTER DATABASE
FASTFIRSTROW hint
OPTION(FAST n)
sp_dbcmptlevel
ALTER DATABASE… SET COMPATIBILITY LEVEL
BACKUP … TO TAPE
Backup to disk only
SET ANSI_NULLS OFF and ANSI_NULLS OFF
None – they will always be one
Deprecated Items that may affect you Deprecated item
Replacement
SQLOLEDB is not a provider
Use SQL Native Client
TIMESTAMP syntax for ROWVERSION datatype
Use ROWVERSION datatype
text, ntext, and image datatypes
Use VARCHAR(MAX), NVARCHAR(MAX) and VARBINARY(MAX)
sp_attach_db
CREATE DATABASE … FOR ATTACH
sp_change_users_login
ALTER USER
DBCC DBREINDEX
REBUILD option of ALTER INDEX
DBCC INDEXDEFRAG
REORGANIZE option of ALTER INDEX
DBCC SHOWCONTIG
sys.dm_db_index_physical_stats
Deprecated Items that may affect you Deprecated item
Replacement
sp_configure ‘allow updates’
No effect
sp_configure option ‘priority boost’
No effect
sp_addapprole
CREATE APPLICATION ROLE
sp_addlogin
CREATE LOGIN
sp_adduser
CREATE USER
sp_grantdbaccess
CREATE USER
sp_addrole
CREATE ROLE
sp_changeobjectowner
ALTER SCHEMA or ALTER AUTHORIZATION
sp_defaultdb
ALTER LOGIN
Deprecated Items that may affect you Deprecated item
Replacement
sp_grantlogin
CREATE LOGIN
sp_denylogin
ALTER LOGIN DISABLE
sp_revokelogin
DROP LOGIN
GRANT ALL DENY ALL REVOKE ALL
Must specify permissions independently
Lots of system tables
Compatibility views and Catalog views
WRITETEXT UPDATETEXT READTEXT TEXTPTR()
None.
Deprecated Items that may affect you Deprecated item
Replacement
3 and 4-part column names in queries
Use 1 or 2-part names only such as SELECT t.ProductName
GROUP BY ALL
Use custom case-by-case solution with UNION, CTE, or derived table
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
This Chapter
• This chapter is all about installing and upgrading to SQL Server 2008 • Included in this chapter are: – Installing SQL Server 2008 for the first time – Upgrading an entire SQL Server 2000 and 2005 to SQL Server 2008
This Chapter
• Related topics not covered in this chapter but covered later: – Upgrading a single database to SQL Server 2008
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
First Time Install of SQL Server 2008
• There are several items you need to know before you install SQL Server 2008 for the first time 1. 2. 3. 4. 5. 6. 7.
What type of server are you installing? Hardware, software requirements 32-bit or 64-bit? Which machine(s) need to run the install program? What security model will you use? Where will you put the files? What default server collation will you use?
• We’ll look at each one individually
Things You Need to Know Before Installing SQL Server 2008
1. What type of server are you installing? • •
The type of server determines the hardware What is the database profile of your server?
• 1 small database (DB) • Multiple small DBs • 1 medium DB • 1 medium DB and multiple small DBs • Multiple medium DBs • 1 Large DB • 1 Large DB and multiple small DBs
• 1 Large DB and 1 medium DB • 1 Large DB and multiple medium DBs • 1 Very Large DB (VLDB) • 1 VLDB and multiple medium DBs • 1 VLDB and 1 Large DB • 1 VLDB and multiple large DBs
Things You Need to Know Before Installing SQL Server 2008
• What other functions will this server perform? • SQL Server and Reporting Services on same machine • SQL Server and Analysis Services on same machine • Standalone Reporting Services server • Reporting Services web farm
• Standalone Analysis Services • Standalone SQL Server • File server • Domain controller • Mail server • SharePoint server
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
2. Hardware, software requirements –
–
The more you know about the type of server you are installing, the more you can accurately predict server needs Disk space is cheap but make sure you plan for growth
Things You Need to Know Before Installing SQL Server 2008
2. Hardware, software requirements – – –
Small and medium size databases need RAID 1 (OS, logs) and RAID 5 (data files) sets Large and VLDBs need RAID 10 SANs (Storage Area Networks) Don’t forget about backups!
Things You Need to Know Before Installing SQL Server 2008
3. 32-bit or 64-bit? –
Did you know? Windows Server 2008 will be the last 32-bit server from Microsoft The advantage of 64-bit architectures are that you can address more physical memory In a 32-bit system with 16GB of physical RAM, you can only directly address 4GB
–
– •
You can still use the rest but it uses Address Windowing Extensions (AWE) to “map” to the memory whereas 64-bit can address the memory directly
Things You Need to Know Before Installing SQL Server 2008
3. 32-bit or 64-bit? – –
64-bit supports larger numbers of processors and more linear scalability per processor This may mean lower Total Cost of Ownership (TCO) since you can “do more with less hardware”
Things You Need to Know Before Installing SQL Server 2008
4. Which machine(s) need to run the install program? – – –
One install on the server One install on each workstation that needs the management/development tools Possible need to install SQL Server Native Client on clients
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Two choices: • •
Windows Authentication only Combination of Windows Authentication and SQL Server Authentication (“Mixed Mode”)
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Windows Authentication only •
Can grant or deny Windows user or Windows group access to SQL Server
Things You Need to Know Before Installing SQL Server 2008
5.
Windows Authentication Example: – – –
•
Chad is the user “chadw” chadw belongs to two Windows groups: Users and SQLDevelopers The DBA adds “SQLDevelopers” group as a login to the SQL Server
Question: Can Chad now login to the SQL Server? –
•
Yes
Question: Who else can log in? –
Anyone else who is a member of the SQLDevelopers group
Things You Need to Know Before Installing SQL Server 2008
5.
Windows Authentication –
PROs: •
No passwords are stored in SQL Server or sent across the network Centralized management of users and groups (in Active Directory)
•
–
CONs •
The SQL Server DBA is no longer in control of security –
• •
Windows admins can now add/remove users from Windows groups thus elevating/denying privileges in SQL Server
Non-Windows machines have trouble logging in Anyone without a Windows account cannot use SQL Server
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Mixed Mode •
Can grant or deny Windows user or Windows group access to SQL Server DBA can also create SQL Server logins
• –
Have a login name and a password
Things You Need to Know Before Installing SQL Server 2008
5. Mixed Mode Example – – –
Scott’s iPhone runs an application that uses SQL Server Scott is a consultant who not have a Windows account The DBA adds a SQL Server login “scottw” with a password
Things You Need to Know Before Installing SQL Server 2008
5. Mixed Mode –
PROs: • •
–
Best of Windows Authentication mode SQL Server can control security more granularly
CONs •
Passwords are passed across network –
•
Can be encrypted
Microsoft says this is less secure
Things You Need to Know Before Installing SQL Server 2008
5.
Account Provisioning –
Depending on what features you install, you must determine how they log in The following features are require a service account:
– • • • • • • •
SQL Server Database Engine SQL Server Agent SQL Server Browser SQL Server Analysis Services * SQL Server Reporting Services * SQL Server Full-text Daemon Launcher * SQL Server Integration Services *
* Only if you install this feature
Things You Need to Know Before Installing SQL Server 2008
• • •
Always run SQL Server services using the lowest possible user privilege No need to run any SQL Server service as Administrator Use separate accounts for all SQL Server services (no account sharing)
Things You Need to Know Before Installing SQL Server 2008
5. SQL Server Database Engine –
Usually a domain account • •
–
Do not use NETWORK SERVICE account LOCAL SYSTEM is a bad idea since it has elevated privileges
May need to assign additional permissions to the user account to allow SQL Server to access directories for backups or import/export
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
6. Where will you put the files? –
During the installation you are asked: • • •
Where do you want the “Program Files” to go? Where do you want the database and log files to go? Where do you want your backups to go?
Things You Need to Know Before Installing SQL Server 2008
6. Default locations: Item
Path
Can Be Changed?
SQL Server Program Files
\Program Files\Microsoft SQL Server\MSSQL10.\
Yes
Reporting Services Program Files
\Program Files\Microsoft SQL Server\MSRSL10.\
Yes
Analysis Services Program Files
\Program Files\Microsoft SQL Server\MSASL10.\
Yes
SQL Server Data Files
\Data\
Yes
SQL Server Backups
\Backup\
Yes
Each installation is assigned an INSTANCE_ID. How this is named is covered later
Things You Need to Know Before Installing SQL Server 2008
6. Where should you put the files? – –
–
Program files should likely be in default location Data files and log files should be on RAID 5 and RAID 1 respectively for small/medium installations Backups should be on RAID 1 for small/medium installations
Things You Need to Know Before Installing SQL Server 2008
• Customize your installation:
Things You Need to Know Before Installing SQL Server 2008
7. What server collation should you use? – –
During setup you must decide the default server collation Collation is for working with character data
•Things You Need to Know Before Installing SQL Server 2008 SELECT * FROM Customer WHERE Lastname=‘Pérez’
–
Which of the following results would you want returned? • • • •
Perez PEREZ perez Pérez
• • • •
PÉREZ pérez Péréz PÉRÉZ
• • • •
péréz Peréez PERÉZ peréz
Things You Need to Know Before Installing SQL Server 2008
7. Collation defines: –
Sort order •
–
Does “A” sort before “a”?
Whether accents, binary, and case-sensitivity matter •
–
Are “Pérez” and “perez” the same?
The characters available in a column •
More on this later...
Things You Need to Know Before Installing SQL Server 2008
7. There are two types of Collations –
Windows Collation • •
What is the collation of your Windows machine? Used when: –
You know that all of your Windows machines have the “correct” collation settings
Things You Need to Know Before Installing SQL Server 2008
7. There are two types of Collations –
SQL Server Collation •
Who cares what the collation of my Windows machine is! Used when:
• – –
You want to install multiple instances with different collations Your Windows machines have different collations but you want all SQL Servers to have the same collation
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Instances
• Instances are “installations of SQL Server” • There are two types: – Default – the name of the computer is the name of the InstanceId – Named – you must provide a unique InstanceId
Instances
• Most sites usually have a single instance – Whether it is a default or named instance is hard to assume
• Instances compete for: – CPU – Memory – Disk space
Instances
• Single instances are appropriate for: – Situations where having a single server for security and management is best – Serving multiple databases to a single user – All databases should be on the same version/edition of SQL Server
Instances
• Multiple instances are appropriate for: – Situations where having a single server for all databases would be a security breach – Databases need be on the different versions/editions of SQL Server – Same database needs to exist on different versions/editions of SQL Server (for testing)
Instances
• Multiple instances can be: – Mixed versions • SQL Server 2000, 2005, 2008
– Mixed Editions • Enterprise, Developer, Standard, Express
– Mixed Releases • SQL Server 2008 RTW, SQL Server 2008 SP1
Instances
• There can be only one default instance • Instance names must be: – Unique – First character must be a letter – Not contain underscores or # – 16 characters or less – Cannot use the word “DEFAULT” anywhere – No funny business
Instances
• Instance Miscellany: – Enterprise Edition: up to 50 instances – Other editions: up to 16 instances
Instances
• Licensing – All editions now allow CPU license to cover “unlimited instances per operating environment”
Things to Know After You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Service Packs and Hotfixes
• Like all software, your SQL Server needs to be updated • You need updates for any and all features you’ve installed – SQL Server – Analysis Services – Reporting Services – And all the rest that don’t fit on this page
Service Packs and Hotfixes
• Primarily, SQL Server is updated using the Incremental Service Model (ISM) – The ISM is meant to be a standardized approach to keeping SQL Server up to date – All ISM releases can be requested by any customer regardless of their support offering
Service Packs and Hotfixes
• ISM is a way to distribute “hotfixes” – Critical on-demand (COD) • You must be vetted by Microsoft to qualify
– On-demand (OD) • Same exact requirements as COD…
Service Packs and Hotfixes
• ISM model continued – Cumulative Update (CU) • Update released every two months • Incremental numbers (CU1, CU2, CU3) • Contains all previous COD and OD hotfixes rolled into one update • May contain other fixes as well • All previous CUs since last service pack
Service Packs and Hotfixes
• ISM model continued – General Distribution Release (GDR) • “A GDR addresses an issue that has a broad customer impact, that has security implications, or that has both” – “An Incrementing Service Model…” – Microsoft Product Support, 2008
• Cannot be requested – Released when MSFT decides
Service Packs and Hotfixes
• Service Packs – Usually rollup every COD, OD, CU and GDR since the last service pack – Service packs are cumulative • SP3 includes all of SP1 and SP2
– Often include new features
Service Packs and Hotfixes
• ISM model:
Service Packs and Hotfixes
• We use “Product Level” to determine the service pack level – RTM = “release to manufacturer” • The initial release
– SP1 = “Service pack 1”
Service Packs and Hotfixes
• We also use “Build Numbers” to track what has been installed – Tell us which hotfixes, COD, OD, service packs, et al have been installed – Most hotfixes increment the build number
Service Packs and Hotfixes
• Build Numbers – To determine the service pack level, you can check the build number – To determine hotfix, COD, OD, et al, sometimes you need to look at the file version
• Build numbers are built around initial release and service packs
Service Packs and Hotfixes
• Sometimes you need the file version and the build number Version
Build Number File
File Version
SQL 2005 SP1
9.00.2047.00
9.00.2047.00
SQL 2005 SP2
9.00.3042.00
Microsoft.SqlServer.Mainten ancePlanTasks.dll
9.00.3042.00
SQL 2005 SP2 refresh
9.00.3042.00
Microsoft.SqlServer.Mainten ancePlanTasks.dll
9.00.3043.00
SQL 2008 RTM
10.0.1600.22
SQL 2008 CU1
10.0.1763
Service Packs and Hotfixes
• To determine your SQL Server build number: SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')
Service Packs and Hotfixes
• “How do I get the hotfix/service pack/etc?” – Some are publicly available (from microsoft.com) – Others require you filling out an “I agree that, if this breaks my system, it is my own fault” type of form – Others require contacting Microsoft and convincing them that you are eligible for the hotfix
Service Packs and Hotfixes
• Remember: A CU will be released every two months – Well, not exactly. But close enough – Put a recurring task to search for “SQL Server 2008 cumulative update”
Final Thoughts
• Be wary – COD are least tested – Service Packs are most tested – Install what you need – Use a test box
Final Thoughts
• Be prepared – Often, removing a hotfix/service pack/etc requires an uninstall/reinstall – Be sure you back everything up before installing – Be prepared to support multiple versions/builds
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
What Is Installed • Management Tools – – – – –
SQL Server Management Studio (SSMS) SQL Server Configuration Manager sqlcmd Import and Export Data PowerShell
• Development Tools – SSMS – Business Intelligence Development Studio – Visual Studio 2008
• Performance Tools – SQL Profiler – Database Engine Tuning Advisor
• Documentation and Tutorials – Books Online (BOL)
What Is Installed
• SQL Server Management Studio (SSMS) – The tool you’ll likely spend the most time with since it does both DBA and development • Backups, security, jobs • Stored procedures, queries, creating/modifying objects
– Deserves its own video!
What Is Installed
• SQL Server Configuration Manager – Allows DBAs to determine network configuration and aliases of servers – Used sparingly – often lots right after install and then 1x per month thereafter
What Is Installed
• sqlcmd.exe – Command line query and script execution engine – Can do most of what SSMS can do – Powerful scripting capabilities
What Is Installed
• Import and Export Data – A wizard for importing and exporting….. DATA! – Uses SQL Server Integration Services engine – What type of data? • SQL Server, Oracle, DB2, Sybase • Excel, Access, text/flat files • Any OLEDB or ODBC
Upgrading to SQL Server 2008
Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Upgrading
• First thing: decide what needs to be upgraded – Can upgrade an entire instance* – Can upgrade just one user database – Can upgrade all user databases
Upgrading
• This chapter only looks at upgrading an entire instance – Other chapters look at upgrading individual DBs
What Can You Upgrade?
• Upgrades only support 32-bit to 32-bit or 64-bit to 64-bit – No 32-bit to 64-bit native upgrade path – If needed, you will need to move individual DBs
What Can You Upgrade?
• What does the term “upgrade” mean? – There are two different ways to view: • Upgrading from a prior version (2000, 2005 only) • Upgrading from a lower edition of SQL 2008
What Can You Upgrade?
• Upgrading from a lower edition of SQL Server 2008 is usually: – Upgrading from Express to a “paid” version – Upgrading from Workgroup to Standard – Upgrading from Standard to Enterprise
What Can You Upgrade?
• Requires least amount of planning and prep-work
What Can You Upgrade?
• Upgrading from a different version of SQL Server requires significant planning – Are you upgrading to same edition? – Are you upgrading 32-bit to 64-bit? – Will file paths stay the same or change? – Will CPU or memory requirements change? – Is your SQL going to cause problems?
What Can You Upgrade?
• Upgrading from SQL Server 2000 to SQL Server 2008 is a challenge – So many architecture changes
• Upgrading from SQL Server 2005 is relatively simple
Things That Will Block Your Upgrade
• Basic stuff generally – New server doesn’t meet hardware/software requirements – Will be blocked if there is a pending restart – Windows Installer service must be running
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
The SQL Server Upgrade Advisor
• Regardless of type of upgrade, install and run the SQL Server Upgrade Advisor – Ships with SQL Server – Reports on incompatibilities and problems
• Before upgrading – Correct any errors reported – Note and act upon the warnings – Note the informational messages
Upgrading to SQL Server 2008
Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
The SQL Server Upgrade Advisor
• Regardless of type of upgrade, install and run the SQL Server Upgrade Advisor – Ships with SQL Server – Reports on incompatibilities and problems
• Before upgrading – Correct any errors reported – Note and act upon the warnings – Note the informational messages
Upgrading SQL Server 2000 DTS Packages Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
SQL Server 2000 DTS
• SQL Server Integration Services (“SSIS”) is the replacement for SQL Server Data Transformation Services (DTS) • SSIS is radically different from DTS and requires special consideration when upgrading
SQL Server 2000 DTS
• In an in-place upgrade, DTS packages will remain DTS packages in SQL Server 2008 – The upgrade will not “upgrade” your DTS packages – You can continue to edit the DTS packages if you install the SQL Server 2000 DTS Designer – You can even use the Microsoft SQL Server Backward Compatibility feature to run DTS packages as-is • 32-bit only!
SQL Server 2000 DTS
• To migrate DTS packages, run the Package Migration Wizard (“PMW”) – Right click on the package and select “Package Migration Wizard” – Can upgrade one package at a time or all packages – Complete the steps to upgrade
SQL Server 2000 DTS
• After migrating to SQL 2008, you might be surprised • SSIS is similar to DTS just like Microsoft is similar to Apple SSIS / DTS
Microsoft / Apple
Both are ETL tools
Both write operating systems
Both installed with SQL Server
Both sell MP3 players
Different ways of doing the same things
Different ways of doing the same things
SQL Server 2000 DTS
• Be prepared to do a lot of rewriting and re-architecting – Refactoring may take more time up-front but will likely save hours and hours in the long run – The ActiveX Script Task has been deprecated • Often the functionality is replaced by a built-in SSIS task
– Complex data transfers will often need to be rewritten
After You’ve Upgraded to SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
After You’ve Upgraded
• Life is good, right? • Check to make sure everything came over as expected – It probably did
• It’s a good time to change service account passwords – If you don’t already have a policy for doing so
After You’ve Upgraded
• Things you should test – “The Obvious” – Do your DTS packages still work? – Do your jobs still run? – Your pre-upgrade Profiler trace queries – do they still return same results? – Does your application still work? – Do the things that the Upgrade Advisor warned you about fail?
After You’ve Upgraded
• Things you should test – “The Not-SoObvious” – Do your datetime queries/applications still run as expected? • Failures using the Enterprise Library 4.x with .NET 3.5 and SQL 2008 for datetime and smalldatetime
After You’ve Upgraded
• Next steps – Understand the configuration parameters – Check your database compatibility – Review “Suggested Changes After Upgrade” video
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
This Chapter
• What this chapter covers – What has been installed – An overview of each of the tools • Answering The Three W’s: – What it is, What it does, Who Uses It
– An overview of configuring the server
This Chapter
• What this chapter does NOT cover – An overview of configuring databases – In-depth coverage of all tools – Every single server option available – Cover every tool available
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
An Overview of What Is Installed
• There are tools available for: – Administrators • • • •
SQL Server Database Administrators (DBAs) Reporting Services Administrators Analysis Services Administrators Systems Administrators
– Developers • • • •
SQL Programmers Report developers Integration Services developers Analysis Services developers
An Overview of What Is Installed
• There are tools available for: – Command line only – GUI (“Graphical User Interface”) – APIs (“Application Programming Interface”)
• Let’s look at each
Command line tools
• Command line tools primarily target DBAs Tool
Who uses?
Description
sqlcmd.exe
DBAs, Developers Command line queries/batches
dtexec.exe
DBAs
Execute SSIS packages
sqldiag.exe
DBAs
Diagnostics, system info, monitor, troubleshoot
bcp.exe
DBAs
Import/export text files
profiler.exe
DBAs
Trace server activity
GUI tools
• GUI tools are for everyone! Tool
Who uses?
Description
SQL Server Management Studio (SSMS)
DBAs, Developers Queries, management, security
Business Intelligence Development Studio (BIDS)
Developers
Develop/edit reports, SSIS packages, SSAS
Visual Studio 2008
Developers
It is the “real” name of BIDS
SQL Server Configuration Manager
DBAs
Manage services, aliases, and network configuration
GUI tools
• More GUI tools Tool
Who uses?
Description
SQL Server Profiler
DBAs
Trace server activity
Database Engine Tuning Advisor
DBAs
Performance tuning, optimizations
Data Profile Viewer
DBAs
View stats on column/table usage
Command line tools
• APIs are for developers and integrators API
Who uses?
Description
SQL Server Management Objects (SMO)
Developers
Manage SQL Server via programming interfaces
SQL Server Replication Developers Management Objects (RMO)
Manage replication via programming interfaces
Data Collector
Developers
Capture, store and report on performance/health stats
Native Client
Developers
Combines both ODBC and OLE-DB into one DLL
WMI
Developers
Use WMI for server events and configuration
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• You can configure SQL Server at the instance and database level • This chapter covers the instance level • Next chapter covers the database level
Configuration
• Instance-level configurations affect all connections and all databases for that instance • You can change settings using – – – –
SQL Server Management Studio SQL Server Configuration Manager sqlcmd.exe SQL Server Management Objects (SMO) API
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Service Accounts
• Service accounts are not static – Change passwords – Change accounts
• Changing anything here can be an exciting experience • To change, use the SQL Server Configuration Manager
Configuration: Service Accounts
• To modify the service accounts, use the SQL Server Configuration Manager
Configuration: Service Accounts
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Network Protocols
• Network protocols define what connections SQL Server will accept
Configuration: Network Protocols
• If SQL Server has TCP/IP enabled but all others disabled, only clients using TCP/IP for connections would be allowed – Both the server and client must use the same protocol
Configuration: Network Protocols
• Shared Memory – Connections and data are not transferred across IP stack; done in memory – Clearly this only works when the client and server are on the same machine – For remote connections, must use one of the other network protocols
Configuration: Network Protocols
• How to Enable Remote Connections – Default for Express and Developer editions is to turn off remote connections • This is done by disabling all protocols except Shared Memory
• To enable remote connections: 1. Enable TCP/IP or one of the other network protocols 2. Restart SQL Server instance
Configuration: Network Protocols
• TCP/IP is most common – Requires a unique port per instance – Default port for default instance is 1433 – Named instances use a random-first-time port
Configuration: Network Protocols
• Change/view settings in the Configuration Manager – Any changes here require restarting instance
Configuration: Service Accounts
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration: Memory, CPU
• Memory and CPU usage can change – When new hardware is added – When the application’s usage changes
• Some changes require restarting the SQL Server service but most are dynamic • Let’s look at memory in this video
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Memory, CPU
• Memory is used the most in two areas – Buffer cache • Stores the data users are working with in memory to avoid reading disk on commonly-requested data
– Procedure cache • SQL Server caches the “how” of running queries and routines to avoid having to figure out the fastest route to the data on subsequent calls – The “how” is the “Execution Plan”
Configuration: Memory, CPU
• Sizing the Buffer Cache – Too small means that users have to wait for SQL Server to access hard disk – Too large means that the operating system and/or other applications do not have enough memory
Configuration: Memory, CPU
• Buffer cache as a percentage of the data file size – The larger the database, the smaller the percentage of the size • A 10TB system with 10 concurrent users likely needs a 1-2% buffer cache • A 10GB system with 100 concurrent users may need as much as a 25% buffer cache
Configuration: Memory, CPU
• The Buffer Cache should be expected to be between 5 and 25% of the data file size for an online transaction processing (OLTP) system – Should be a bit larger percentage for reporting
Configuration: Memory, CPU
• Sizing the Procedure Cache – On a 32-bit system, the procedure cache is limited to 1GB • Cannot use Address Windowing Extensions (AWE)
– On a 64-bit system, procedure cache can be larger • This is necessary on larger systems
Configuration: Memory, CPU
• Memory options you can change – Max server memory • Always, always set this to avoid a too-large buffer cache
– Min server memory – Using Address Windowing Extensions (AWE) • Only used in 32-bit installations when there is > 4GB of RAM
Configuration: Memory, CPU
• Memory options you can change (cont.) – Index create memory • Specifies the amount of memory to use in index creation sorts • 0 means dynamic • Have a good reason to change
Configuration: Memory, CPU
• Memory options you can change (cont.) – Minimum memory per query • Specifies the amount of memory to use for execution of a query • 0 means dynamic – 1024KB is default • Have a good reason to change
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Memory, CPU
• CPU settings that can be changed – Enable/disable processors • Licensing needs • System performance
– Affinity mask options • Processor affinity: binds processors to specific threads to prevent switching mid-execution • I/O affinity: binds disk I/O to specific processors
Configuration: Memory, CPU
• CPU settings that can be changed (cont.) – Maximum worker threads • CPUs do work on threads • Do you need to set a maximum? – Default is “0” which means dynamic
– Boost SQL Server priority – specifies whether SQL Server process should execute at a higher priority • Advanced option
Configuration: Memory, CPU
• Parallel queries are queries whose work is split among CPUs • “Cost” is a metric of time – A query with a cost of 1 says that theoretically will take 1 second • This was determined pre-release in the Microsoft labs therefore it will not likely match your system performance
Configuration: Memory, CPU
• You can configure how SQL Server handles parallel queries – Cost threshold for parallelism • SQL Server does not run a parallel plan for queries that “cost” less than this amount • Default value is 5 – This means that a query has to be estimated to run for 5 seconds before SQL Server will consider using a parallel plan – 5 is the default for a very good reason
Configuration: Memory, CPU
• You can configure how SQL Server handles parallel queries (cont.) – Max degree of parallelism • Controls how many processors are available to run a single statement • Default value is “0” which means to use all available CPUs • Can be overridden at query level
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
Configuration: Security
• Security settings that can be changed: – Authentication mode • Requires restarting SQL Server service to take effect
– Login auditing • Bare auditing – doesn’t record too much information – None, Failed only, Successful only, Both
– Server Proxy Account • Used with xp_cmdshell • Not likely to use but, if you do, be careful
Configuration: Security
• Security settings that can be changed (cont.): – Enable Common Criteria Compliance • Forces SQL Server to adhere to Common Criteria compliance • Developer and Enterprise edition only • More info: http://www.commoncriteriaportal.org/
– Enable C2 audit tracing • Audits all attempts to access statements and objects to file(s)
– Cross database ownership chaining • Coming up in later chapter
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Connections
• Connection options affect all connections to the instance – Some can be overridden at connection level
• Triple check this in development/testing before deployment – One setting change can cause critical parts of an application to fail
Configuration: Connections
• Connection options you can change: – Maximum number of concurrent connections • Default is “0” which is no limit
– Use Query Governor to prevent long-running queries • Great idea in theory; better to use Resource Governor
Configuration: Connections
• Connection options you can change (cont.): – Allow remote server connections • Do you want remote servers to run stored procedures on this server?
– Require distributed transactions for server-toserver communication • Uses Microsoft DTC to ensure that transactions completely succeed on all servers
Configuration: Connections
• Common connection options – All are “Off” by default – This allows client to set it’s own defaults Setting
Description
Default
ansi nulls
Does the statement “WHERE LastName = NULL” return any rows?
Statement would return 0 rows; requires ISNULL to test for NULL values
no count
“34 rows affected” messages are returned to client
quoted identifier
Do double quotes identify object names or query criteria?
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
Configuration: File locations & Compression
• Defaults for new databases: – Default index fill factor • How full should a “page” be? • Default is to fill page completely
– Backup compression • Sets whether a backup is compressed by default • Any backup can be compressed or uncompressed regardless of this setting • For Enterprise and Developer edition
Configuration: File locations & Compression
• Defaults for new databases (cont.) – Database default locations • Configure the default locations for data and log files individually • For new databases only • Default locations are only used in statements that do not specify file locations
CREATE DATABASE ScottsDB
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008 • Demo
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
How to Change Configuration Settings
How to Change Configuration Settings
• How to change configuration settings – Use SSMS for interactive or jobs – Use sqlcmd.exe for scripts
• Can use the GUI and Transact SQL scripts
Configuration
• GUI is great for learning • Scripts are better for – Change tracking – Undoing – Sharing across machines/users
• Changes require two things: – The actual change – Acknowledgement that the change should be applied
Configuration
• Tips – Have a “backup” or “back out” plan – Understand that SQL Server will often clear out portions of its memory after configuration changes • This can have severe short-term impact!
Configuration
• “Configured values” are what SQL Server will change to once you click “OK” • “Running values” are what SQL Server is currently using – Running values is a read-only view of what the system is using right now
Configuration
• To change configurations in the GUI: 1. Launch SQL Server Management Studio 2. Login as a sysadmin 3. Right click on server and select Properties
• After changes, click “OK” to save •
Try, try, try not to do this during production hours!
Configuration
• sp_configure is the Transact-SQL way to modify server configurations • To use sp_configure, you must first enable the “advanced” options EXEC sp_configure ‘show advanced’, 1 RECONFIGURE
Configuration
• sp_configure requires use of RECONFIGURE or RECONFIGURE WITH OVERRIDE to take effect without a restart of SQL Server service – Otherwise changes will not take effect until you stop/start
EXEC sp_configure ‘show advanced’, 1 RECONFIGURE
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
This Chapter
• What this chapter covers – Databases, data bases and DBs • Creation and management
– Database file structure – An overview of configuring the database
This Chapter
• What this chapter does NOT cover – Every single database option available – Performance tuning and optimizations – Monitoring
This Chapter
• Pre-requisites to this chapter – You should already know how to create a database using the GUI
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
SQL Server Databases
• SQL Server databases have at least two files – Data files – Transaction Log files (commonly called “log files”)
• A database has at least one of each – We’ll look at when to have multiples of each later
SQL Server Databases
• Data files store everything that a user needs such as – The data – The indexes – Source code for routines (stored procedures, functions) – Security information (users, roles, permissions)
SQL Server Databases
• Transaction log files are used to protect SQL Server databases from “failure” • Failure could be – User error – Disk failure – Software failure – Dreaded BSOD (Blue Screen of Death)
SQL Server Databases
• Every write in a database is done inside of a transaction • All transactions are written first to the log and then to the data file • This is known as “Write Ahead Logging” – http://en.wikipedia.org/wiki/Write_ahead_logging
SQL Server Databases
• You begin a transaction with the BEGIN TRAN statement • A transaction may include many statements
SQL Server Databases
• A transaction remains “open” until either you: – COMMIT TRAN – ROLLBACK TRAN
(called “rolling back”)
• Once “closed”, the transaction may never be re-opened
SQL Server Databases
DECLARE @ProcessingHistoryId INT DECLARE @PayAmount DECIMAL(19,2) = $1.94 BEGIN TRAN INSERT PayrollProcessingHistory VALUES (‘Whigham’, ‘2009-04-01’, @PayAmount) SELECT @ProcessingHistoryId = SCOPE_IDENTITY()
INSERT ChecksToWrite VALUES (@ProcessingHistoryId , @PayAmount) COMMIT TRAN
SQL Server Databases
• Transactions in SQL Server are durable – Once the transaction is committed, the transaction will remain in the database even in the event of a system crash
• Demo of write-ahead logging
*More on transactions in our “SQL Server 2008 Programming” course
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
System Databases
• SQL Server uses SQL Server for configuration – Some configuration is in the registry – Some configuration is in XML files
• The majority of database and instance configurations are stored in the five system databases
This Chapter
• The five system databases: – master – model – msdb – tempdb – Resource
• Let’s look at each one
The master Database
• The master database: – Is the first database “loaded” by SQL Server service on start – Stores instance-level configuration such as memory, CPU, and the info set by sp_configure – Stores information about each database on the server – Stores the logins into the server – Stores routines that are shared among all databases
The model Database
• The model database: – Is a template for new databases – Whatever objects are in model will be created when a new database is created – Useful for standardization
The msdb Database
• The msdb database: – Stores any and all automation configuration and information • Log shipping, mirroring, jobs, alerts, mail • History of all of the above
– May optionally store SSIS packages – Stores history about backups – Used heavily by SQL Server Agent
The tempdb Database
• The tempdb database: – Is what the name implies: temporary storage – Temp tables are created here – Row versioning is done here – Is “wiped clean” at shutdown and created at startup • You can actually delete the database files on the hard drive after shutdown and SQL Server will create them anew
The Resource Database
• The resource database: – Hidden from view; must view at the disk level – A read-only database that contains all of the “system objects” in SQL Server • Each database has “pointers” to these system objects
– Makes it easy to upgrade • During upgrade, MSFT can just replace the resource database and immediately all databases on server have latest source code and objects
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Types of Databases
• Databases are either: – System databases – shipped with SQL Server; from Microsoft – User databases – everything else; not an official term
• A typical SQL Server has many user databases
Types of Databases
• User databases can fulfill many different functions – Customer Relationship Management (“CRM”) – Payroll – Document storage – Website analytics package – Configuration – Project management – Instrument measurement & Quality Control
Types of Databases
• Each of these databases has multiple needs – A CRM database needs to provide • Heavy read and reporting capability • Light to Moderate write capability • Could be anywhere from 5 reads for every write (5:1) up to 50:1 reads to writes
Types of Databases
• An instrument measurement database needs extremely fast write capability – 250+ writes per second – Reads are done at reporting level • Defect reports, abnormalities
– Could be 1000:1 writes for every read
Types of Databases
• A website analytics package might use multiple databases – One database needs extremely fast write capability to capture real time data – Another database needs extremely fast read capability for reporting
Types of Databases
• Database needs change with time – End of month reporting – End of year reporting – Tax season
Types of Databases
• Databases are classified as having one or more of these attributes: – Online transaction processing (“OLTP”) – Online analytical processing (“OLAP”)
• A database might serve both needs • It is generally better to serve a single purpose
Types of Databases
• A data warehouse is an example of an OLAP database • Data warehouses are “fed” by the OLTP systems on a scheduled basis – This ensures that the reports will be up to date – Often called a “batch load” or a “bulk load”
Types of Databases
• SQL Server Analysis Services (“SSAS”) stores a multidimensional data warehouse – Requires installing SSAS – Stores data in “cubes” – Use SSIS to load data from the SQL Server OLTP and/or OLAP database into the cubes * More information can be found in our SQL Server 2008 Analysis Services course
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases
• You can create databases: – In the GUI (SSMS) – Transact-SQL scripts – Using SQL Server Management Objects (SMO)
Creating Databases
• The GUI is great for one-time use • Scripts and SMO are great for standardization and re-creation – A software vendor that wants to create a SQL Server database during setup – A DBA needs the same database installed on multiple servers
Creating Databases
• Transact-SQL scripts can be executed in – SSMS – sqlcmd.exe – .NET, Java, or any language that can connect to SQL Server – 3rd party query tools
Creating Databases
• SQL Server Management Objects (SMO) can be used – .NET, Java, or any language that can reference the necessary assemblies
Creating Databases
• Databases are just files – Data files – Transaction log files
• Files must be local – No mapped drives or UNC paths
Creating Databases
• Data files come in two flavors: – Primary data file: the first data file created • Generally has an “.mdf” extension
– Secondary data files • Generally have an “.ndf” extension
Creating Databases
• There is only one transaction log regardless of how many physical log files • Log files generally have an “.ldf” extension
Creating Databases
• “When do I need more than one data file and/or more than one log file?” – Two reasons: • Too much data to store in a single data file or extreme logging needs mean that too much logging for one single file • Performance
Why should I use multiple files?
• Space and file management – May not have space available for growth on current drive; add new drive and new file – Data or Log file may be too large for one file – Data or Log file may not be easily re-created on another machine if it gets too large • When restoring a database, you cannot split files!
Why should I use multiple files?
• Performance – The more disks, the faster performance – SQL Server can read two files on two separate disks in parallel • Can nearly double read and write access
Why should I use multiple files?
• Files can be added dynamically – Instant access – Essential at a time when you’ve run out of log space!
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases
• After the database has been created, all file manipulation is done by referring to the logical file name – Adding space to a file – Shrinking a file – Removing a file
Creating Databases
• The logical file name is an abstraction layer – It need not refer to the database by name
• Physical file names cannot change while database is online – Logical file names can change at any time without affecting database status
Creating Databases
• Autogrowth is a safety net to prevent your database from preventing users from doing their jobs • The file will not grow until the file is 100% full – It then grows according to the file growth parameters
Creating Databases
• Autogrowth parameters: – Grow the file by a percentage of current size or a fixed amount (in MB) – Set maximum size
• Hint: Always set a maximum size – Do not let the file grow until it fills up the disk
Creating Databases
• Best Practices for Creating Databases – Create the database the size that you expect it to be in 12-18 months • This prevents autogrowth from happening during peak hours
– Do allow autogrowth • Keep parameters small and easy so that database can grow quickly and be usable quickly
Creating Databases
• Best Practices for Creating Databases (cont.) – Have a plan for growing the database in 12-18 months (or before autogrowth kicks in) – Always, always keep an eye on the file sizes • Particularly the transaction log
Creating Databases
• Expanding a file is easy and dynamic • We’ll take a look at working with multiple file databases in an upcoming video
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Filegroups
Filegroups
• A data file belongs to a filegroup • A filegroup contains n data files • A database may have n filegroups – A data file belongs to only one file group
• Filegroups allow you to do data placement and backup/restore on an entire filegroup
Creating Databases
• Each database has a PRIMARY filegroup – The primary data file belongs to the PRIMARY filegroup
Filegroups
• Transaction logs do not belong to filegroups
Multiple File Databases
• One advantage of filegroups is data placement – Tables can have “Regular Data” on one filegroup and “Text/Image” data on another CREATE TABLE dbo.MyTable ( MyColumn nchar(10) NOT NULL, MyVarColumn varbinary(MAX) NOT NULL ) ON RegularDataFG TEXTIMAGE_ON ImageDataFG GO
Multiple File Databases
• Filegroups can also be marked as read only – Any objects on the filegroup are read only while the rest of the database remains read/write ALTER DATABASE LearnItFirst MODIFY FILEGROUP [Sales2001] READONLY
Filegroups
• When there are multiple files in a filegroup, SQL Server uses a proportional file strategy – SQL Server does not fill up one file and then fill up the next file • Doing so would kill performance on a system that had files on separate drives
Filegroup Example
• LearnItFirst database has two data files in the MyFG filegroup – DataFile1: 100MB free – DataFile2: 200MB free
• SQL Server will write one extent to DataFile1 and two extents to DataFile2 – An extent is eight pages
Filegroup Example
• Once all files in the filegroup are full, it will expand the first file and write until it is full before expanding the second file
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Multiple File Databases
Multiple File Databases
• Adding a log file is easy – The log will now be “striped” across all log files – Can be done using GUI, TSQL or SMO
• Adding a data file is just as easy – The data will now be ”striped” across all files in the filegroup – Can be done using GUI, TSQL or SMO
Multiple File Databases
• When you add a new file to the database, it defaults to the PRIMARY filegroup – Best practice for multiple data file databases is to create a separate filegroup for user data and leave the primary data file as the sole file in the PRIMARY filegroup
Multiple File Databases
• Removing a file is hard – Requires “emptying” the file – Emptying the file requires you to move the data on the file to other data files in the same filegroup
• Demo of – A database filling up – Adding/removing files
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring SQL Server Databases
• Instance configuration affected every database on the instance – A single SQL Server instance may serve 1,000+ databases
• Each database may serve a different purpose and may benefit from individual configuration
Configuring SQL Server Databases
• SQL Server databases are autonomous; any configuration to one database has no effect on other databases • Different functions require different configurations – OLTP, OLAP, BLOB storage, Historical data only, mix
BLOB: Binary Large Object
Configuring SQL Server Databases
• This section focuses on the individual database configuration • We’ll cover the main options only – There are more than 30 different options
• We’ll cover upgraded databases in an upcoming section
Database Configuration
Database Configuration
• Database options are set by choosing between a predefined set of values – Most common is True/False
• Some database options override instance options – ANSI NULLs, Quoted Identifiers, et al
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
• Let’s take a look at each of these in the next video!
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Automatic” Category control how SQL Server proactively manages your database – “Auto Close” – Takes the database offline when the last user leaves the database • Defaults to FALSE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Create Statistics” • Statistics are used by SQL Server to compute the fastest route to data • If TRUE, SQL Server will create statistics on columns the first time they are referenced. • Defaults to TRUE; Do not change unless you know this will help your situation
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Shrink” • If your database has free space, SQL Server periodically will remove the free space and give it back to the operating system • Defaults to FALSE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Update Statistics” • Statistics can get out-of-date (“stale”) due to large amounts of UPDATEs and INSERTs • When SQL Server encounters a query with out-of-date statistics, the statistics are updated prior to the query execution – This can cause a lag
• Defaults to TRUE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Update Statistics Asynchronously” • Statistics are updated when encountered but the query does not wait for the statistics update command to complete • Defaults to FALSE
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Cursor” Category control how SQL Server behaves with cursors – Few applications use cursors today – Be careful about testing before/after you change
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Some settings in the “Miscellaneous” Category override Instance-level defaults – Can set “Default Connection Options” at instancelevel – Can be override at both database-level and connection-level WARNING: Some apps depend on certain settings so do thorough testing when making changes
Database Configuration
• Other settings in the “Miscellaneous” Category are database-specific – Date Correlation Optimization Enabled • Used for foreign keys on DATETIME columns • Defaults to FALSE SELECT * FROM Parent p JOIN Child c ON p.DateTimeKey = c.DateTimeKey
Database Configuration
• Other settings in the “Miscellaneous” Category are database-specific – Cross-database Ownership Chaining Enabled • Covered in next chapter
– Trustworthy • Covered in next chapter
– VarDecimal Storage Format Enabled • In SQL Server 2008, all tables use the VarDecimal Storage Format • Was a changeable setting in 2005
Database Configuration
• More settings in “Miscellaneous” – Parameterization • • • •
Specifies how SQL Server works with query plan re-use Two choices: SIMPLE or FORCED Defaults to SIMPLE Can be overridden at query level
Database Configuration
• Parameterization continued… – When parameterization is SIMPLE and SQL Server recognizes a query that it thinks will be used in the exact same way but with different SARGs (Search ARGuments), it will attempt to create a reusable execution plan • Previously called “auto-parameterization”
Database Configuration
SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 1
SQL Server recognizes that this query will likely have multiple variations SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 1 SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 5
SQL Server creates a reusable, parameterized query plan that satisfies both queries
Database Configuration
• In SIMPLE parameterization, SQL Server will only create reusable plans for queries that it believes it can be reused • In FORCED parameterization, any literal in any SELECT, INSERT, UPDATE or DELETE statement is parameterized
Database Configuration
• FORCED parameterization can offer a tremendous performance gain for applications in which the database is not auto-parameterizing queries
Database Configuration
• FORCED parameterization can slow down databases as well due to a bloated plan cache or reusing sub-optimal plans • See this article for more: – http://msdn.microsoft.com/en-us/library/ms175037.aspx
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Recovery” Category control how SQL Server verifies the contents on a “page” – SQL Server stores data and indexes on “pages” – The “Page Verify” option tells SQL Server how to check that the data/index page is not corrupt – Data can become corrupt due to incomplete I/O writes and/or disk errors
Database Configuration
• “Page Verify” has three choices: – CHECKSUM (the default) • SQL Server computes a checksum of what the page’s checksum would be if the wrote was successful • After the write, SQL Server compares the checksum values • If they are not the same, the page is logged in msdb’s suspect_pages table
Database Configuration
• “Page Verify” has three choices (cont.): – TornPageDetection • SQL Server writes a “flag” bit for every 512 bytes • By counting the “flags”, SQL Server can determine if the page has the correct number of bytes • However, it cannot determine that the data is corrupt inside of the 512 bytes – It just counts that 512 bytes were written
Database Configuration
• “Page Verify” has three choices (cont.): – None • No page verification is done
• Suggestion: Leave the default setting asis (CHECKSUM)
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Service Broker” Category control… well, the Service Broker • Service Broker allows for messaging and queuing applications
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• “State” controls whether SQL Server database is – Online or Offline • i.e. accepting connections or not
– Read-Only or Read-Write
• Databases default to Online, Read-Write
Database Configuration
• “State” also controls whether SQL Server is – Single-user, Multi-user or “Admins Only” • “Admins Only” is technically called RESTRICTED_USER
– Whether “Transparent Data Encryption” is enabled • More on this later (requires more setup)
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Indexes
• SQL Server has three types of indexes • (or “indices” if you prefer)
– Clustered indexes – Nonclustered indexes – Full-Text indexes • Not covered in this course
Indexes
• Most tables have both clustered and nonclustered indexes – Only tables that store large/wide text columns that need frequent searching would use Full-Text
• The purpose of an index is to speed up searches
Indexes
• Queries use “Search Arguments” or SARGs SELECT PhoneNumber FROM PhoneBook WHERE LastName=‘Whigham’ AND FirstName=‘Scott’
• To satisfy the SARGs, it helps if the data is sorted in LastName, FirstName order
Indexes
• A clustered index is really how the actual data is stored – A clustered index on “LastName, FirstName” means that the actual data on the page is sorted in alphabetical order by LastName, FirstName
• When you create a primary key, the default is to create a clustered index on that key
Indexes
• Consider this query: SELECT LastName, FirstName FROM PhoneBook WHERE PhoneNumber = ‘+1(214) 555-1212’
• Would having the data sorted by LastName, FirstName order help this query?
Indexes
• Solution: create a nonclustered index on PhoneNumber! – A nonclustered index is a copy of the index keys and the primary key sorted by the index keys – Now a copy of the PhoneNumber column is created along with a pointer back to the primary key
Indexes
• Indexes use a fill factor to determine how full to create the page – A fill factor of 50% means that the page will be created 50% full (thus leaving 50% for new rows to be written to the page)
Indexes
• A full index/data page cannot have any more data written to it – Imagine a phone book page full of “Smith” entries – No room to write any more – Next page is a full page of “Sullivans”
• A new “Smith” moves to town – How does this record get added to the PhoneBook table?
Indexes
• Solution: A Page Split! – When a page split occurs, a new page is created (somewhere on the disk) – Half of the original page’s data is “cut and pasted” onto this new page along with the new row
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Indexes
• Page splits cause poor performance in multiple ways – The time it takes to allocate a new page (and maybe even having to grow the database) – The time it takes to “cut and paste” (and log) – The pages are not contiguous any longer thus causing more work at the disk level
• This is called “Index Fragmentation”
Indexes
• Solution: You need to Rebuild or Reorganize Your Indexes Periodically – Called “Defragmentation” – Speeds up performance
Indexes
• An index “REBUILD” creates the best performance • SQL Server re-creates the index internally again • Once complete, the existing index is dropped • There is a period of time where the index/data is unavailable (“locked”) • No pages are split after running – all are contiguous
Indexes
• An index “REORGANIZE” does a “best effort” defragmentation but leaves the data online • Reorganizes the “leaf nodes” of the index • Some fragmentation may still exist since it does not fix page splits
Indexes
• To rebuild an index, use the ALTER INDEX command -- Rebuild all indexes on table ‚Members‛ ALTER INDEX ALL ON Members REBUILD WITH(ONLINE = ON) -- ONLINE=ON keeps the table available -- during the rebuild process
Indexes
• To rebuild an index, use the ALTER INDEX command -- Rebuild only the primary key index ALTER INDEX PK_Members ON Members REBUILD WITH(ONLINE = ON, FILLFACTOR=50) -- Keeps data available but uses a new -- fill factor of 50%
Indexes
• To reorganize an index, use the ALTER INDEX command -- Rebuild only the primary key index ALTER INDEX PK_Members ON Members REORGANIZE -- Reorgs are always online operations
Indexes
• Index defragmentation needs to be part of your normal maintenance – How often you do each requires a thorough understanding of your table data and usage
• You will create jobs and maintenance plans to do this work using Transact-SQL scripts
Indexes
• Scenario: “Members” table adds 10,000 new rows per week and is heavily read – Primary key is an auto-incrementing IDENTITY – Fill factor for all indexes is 75% – One primary key (on IDENTITY column) – Six nonclustered indexes to cover common queried columns
Indexes
• Should we rebuild or reorg? – Wrong question! – Start with the idea that you want to rebuild but will “fall back” to a reorg in the event that the table is too heavily used • In this event, scheduled maintenance windows (if possible) can be the time that you do rebuilds
Indexes
• Frequently Asked Questions – How often should I defrag? • As often as possible without affecting users • When the table becomes fragged!
– Should I rebuild/reorg all indexes at once or do them individually? • It’s easier management-wise to do all at once • Large indexes/tables may require separate operations
Indexes
• Let’s do a demo!
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Partitions
• Large tables often have data that is used in different ways – An “Orders” table might have 2-10 years of data – Some data is heavy read/write while other data is read-only
Partitions
• With large tables – Index maintenance can take hours – Queries against large tables can take a long time • In a table of 200,000,000 rows, how long would it take to find all orders placed on Jan 1, 2001 at 23:43:23.243?
Partitions
• Large tables can be partitioned into smaller, more manageable subsets – An “Orders” table can be partitioned into one partition per year – A query for rows WHERE OrderDate = ‘Jan 21, 2001’ now only needs to access a single partition
• Indexes can be partitioned as well
Partitions
• Partitions allow you to create subsets of the data and to place them strategically – Partitions can be spread across multiple filegroups – Queries only need to access the partition(s) that serves their data
Partitions
• Partitions can be spread across multiple filegroups – Some partitions can be read-only while others are read-write – Can backup more heavily-written-to partitions more frequently
Partitions
• Rows are partitioned horizontally – This means that the entire row is in a single partition
• Time is a common partitioning strategy – One partition for each year – One partition for each quarter – One partition for each month
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Partitions
• Creating partitions requires: – A Partitioning Function – A Partition Scheme – A table/index
Partitions
• Partitioning Functions define “what goes where” – You define boundaries and ranges
• Boundaries define the number of partitions • Ranges define how the boundaries are defined
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008)
Partitions
• This partition function can be used for any table/index that has an INT column • Three partitions will be created above • RANGE defines how the partitions are defined
Partitions
RANGE RIGHT FOR VALUES (2007, 2008) Partition 1
Partition 2
Partition 3
WHERE TheYear < 2007
WHERE TheYear >= 2007 WHERE TheYear => 2008 AND TheYear < 2008
TheYear is an INT column in the above table
Partitions
RANGE LEFT FOR VALUES (2007, 2008) Partition 1
Partition 2
WHERE TheYear <= 2007 WHERE TheYear > 2007 AND TheYear <= 2008
Partition 3 WHERE TheYear > 2008
Partition Schemes
• A Partition Scheme maps the partition function(s) to a filegroup(s) – Partition functions can belong to multiple partition schemes – One partition scheme can only contain one partition function
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008) GO CREATE PARTITION SCHEME MyScheme AS PARTITION ScottsPF TO (myFilegroup1, myFilegroup1, myFilegroup2)
Partition Schemes
• The preceding sample mapped the three partitions from “ScottsPF” to two different filegroups TO (myFilegroup1, myFilegroup1, myFilegroup2)
• Partition schemes can also map all partitions to the same filegroup ALL TO (myFilegroup1)
Partition Schemes
• You can partition tables and/or indexes – If you partition the table, the indexes are partitioned by default (unless otherwise specified during index creation)
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008) GO CREATE PARTITION SCHEME MyScheme AS PARTITION ScottsPF ALL TO (‘default’) GO CREATE TABLE MyTable (MyID INT, TheYear INT) ON MyScheme(TheYear)
Partition Schemes
• Let’s do a demo!
Managing Database Files
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Managing Database Files
• We’ve already seen how to grow databases – By adding space to an existing file – By adding additional data or log files
• Now it’s time to see how to make our “footprint” smaller!
Managing Database Files
• Up first: Removing a Database • When you remove a database, you are removing – Users – Data (tables, indexes) – Stored procedures, functions, triggers, etc
Managing Database Files
• Removing a database deletes the files – Once a database has been deleted, it cannot be un-deleted! • Edit -> Undo will not help!
• You cannot delete a database that is in use
Managing Database Files
• To delete a database in the GUI, right click on the database and select Delete • To delete a database using Transact-SQL: DROP DATABASE MyDb
Dropping a database
Managing Database Files
• To protect ourselves from dropping databases, we employ two strategies: – Backups – Scripts
• We’ll look at backups in an upcoming chapter • Let’s look at scripts now!
SQL Server Scripts
• Sql scripts can do almost anything – Create, alter, drop databases – Create, alter, drop tables, functions, stored procs – Insert, update, delete rows
• All “real” DBAs use scripts • Scripts are typically stored in .sql files
SQL Server Scripts
• Scripts can be executed – In SSMS – Using sqlcmd.exe – Inside of any custom or 3rd party app that can connect to SQL Server – In batch files (that call sqlcmd) – SQL Server Management Objects (SMO) – PowerShell • Which can handle it’s own scripts and sql scripts!
SQL Server Scripts
• You can also generate scripts – Create, alter, drop databases – Create, alter, drop tables, functions, stored procs – Insert, update, delete rows
• Script generation can be done – In SSMS – “By Hand”
SQL Server Scripts
• Let’s demo!
Managing Database Files
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Managing Database Files
• Shrinking databases is a sometime necessity • You can shrink – Individual data files or log files – An entire database (all files)
• Shrinking can be done with the GUI, TSQL, or any of the other methods
Managing Database Files
Shrinking a Database
• There are two options for shrinking an entire database 1. Remove all free space after the last page • PROs: Fastest, least impact • CONs: Not likely to remove as much as the 2nd option
Shrinking a Database
• There are two options for shrinking an entire database (cont.) 2. Reorganize the entire database to compress the pages at the “front” of the file, then remove free space • PROs: Removes maximum amount of free space, can specificy a “Target Percentage” of free space (“I want 10% free space left in file”) • CONs: Slow, performance hog
Managing Database Files
Shrinking a Database
• To shrink a database using Transact SQL, use DBCC SHRINKDATABASE DBCC SHRINKDATABASE ( database_name [ , target_percent ] [ , { NOTRUNCATE | TRUNCATEONLY } ] )
Shrinking a Database
• NOTRUNCATE – Compress pages to “front” of file – Does not release freed space to OS – Only available in T-SQL
• TRUNCATEONLY – Does not compress the pages to the front of the file – Releases freed space after the “last” page to the OS – Default in SSMS
Shrinking a Database
-- TRUNCATEONLY: DBCC SHRINKDATABASE(N'LearnItFirst.com') -- TARGET SIZE of 10% free space: DBCC SHRINKDATABASE(N'LearnItFirst.com‘,10) -- NOTRUNCATE: DBCC SHRINKDATABASE(N'LearnItFirst.com', NOTRUNCATE)
Shrinking a Database
• Shrinking individual files offers more flexibility – Can specify target sizes, truncation options or empty individual files DBCC SHRINKFILE ( { logical_file_name | file_id } { [ , EMPTYFILE ] | [[ , target_size ][ , {NOTRUNCATE | TRUNCATEONLY }]]} )
Shrinking a Database
• EMPTYFILE – Moves pages to another file in the file group
• target_size – Cannot compress data – Defaults to MB – If not specified, uses the creation size
Shrinking a Database
• Let’s demo!
Suggested Changes After the Upgrade Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Testing Acceptance/sign-off
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade: Patching
• It’s likely that, after your upgrade, there are hotfixes, patches, service packs, etc available • It is often difficult to determine which is needed – How do you know you are experiencing a bug until you test your server?
After the Upgrade: Patching
• Remember that: – Service packs are the most-tested – Cumulative Updates are the next-most-tested – Critical Updates are the least-tested
• You don’t have to install the most recent updates – Unless there is a specific security issue
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade: Compatibility
• The first and most obvious thing to understand is compatibility level • SQL Server allows a database to use one of three compatibility levels – 80 (SQL Server 2000) – 90 (SQL Server 2005) – 100 (SQL Server 2008)
After the Upgrade: Compatibility
• Changing a database’s compatibility level provides partial backward compatibility – Setting a database’s compatibility level to 80 makes that database behave as though it was on a SQL Server 2000 instance – Allows you to upgrade to SQL Server 2008 without making breaking application changes
After the Upgrade: Compatibility
• When you upgrade a database/instance, the previous version remains as the compatibility level – If you upgrade “Northwind” from SQL Server 2000 to 2008, the Northwind database remains in SQL Server 2000 compatibility level • All other databases on SQL Server 2008 are in SQL Server 2008 compatibility level
After the Upgrade: Compatibility
• Let’s demo!
Suggested Changes After the Upgrade Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• If you upgraded from SQL Server 2005 – There are no database-level configuration changes needed
• If you upgraded from SQL Server 2000, you might want to make changes
After the Upgrade
ALTER DATABASE ‘AdventureWorks’ SET ANSI_NULL_DEFAULT ON , ANSI_NULLS ON , ANSI_PADDING ON , ANSI_WARNINGS ON , ARITHABORT ON , AUTO_CREATE_STATISTICS ON , AUTO_UPDATE_STATISTICS ON , NUMERIC_ROUNDABORT ON , PAGE_VERIFY CHECKSUM
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• What should you test? – Have the developers unit test any applications using the database • Not always possible with 3rd party apps
– Reports should be verified to ensure that they run – SQL Server jobs should be run to ensure they still “work” as expected – DTS and/or SSIS packages
After the Upgrade
• Known trouble areas – SQL Server jobs • If jobs point to specific files/folders, verify that – SQL Server service(s) have access – The file/folder paths have not changed or, if they have, change job definition
– DTS to SSIS – Database compatibility
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• Once the DBA has completed his/her testing, recruit other departments – Developers test “plumbing” – Quality Assurance (“QA”) tests application compatibility – System administrators test backup routines
The SQL Server 2008 Sample Databases Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
The SQL Server 2008 Sample Databases
• SQL Server 2008 does not ship with any sample databases – SQL Server 6.x shipped with the pubs database – SQL Server 7.0 and 2000 shipped with pubs and Northwind – SQL Server 2005 shipped with AdventureWorks and AdventureWorksDW
The SQL Server 2008 Sample Databases
• There are sample databases for SQL Server 2008 – You must be smart enough to figure out • • • •
What they are called How to download them How to install them Which to use for what purpose
The SQL Server 2008 Sample Databases
• Let’s answer each question individually : – What they are called • The sample databases are called collectively “AdventureWorks” • They are the sample databases for the fictitious AdventureWorks Cycling Company
What is AdventureWorks
• The AdventureWorks Cycling Company – Sells through internet and brick & mortar channels – Covers commonly used patterns for working with products, sales, contact management and more
The SQL Server 2008 Sample Databases
• AdventureWorks stores roughly three years worth of sales data in: – An OLTP database – An OLAP database
The SQL Server 2008 Sample Databases
• How do you get the sample databases? – The files are available at http://www.codeplex.com/MSFTDBProdSamples – Shameless marketing from MSFT
The SQL Server 2008 Sample Databases
• How do you get them onto your SQL Server? – One of three ways: • Install from .msi • Restore from backup • Run the scripts
– Good luck!
The SQL Server 2008 Sample Databases
• How do you know which to use for what purpose? – There are three different types of databases: • OLTP, OLAP and a “Light” version of the OLTP
The SQL Server 2008 Sample Databases
• This is where it gets confusing… – Each database has several names depending on where you look – There are as many as three different “versions” of each database in the default installation – There are many different files you can download once you actually figure out where to look for it
The SQL Server 2008 Sample Databases
• A look at the default install using .msi installation:
The SQL Server 2008 Sample Databases
• On the CodePlex website, they make mention of three databases in the AdventureWorks suite: – AdventureWorks OLTP – AdventureWorks OLAP – AdventureWorks LT (Light
The SQL Server 2008 Sample Databases
• AdventureWorks OLTP actually goes by many names – AdventureWorks OLTP in documentation/website – AdventureWorks OLTP and AdventureWorks 2008 OLTP in the downloaded .zip file – AdventureWorks and AdventureWorks2008 in installation
The SQL Server 2008 Sample Databases
• AdventureWorks OLAP also goes by many names – AdventureWorks OLAP in documentation/website – AdventureWorks 2008 Data Warehouse in the downloaded .zip file – AdventureWorksDW and AdventureWorksDW2008 in installation
The SQL Server 2008 Sample Databases
• AdventureWorks LT (Light) is the fewertables version of AdventureWorks OLTP – Meant to be simpler to learn and train people on
The SQL Server 2008 Sample Databases
• Which should I use when? – “I’m a total beginner to SQL Server” • AdventureWorksLT
– “I want to learn about the new datatypes and/or new features in SQL Server 2008” • AdventureWorks2008
– “I want to learn about data warehousing” • AdventureWorksDW
– “I want to learn about the new features in SQL Server 2008 data warehousing” • AdventureWorks2008DW
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
What We’re Going to Cover
• Encryption overview • Linked servers • SIDs and User accounts
Chapter Overview
• This chapter focuses on – How to create and manage logins – How to design and manage database security – Understanding Transparent Data Encryption – Creating and managing Linked Servers
Chapter Overview
• SQL Server Security is not that difficult – Understanding the main security model is easy – Remembering what each term does is easy – Understanding what to do when is easy once explained
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
What We’re Going to Cover
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Security
• A primary principle of security is to “reduce the attack surface area” • Another main focus is to only allow the least amount of privileges needed to do a task • This chapter teaches how to do both
SQL Server Security
• There are many levels of security in SQL Server – Physical machine – Operating System – Network – Instance – Database
SQL Server Security
• Physical Machine – Who has access to the actual machine? • Hard drives in many servers can easily be removed
– Recommendations to reduce attack surface area: • Server in a locked room with access limited to systems admins only
SQL Server Security
• Operating System – Should you run other software on this machine? • Internet Information Server (IIS), Exchange, SharePoint
– Which users have logins to the server? • If they can login to the server, can they make a file copy of SQL Server databases?
– Recommendations to reduce attack surface area: • The more critical/sensitive/large the database(s), the more likely you need a dedicated machine • Only allow system administrators and DBAs to login to the operating system
SQL Server Security
• Network – By changing default network settings you can reduce the surface area • Changing default instance TCP port from 1433 to another value
– Recommendations to reduce attack surface area: • Place your web SQL Server behind a firewall to prevent remote access (if possible) • Consider allowing local-only connections for applications installed on the server
SQL Server Security
• Instance-level – Defines who can login to the SQL Server instance and what permissions they have instance-wide – sa has instance-wide administrator privileges – Anyone who can login to the SQL Server is called a “login”
SQL Server Security
• Database-level – Defines which logins are able to use this database and what permissions each has in that database – By default, database permissions are granular • Permissions in “AdventureWorks” do not mean a user has those same permissions in another database
SQL Server Security
• Let’s look in-depth at instance-level and database-level security next!
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video…
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Security
• Something you need to know before we begin: – Permissions are GRANT, DENY and REVOKE • REVOKE is like an eraser • DENY cannot be overturned • One GRANT is all it takes
SQL Server Security
• Other basic terms to know are: – Logins – Users – Roles – Principals
• Let’s do a quick definition of each!
SQL Server Security
• New term: “Login” – To use a SQL Server, you must have a login – The “sa” login has full rights to the instance – Having a login does not necessarily give you access to a database
SQL Server Security
• New term: “User” – To use a database, you must have a user account • For brevity, just called a “user”
– Just having a user account does not mean you can access a particular table
SQL Server Security
• New term: “Role” – Logins or users can be grouped into a “Role” and then permissions granted to the role • Called “Groups” in Windows and other systems
– There are server roles (instance-level) and database roles – SQL Server combines login/user/role level permissions when evaluating requests
SQL Server Security
• New term: “Principal” – “Principals are entities that can request SQL Server resources”1 • A resource is anything from logging in to manipulating data
– A principal is an abstraction layer that encompasses users, roles, and logins • There are server principals (logins, server roles) and database principals (users, database roles) 1 – Source: SQL Server Books Online
SQL Server Security
• Why the abstraction layer? – Instance-wide permissions can be granted: • To an individual login • To a role that the login belongs to
– Remember that SQL Server combines the login, user and role permissions to determine whether a user is GRANT or DENY • A single DENY prevents access
SQL Server Security
• Why the abstraction layer?
(cont.)
– A “Login” can be created from: • Windows Group – All members of the group are granted/denied access
• Windows User account – A single Windows User account is granted/denied access
• SQL Server Authentication login – The DBA creates a login/password and grants/denies access
SQL Server Security
• Why the abstraction layer?
(cont.)
– By using the generic term “principal” when talking about permissions, it encompasses roles as well as logins of all types – Easier to say “Make sure to DENY all principals” than to say “Make sure you DENY John, Mark, Jane, the CustomerSupport role, and the public role”
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video…
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• The SQL Server DBA is like the door person at an exclusive, invitation-only party – To enter requires your name being on “the list” – If your name isn’t on the list, you can’t get in!
• “The List”, in SQL Server terms, is the list of logins
SQL Server Logins
• A login can be created from: – Windows User account – Windows Group – SQL Server login
• Logins can be granted or denied access
SQL Server Logins
• Example scenario: – The Windows Group “SQLDevs” is granted access – Chad is a member of SQLDevs – Chad’s latest code check-in caused major data problems – What are our options to prevent Chad from logging in?
SQL Server Logins
• Solution #1: Remove Chad’s Windows User account from the Windows group – PROs: • Problem solved quickly and easily
– CONs: • The Windows Group may have other necessary permissions outside of SQL Server • SQL Server DBA may not have rights to Active Directory
SQL Server Logins
• Solution #2: 1. Add Chad’s Windows User account as a login 2. Deny Chad’s Windows User login access 3. Now all other SQLDevs except Chad can login
• PROs – Does not affect Chad’s other Windows permissions outside of SQL Server – SQL Server DBA can implement regardless of Active Directory access
• CONs – Requires an extra step
SQL Server Logins
• Let’s demo!
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• SQL Server authentication logins are used when a user/application does not have a Windows account – A 3rd party application uses a custom login
• The “sa” account is a SQL Server login
SQL Server Logins
• To use SQL Server logins requires Mixed Mode – Changing authentication mode requires restarting SQL Server
SQL Server Logins
• SQL Server logins can use password policies of the computer/domain
SQL Server Logins
• Administrative Tools -> Local Security Policy
SQL Server Logins
• There are other logins as well for specific situations – Certificate-based logins and logins mapped to an asymmetric key • Used to sign routines for extra/elevated security
– Logins mapped to a credential • Credentials are simply a fancy database name for a Windows user/password
SQL Server Logins
• All logins have – A default database – Default language for error messages – Server role membership(s) – User mapping – Securables – Status
• Let’s look at each in the next video!
SQL Server Logins
• But first – demo!
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server Security
• The default database is the database the user is placed in upon login unless the user requests a different database • The default language specifies the language of error messages and functions
SQL Server Security
• Server roles allow for instance-wide grouping of permissions – Inflexible • Permissions cannot be changed • Cannot create your own roles
– To use, put logins in and take them out
SQL Server Security
• sysadmin role has all permissions – “sa” login is a member of sysadmin
• dbcreator sounds good but…. – Members can DROP any database on the server – (Documentation does not list that little goodie…)
SQL Server Security
• Be careful - each server role has the ability to add other logins to it! – If I add Chad to the “dbcreator” role he can add others to it
• Suggestion: Grant permissions in the Securables tab (upcoming)
SQL Server Logins
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server Security
• A login must be made a user of a database upon creation – Unless it is a member of sysadmin server role
• User accounts coming up in a future video…
In the next video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server 2008 Security
Presented by Scott Whigham
In this video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
Securables
• First thing: SQL Server uses the GDR for permissions – GRANT – DENY – REVOKE (i.e. The Eraser)
Securables
• Securables are another abstraction layer – Securables are simply things that you define permissions to • Tables, views, etc
• You define permissions on securables to principals
Securables
• Securables have “scope” – Server • Database, Login
– Database • User, role, schema
– Schema • Table, view, procedure, function
Securables
Securables
• For each of these, there are three basic permissions – CONTROL – ALTER – VIEW DEFINITION • Each level can have individual permissions as well – Logins can have IMPERSONATE permission – TABLES have SELECT permission
Securables
• CONTROL permission effectively gives ownership rights – Includes the ability to give others rights as well -- Give the login Chad sysadmin-level rights:
GRANT CONTROL SERVER to Chad
Securables
• ALTER permission gives ability to modify – ALTER ANY <entity> • Permission applied for all entities of that entity type
– ALTER < entity> • Permission applied for only the given <securable>
Securables -- Give the login Terry the ability to change -Chrissy’s password
GRANT ALTER ON LOGIN::Chrissy TO Terry -- Give Chrissy the option to change -- anyone’s password GRANT ALTER ANY LOGIN TO Chrissy -- Yikes!!!!!
Securables
• VIEW DEFINITION permission allows for viewing: – Source code if given to stored procedures, functions, triggers, views – Login names/info if given at Login level – Table metadata if given at table level
Securables -- Give the login Terry the ability to view -information about Chrissy’s login GRANT VIEW DEFINITION ON LOGIN::Chrissy TO TERRY
-- Give Chrissy the option to view the source -code for a view vw_Customer GRANT VIEW DEFINITION ON vw_Customer TO Chrissy
Securables
• Each permission can also be granted using the WITH GRANT option – If I give Chad the VIEW DEFINITION permission on a stored procedure, he can then give that to Chrissy
Alternative to db_creator role
• Remember our problem with db_creator? – Any member can now ALTER or DROP any user database
• It is recommended that you do not use db_creator
Alternative to db_creator role
• To give logins the ability to create a database without giving them “extra” permissions, grant them the server permission “Create any database”
Alternative to db_creator role
Alternative to db_creator role
Alternative to db_creator role
Securables
• Let’s demo the server-scoped securables
SQL Server 2008 Security
Presented by Scott Whigham
In this video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
Status
• Logins can be: – Disabled or Enabled – Grant or Deny – Locked out!
A most confusing screenshot…
Account Status
• Logins that are “Disabled” or have “Deny” cannot login – No functional difference
• If you “Deny”: DENY CONNECT SQL to [login]
• If you “Disable”: ALTER LOGIN [login] DISABLE
Status
• Disabling or denying logins: – Allows you to deny access to logins that you may later need to restore access • Consultants, staff on vacation or out of the office
– Allows you to deny access to creators of objects (databases, tables, etc) yet still let others use those objects • Alternative would be to transfer object ownership to other login and then drop the login
Locked out!
• A SQL Server authentication login can be locked out due to policies
Locked out!
• There are two ways to reset a locked-out account: – By resetting the password – By resetting the lockout time
• It is recommended that you reset lockedout accounts by resetting the password – Resetting the lockout time will clear any password history associated with the account!
Locked out!
• To reset a locked-out account by resetting the password, use ALTER LOGIN ALTER LOGIN [Scott] WITH PASSWORD=‘e%3I' UNLOCK
Locked out!
• To reset a locked-out account without resetting the password, use ALTER LOGIN … WITH CHECK_POLICY ALTER LOGIN [Scott] WITH CHECK_POLICY = OFF GO ALTER LOGIN [Scott] WITH CHECK_POLICY = ON GO -- Account may now login using existing password
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Accounts Added by SQL Server
• When you install SQL Server, there are several accounts added by default – SQL Server authentication accounts – Windows authentication accounts – Certificate-based logins
SQL Server accounts added
• sa – Full permissions – Can be disabled – Only available in Mixed mode – Recommendation: DON’T USE IT • Create Windows authentication logins for your sysadmins instead
Certificate-based accounts added
• Logins that are surrounded by the double hash signs are Certificate-based SQL Server logins – – – –
##MS_SQLReplicationSigningCertificate## ##MS_PolicyEventProcessingLogin## ##MS_PolicySigningCertificate## ##MS_PolicyTsqlExecutionLogin## (and several more)
• These get installed depending on which features you have installed
Windows-based accounts added
• SQL Server 2008 also adds several Windows-based logins – NT AUTHORITY\SYSTEM – an internal, default Windows group that is granted sysadmin privilege • Do not modify this account’s privileges • Used by Microsoft Update, service packs, hotfixes, and the SQL Writer Service (and more) • For more information, view KB article 932881
Windows-based accounts added
• SQL Server 2008 creates a local Windows group for each service that you install • These groups are subsequently granted SQL Server logins and given sysadmin privilege
Windows-based accounts added
• For a default install of SQL Server, there are two critical Windows Groups installed: – SQLServerMSSQLUser$<machine_name>$ – SQLServerSQLAgentUser$<machine_name>$
• Remember: Instance IDs for a default instance are MSSQLSERVER
Windows-based accounts added
• For the server named LEARNITFIRST with a default instance of SQL 2008: – SQLServerMSSQLUser$LEARNITFIRST$MSSQLSERVER – SQLServerSQLAgentUser$LEARNITFIRST$MSSQLSERVER
• For a server named COLTRANE with a named instance of SQL Server with an instance ID of “Development”: – SQLServerMSSQLUser$COLTRANE$DEVELOPMENT – SQLServerSQLAgentUser$COLTRANE$DEVELOPMENT
Windows-based accounts added
• SQLServerMSSQLUser group: – Any user in this group can login to SQL Server as a sysadmin – Also has the full privileges of the SQL Server service account – No need to directly edit this group; SQL Server will do the right thing when you change service accounts through the SQL Server Configuration Manager
Windows-based accounts added
• In SQL Server, you will see these new accounts listed as: – NT SERVICE\ – NT SERVICE\SQLAgent
Account Provisioning
• Any accounts that you provision to be system administrators during installation will become sysadmins – Make sure to provision your Windows account during installation!
Account Provisioning
Changes in security
• Unlike previous editions, operating system and domain administrators are not sysadmins in SQL Server by default – BUILTIN\Administrators is no longer added by default – Can be added manually if necessary
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Users
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Database-Level Security
• A login allows access to the SQL Server • A user account is required to access a database • Remember: logins and users are actually “principals”
Database-Level Security
• You, the DBA, can grant user accounts: – When creating the login – After the login has been created – Using SSMS’s GUI tools – Using T-SQL’s CREATE USER command
Database-Level Security
• There are four built-in user accounts – dbo • Full privileges; the person who creates the database is the dbo by default unless changed • Sysadmins are dbo of every database
– Guest • Disabled by default
– INFORMATION_SCHEMA • Internal
– Sys • Internal
Database-Level Security
• A login has no user accounts created for it by default – If you enable Guest in a database, this means that any login can use that database even without a user account – Guest is disabled by default; It is not a standard practice to enable Guest
Database-Level Security
• A login may have zero or one user account per database – A login without a user account in a database cannot access that database unless the Guest account is enabled
Database-Level Security
• User names do not have to match login names – Let’s look at the login ScottWhigham: Database
User mapping
master
guest
model
guest
msdb
ScottWhigham
tempdb
guest
AdventureWorks
<none>
ScottsDatabase
dbo
WebsiteMarketing Scott
Database-Level Security
• A login may have zero or one user account per database – A login without a user account in a database cannot access that database
Database-Level Security
• Imagine your SQL Server as a 100-floor skyscraper – Huge foyer complete with menacing looking security guards – Unlike most skyscrapers, you must show your key card (i.e. your credentials) to enter the foyer Image by wallyg at flikr.com
Database-Level Security
• Think of the “secure” foyer as the master database – Anyone who has access to any floor of the skyscraper is allowed in through the turnstiles and into the foyer – Certain staff are only allowed into the foyer
• In SQL Server, this is called a login
Database-Level Security
• The elevators of our skyscraper require a key card – This key card determines which floor(s) you have access to – You may only go to floors that your key card specifically allows Image by SYangPhoto at flikr.com
Database-Level Security
• In SQL Server parlance, the “floor” is really a database – Logins may not use a database unless they have a user account – A user account in one database has no effect on whether a login may access another database • Example: your key card lets you go to the 32nd floor. Would you also expect that key card to allow you permission to the 23rd floor? No!
Database-Level Security
• Let’s demo!
SQL Server 2008 Security: Schemas
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles and Schemas
• The word schema has multiple definitions in SQL Server: – The structure of a database including tables, columns, and relationships – An XML schema collection – A container
Roles and Schemas
• The word schema has multiple definitions in SQL Server: – The structure of a database including tables, columns, and relationships – An XML schema collection – A container
Roles and Schemas
• DBAs and Developers place objects in schemas for – Organization of objects – Security of objects
HumanResources schema
• Organization: • HumanResources schema in AdventureWorks contains all objects containing information about the Human Resources department
Schemas and name resolution
• When you request an object, SQL Server allows up to four “levels”: SELECT * FROM Server.Database.Schema.Object SELECT * FROM Jupiter.LearnItFirst.dbo.Customer
Schemas and name resolution
• If you are on the same server, you can skip the server name -- Logged on to Jupiter server: SELECT * FROM LearnItFirst.dbo.Customer
Schemas and name resolution
• If you are already in the correct database, you can skip the database -- Logged on to Jupiter server and using LearnItFirst:
SELECT * FROM dbo.Customer
Schemas and name resolution
• How would you query a table in the WebMarketing database while you were in the LearnItFirst database? _________________________________________
Schemas and name resolution
• How would you query a table in the WebMarketing database while you were in the LearnItFirst database? SELECT * FROM WebMarketing.dbo.Log
Schemas and name resolution
• If the default schema for your user account is the same as your desired object’s schema, you can omit the schema: -- Logged on to Jupiter server, in LearnItFirst db, and default schema is “dbo”: SELECT * FROM Customer -- Warning: not best practice
Schemas and name resolution
• We’ll cover more on schemas and name resolution later in the chapter
Schemas
• Let’s demo!
SQL Server 2008 Security: Roles
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles (covered already) – Database-level roles – Application Roles
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles (covered already) – Database-level roles – Application Roles
Database-level Roles
• Database-level roles allow DBAs to group users together and assign permissions to the role – Far superior to doing user-level permissions – Fixed database roles = the built-in roles – Flexible database roles = the ones you create
Database-level Roles
• You can use the built-in database roles as well as creating your own • Some commonly used built-in roles are: Role
Description
db_owner
Full rights to database
public
Any user is a member of public; sets base permissions for all connections
db_ddladmin
Create objects of any type; can alter or drop objects they created
db_datawriter
Can modify the data any table
db_datareader
Can read the data from any table
db_securityadmin
Manage role memberships and permissions
Database-level Roles
• Example of using roles: – Your database has 10 users – Your security assessment shows that there are really three types of permissions needed: • Full control (Micah and Thomas) • Development work (Sam and Nica) • Read-only access to all tables but modify to dbo.Customer table (all others)
Database-level Roles User
Permission
Micah, Thomas
Full control
Sam, Nica
Developers
All others
Read all tables; modify dbo.Customer
• Option 1: Assign each permission to each individual user – Pros: easy to understand when creating – Cons: repetitive, difficult to maintain, silly
Database-level Roles User
Permission
Micah, Thomas
Full control
Sam, Nica
Developers
All others
Read all tables; modify dbo.Customer
• Option 2: Create roles for each group – Pros: easy to understand when creating, easy to assign permissions, easy to add/remove – Cons: ?
Database-level Roles
Login
User
Role
Micah
Micah
public, db_owner
Thomas
Thomas
public, db_owner
Sam
Sam
public, db_ddladmin, Developers
Nica
Nica
public, db_ddladmin, Developers
Five other logins
Five users, one for each login
public, db_datareader, CustomerManagers
Roles and Schemas
• Notice that roles can be given the same permissions that users can
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Roles
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles – Database-level roles – Application Roles
Roles
• Application roles allow you to tie a specific application to certain permissions in the database – Step 1: User launches application – Step 2: Application logs into SQL Server – Step 3: Application uses a database – Step 4: App sets or invokes the application role • The application has now lost the user perms and now has the app role perms
Roles
• To use app roles, you still must have: – Login – User
• You can create app roles in SSMS or TSQL
Roles
Roles
• For an application to use the app role, it must know the password • The stored procedure sp_setapprole is how you invoke or set the app role • sp_unsetapprole can be used to restore the original security contexts
Roles
Roles
Roles
• Application roles are for specific situations – For stateful apps, not stateless apps like websites – Useful for kiosk-style apps in which users will not have their own logins – Useful for apps that bypass SQL Server users and create their own table-based security system – Most people will not ever encounter app roles
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
The GDR
• As we saw earlier, permissions are given using the GDR – GRANT – DENY – REVOKE
The SQL Server Security Model
• Deny takes precedence • User cannot perform an action unless they have been granted that permission • Users inherit permissions from roles • It only takes one DENY to prevent!
The SQL Server Security Model
• Micah is a user of the WebsiteMarketing db – public has DENY SELECT on dbo.Log – Micah belongs to db_datareader – Micah’s user account: • GRANT SELECT on dbo.Log • GRANT UPDATE on dbo.Customer
• What are Micah’s effective permissions?
The SQL Server Security Model
• Micah’s effective permissions are: – Read all tables except dbo.Log – Update dbo.Customer
• Micah is a user of the WebsiteMarketing db – public has DENY SELECT on dbo.Log – Micah belongs to db_datareader – Micah’s user account: • GRANT SELECT on dbo.Log • GRANT UPDATE on dbo.Customer
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
Server scoped permissions
• Server scoped permissions – Apply to the connection and are always “on” -- Give the login Chad sysadmin-level rights:
GRANT CONTROL SERVER to Chad -- Give Chrissy the option to change -- anyone’s password GRANT ALTER ANY LOGIN TO Chrissy
Server scoped permissions
Permission
Description
Administer bulk operations
Required to perform bulk operations
Alter any login
Change anyone’s password, et al
Control server
Sysadmin privileges
Create any database
Allows non-sysadmins to create databases
Shutdown
Uhhhhhhhhh
View any database
Can see metadata about every database; PUBLIC has this permission by default
View any definition
Discussed on next slide
View server state
Discussed on next slide
• Server roles often include these •
Adding a login to sysadmin grants “Control server”
Server scoped permissions
• In SQL Server 7.0 and SQL Server 2000, users were allowed to view the metadata about every object in the database -- Return metadata about every object in SQL 2000:
SELECT * FROM sysobjects
Server scoped permissions
• Starting with SQL Server 2005, you must enable permissions: -- Returns only objects that user has perms defined for:
SELECT * FROM sys.objects
Server scoped permissions
• If you need backward compatibility, then you need to GRANT server scoped permissions -- So that any user can view objects in db:
GRANT VIEW ANY DEFINITION to public -- Give users permissions to see health of server
GRANT VIEW SERVER STATE to public
Server scoped permissions
• Please! Do not issue these statements on a production server unless you know that need them! – These changes in security reflect changes to the SQL Server security model that were put in place for a reason
Roles
• When you use SSMS’ GUI to manage permissions, behind the scenes it will execute GDR statements to perform the requested action
Roles
• The easiest way to learn the Transact-SQL statements that DBAs use is to use the Script button to view the statements that SSMS will execute
Roles
• (We’ll do a full Transact SQL review later)
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Database Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
Database scoped permissions
• Database scoped permissions – Apply to objects in the database only – Can assign permissions to: • Users • Roles • Schemas
Admin-related permissions
Permission
Description
Alter any application role
Change passwords
Alter any role
Create/alter/drop app roles, modify privileges
Alter any schema
Create/alter/drop schemas, modify perms, change object ownership
Alter any user
Create/alter/drop users, rename user; change default schema
Backup database
Can perform FULL and DIFFERENTIAL backups
Backup log
Can perform LOG backups
Control
Same permission as dbo
View database state
Required to view dynamic management views (DMVs) related to database state
Developer-related permissions
Permission
Description
Create assembly
Load new assemblies into database
Create function
Can create functions in any schema unless specifically denied
Create procedure
Can create stored procedures in any schema unless specifically denied
Create table
Can create tables in any schema unless specifically denied
Create view
Can create views in any schema unless specifically denied
Select
Same as placing in db_datareader
Insert, Update, Delete
Same as db_datawriter
Database scoped permissions
• Many user-level and flexible role permissions can be handled with the fixed database roles – “DENY Insert, Update, Delete…” is equivalent to adding user/role to db_denydatawriter role – Giving a user create function, view, proc, and table is the same as db_ddladmin
Database scoped permissions
• Roles can be nested – Step 1: Create DbAdmins flexible database role – Step 2: Make DbAdmins members of db_ddladmin
• The DbAdmins now have the same permissions as db_ddladmin – This may not make much sense yet…
Transact SQL
• Give the user Chrissy the ability to create functions -- Always make sure you are in the right db!
USE[LearnItFirst.com] GO GRANT CREATE FUNCTION TO [Chrissy]
Transact SQL
• Remove Chrissy’s ability to create functions REVOKE CREATE FUNCTION FROM [Chrissy] -- Chrissy now has no permissions
Transact SQL
• Prevent Chrissy from ever being able to create functions DENY CREATE FUNCTION TO [Chrissy]
Exercise
• What is the best way to give a user the ability to read all tables in the WebsiteMarketing db? _________________________________________ • Can you think of at least two other alternatives? _________________________________________
Exercise
• What is the best way to give all users the ability to read all tables in the WebsiteMarketing db? – Add user to db_datareader
• Can you think of at least two other alternatives? – Grant SELECT TABLE as database permission – Manually GRANT SELECT on each table in database – Grant SELECT on each table public role
Up next…
• Enough lecture – let’s demo!
SQL Server 2008 Security: Database Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
What is a schema again?
• Schemas are containers – For convenience – Addictive
• Schemas are used in name resolution SELECT * FROM Server.Database.Schema.Object
Schema scoped permissions
• Schema-scoped permissions – Apply to objects in the schema only – Can assign permissions to: • Tables, views, procedures, functions and much more
• A schema is a database-level securable – Permissions on a schema in one database have no effect on a user’s permissions in another database
Schema level permissions
• If you have a schema-level permission, you have that permission for all objects contained within that schema Permission
Meaning
CONTROL
User has owner-like privileges on all objects in schema
ALTER
User may alter all objects in schema
INSERT, UPDATE, DELETE, SELECT
User may perform these operations (specified individually) on all objects in schema
VIEW DEFINITION
User may view definition (i.e. metadata) on all objects in schema
Roles and Schemas
• DBAs can secure objects at the schema level • DBAs can GRANT/DENY permissions on all objects in schema to users
Up next…
• Impersonation and EXECUTE AS • Demos
SQL Server 2008 Security: Impersonation and EXECUTE AS Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Impersonation and EXECUTE AS
• Before we get into deep demo mode, you need to understand impersonation at a high level – Impersonation allows you to change the execution context – This means you can execute statements as another login or another user without physically logging in as that principal
Impersonation and EXECUTE AS
• We use the EXECUTE AS command to perform impersonation – You can impersonate a login or a user – Requires the IMPERSONATE permission
• We use REVERT to switch back to our login
Impersonation and EXECUTE AS
• Sysadmins have impersonate permission for all logins and users – This means that the ‘sa’ can execute statements as any login or user
• Normal users cannot perform impersonation by default – You must grant them IMPERSONATE permission on a specific login or user
Granting login impersonation
-- We are logged in as „sa‟ GRANT IMPERSONATE ON Login::[LearnItFirst] TO [Chrissy] -- Chrissy is a login GO
Granting user impersonation
-- We are logged in as „sa‟ USE [LearnItFirst.com] GO GRANT IMPERSONATE ON User::[LIF] TO [Chrissy] -- Chrissy is a user GO
EXECUTE AS syntax { EXEC | EXECUTE ] AS [;] ::= { LOGIN | USER } = 'name' [ WITH { NO REVERT | COOKIE INTO @varbinary_variable } ] | CALLER
– NO REVERT – once activated, cannot be undone using REVERT
Impersonation and EXECUTE AS
• There are two types of context switching – Server-level (impersonating a login) • EXECUTE AS LOGIN = ‘NewLogin’
– Database-level (impersonating a user) • EXECUTE AS USER = ‘NewUser’
Impersonation and EXECUTE AS
• Server-level impersonation: – Once you switch, you are no longer you; you are the login you are now impersonating – Use REVERT to return to your original context – Can impersonate while impersonating! -- Useful T-SQL functions for server-level impersonation: SELECT SUSER_SNAME() AS CurrentLogin, ORIGINAL_LOGIN() AS OriginalLogin
Impersonation and EXECUTE AS
• Database-level impersonation: – Once you switch, you are no longer you; you are the user you are now impersonating – Use REVERT to return to your original context – Can also impersonate while impersonating
• One “gotcha”: once triggered, you cannot access objects from databases that do not have a guest account
Impersonation and EXECUTE AS -- Useful T-SQL functions for database-level impersonation: SELECT SUSER_SNAME() AS CurrentLogin, USER_NAME() AS CurrentUser
Impersonation and EXECUTE AS
• After we discuss object ownership, we will see there is another level of impersonation
Up next…
• Let’s do a few demos showing permissions, roles, users, and context switching!
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Ownership
• In SQL Server, we talk about database ownership, object ownership, and schema ownership – The owner has full rights
Database Ownership
• Database ownership: – The database owner is a login – The dbo user is the database owner – By placing users in the db_owner role, they have similar permissions • However, the dbo can DENY privileges to the members of the db_owner role
Cross database ownership chaining
• “Cross database ownership chaining” (‘CDOC’) allows database owners of one database to act as owners of other databases without having specific user access
Cross database ownership chaining
• “Cross database ownership chaining” is both a server setting and a database setting – Set it at the server-level to enable for entire instance – Set it at the database-level to allow specific databases to participate
Cross database ownership chaining
• “Cross database ownership chaining” is not recommended – Disabled by default – “… should not be enabled unless it is specifically required.” – Source: Books Online
Schema Ownership
• Schema ownership is extremely important – Remember that all objects in a database are contained within a schema – The schema owner has full rights on all objects in that schema
Object Ownership
• Object ownership: – You can grant CONTROL on certain objects within a schema instead of granting permission on the entire schema
Object Ownership
• Transferring ownership is easy – Can transfer schema ownership – Can transfer object ownership
• Can use Transact-SQL (recommended) or the SSMS GUI screens (lots of clicks!)
Object Ownership
• Example #1 – Terry owns the “Sales” schema – Terry leaves the company and we hire Chad to replace him – We transfer ownership of the “Sales” schema to Chad
Object Ownership
• Example #2 – Chrissy owns a table in the “Sales” schema named “Prospects” – Chad owns the schema – We hire Candy to replace Chrissy – We grant CONTROL on “Sales.Prospects” to Candy – We remove Chrissy!
In the next videos…
• Examples and scenarios • Ownership chains • How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• Let’s walk through a few examples on object ownership first – Remember that the owner has full rights
Example #1
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Can Chad SELECT from dbo.Merchant? – ___________________
• Can Chad execute dbo.AddMerchant? – ___________________
• Why is this? – ___________________
Example #1
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Can Chad SELECT from dbo.Merchant? – Yes!
• Can Chad execute dbo.AddMerchant? – Yes!
• Why is this? – He is the dbo and has full rights to the database
Example #2
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #2
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – No
• Why or why not? – Her effective permission does not contain a GRANT
Example #3
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #3
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – Yes
• Why or why not? – Her effective permission contains a GRANT with no DENY
Example #4
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: member of db_datareader • Chrissy: DENY SELECT ON SCHEMA::[dbo] • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #4
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: member of db_datareader • Chrissy: DENY SELECT ON SCHEMA::[dbo] • Can Chrissy SELECT dbo.Merchant? – No
• Why or why not? – Her effective permission contains a DENY
Object Ownership
• Clear cut examples like the previous ones make things seem easy • What happens when there are multiple objects and multiple permissions?
Example #5
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: DENY SELECT ON dbo.Merchant • Chrissy: GRANT SELECT ON dbo.ViewMerchants • Can Chrissy SELECT dbo.ViewMerchants? – ___________________
Example #5
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: DENY SELECT ON dbo.Merchant • Chrissy: GRANT SELECT ON dbo.ViewMerchants • Can Chrissy SELECT dbo.ViewMerchants? – Yes, and she can see all of dbo.Merchant!
Object Ownership
• When multiple objects access each other sequentially, this is called a chain
Object Ownership
• When the owner of all objects in the ownership chain is the same, permission is only checked at the entry point of the chain – Example: if the dbo owns all objects in the chain
Ownership Chain Demo
Object Ownership
• This makes life easy for the DBA: you only need to concern yourself with whether your user/role has permissions at the entry point – Most real-world database work is done on objects all owned by the same user
In the next videos…
• Ownership chains • How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• When there are different owners in the chain, SQL Server checks the permission at each change in ownership – This is called ownership chaining – It’s a hard life dealing with databases with multiple schema owners that also contain objects controlled by multiple users
Ownership Chain Demo
Example #1
• The dbo creates a view, Admin.GetMerchants, that references dbo.Merchant
• The dbo grants Chrissy SELECT on the view • When Chrissy selects from the view, is ownership chaining in effect? – No, the same owner owns all objects!
Object Ownership
• It is not the schema name that causes ownership chaining – It is the fact that the schemas are owned by different users
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Can Chrissy create this view despite not having the SELECT permission?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Since Chrissy cannot select but can create the view, can she select from her own view and get the data in that way?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Since Chrissy cannot select but can create the view, can she select from her own view and get the data in that way? • No! She must be given SELECT privilege on dbo.Merchant
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view? • No! But for a different reason…
Object Ownership
• Remember: SQL Server checks permissions at each change in ownership • Terry must have: – SELECT permission on Admin.GetMerchants – SELECT permission on dbo.Merchant
• But there is another way……………………….
In the next videos…
• How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view?
Object Ownership
• Remember: SQL Server checks permissions at each change in ownership • Terry must have: – SELECT permission on Admin.GetMerchants – SELECT permission on dbo.Merchant
• But there is another way……………………….
Impersonation in Modules
• The creator of the module (i.e. procedure, view, function, trigger) can specify the execution context – Just because you can doesn’t mean you should…
Example #1: • Chrissy: – Owns Admin schema – Has CREATE VIEW privilege – Creates a view, Admin.GetMerchants, based on dbo.Merchant
CREATE VIEW Admin.GetMerchants AS SELECT * FROM dbo.Merchant GO
Impersonation in Modules
• This example actually runs under the execution context EXECUTE AS CALLER: – EXECUTE AS CALLER says that SQL Server should check permissions at each change in ownership (the default)
Impersonation in Modules
• There are actually four options for switching execution context in modules: – EXECUTE AS CALLER (the default) – EXECUTE AS ‘user_name’ – EXECUTE AS SELF – EXECUTE AS OWNER
Impersonation in Modules
• EXECUTE AS ‘user_name’ – All access to other tables and modules in this module is done via the security context of ‘user_name’ – Creator of module must have IMPERSONATE ‘user_name’
Impersonation in Modules
• EXECUTE AS SELF – All access to other tables and modules in this module is done via the security context of the creator of the module
Impersonation in Modules
• EXECUTE AS OWNER – All access to other tables and modules in this module is done via the security context of the current owner of the module
Example #1: •
Chrissy: – – – –
Owns Admin schema Has CREATE VIEW privilege Creates a view, Admin.GetMerchants, based on dbo.Merchant Has IMPERSONATE „Terry‟
CREATE VIEW Admin.GetMerchants EXECUTE AS ‘Terry’ AS SELECT * FROM dbo.Merchant GO -- Terry must have SELECT on -- dbo.Merchant for any user -- (except dbo) to get data
How to transfer objects…
• The final piece here is knowing how to transfer objects • You can transfer: – Databases to new logins – Schemas and modules to users or roles
To Transfer a Database
• DBAs often create databases and then transfer them to another login – We are stingy about giving out GRANT CREATE DATABASE permissions…
To Transfer a Database
• To transfer a database, use the sp_changedbowner stored procedure – The login must not be an existing user in the database prior to transfer - why? -- Make sure you are in the right DB! USE [LearnItFirst.com] GO EXEC sp_changedbowner „Chrissy‟ -- The login „Chrissy‟ is now the dbo!
To Transfer an Object
• Transferring object ownership is really transferring the object to a new schema or changing the schema’s owner – Generally you want to change the schema’s owner in the real world
To Transfer an Object
• To transfer ownership of a schema: – Generally you want to change the schema’s owner in the real world
To Transfer an Object
• To transfer ownership of a schema in Transact-SQL: ALTER AUTHORIZATION ON SCHEMA::[Customer] TO [Terry]
To Transfer an Object
• If you absolutely, positively have to change the schema of an object (table, view, etc), it is easiest to do so in T-SQL – You can do it in SSMS’ GUI but sheesh! ALTER SCHEMA [NewSchema] TRANSFER „OldSchema.MyTable‟
SQL Server 2008 Security: Encryption Overview Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Encryption Overview
• SQL Server 2008 has native encryption built in – Connections can be encrypted – Source code can be encrypted – Data can be encrypted
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
• Let’s talk about each!
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encrypting Connections to SQL Server
• SQL Server can use 40 or 128-bit Secure Sockets Layer (“SSL”) to encrypt network packets – Slower performance but higher security – Requires a certificate to be installed on the machine that hosts the SQL Server – Best practice is to use a trusted certificate • Self-signed certificates can be thwarted
Encrypting Connections to SQL Server
– Step 1: Install certificate – Step 2: In the SQL Server Configuration Manager, right click on Protocols for
Encrypting Connections to SQL Server
– Step 3: Choose your certificate in the Certificates tab
Encrypting Connections to SQL Server
– Step 4: Choose whether or not encryption is required • The Force Encryption option: – If “No”, encryption is optional – If “Yes”, clients who are not encrypted will not be allowed to connect
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encrypting Source Code
• Source code can also be encrypted – However, whether it can be decrypted or not is another matter – Always have a separate copy of your encrypted source code
CREATE PROC MyProc WITH ENCRYPTION AS PRINT „You cannot see the source code now!‟ GO
Encrypting Data
• When we talk about encryption in SQL Server, we are generally discussing the encryption of actual data • There are many ways to encrypt data inside your SQL Server – Pass in data that is already encrypted – Have SQL Server handle the encryption for you
Encrypting Data
• In the next video, we’ll cover the types and techniques of native SQL Server encryption
SQL Server 2008 Security: Encryption Overview Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encryption
• Encryption can be done at the databaselevel or at the column-level (or cell-level) • Be aware that encryption will result in poorer performing queries
Encryption
• If you don’t have Enterprise Edition… – You must perform column-level encryption manually using functions like: • • • •
EncryptByPassphrase() EncryptByCert() EncryptByAsymKey() EncryptByKey()
– Column-level encryption is covered in detail in out SQL Server 2008 Transact-SQL programming class • Course 161
Encryption
• If you do have Enterprise Edition, then you can use Transparent Data Encryption (“TDE”) – Entire database is encrypted – Requires no coding changes by developers – Encryption/decryption is done automatically and cannot be bypassed – Performance hit of “3-5%” – Source: Microsoft article, “Database Encryption in SQL Server 2008” via http://learnitfirst.com/go.aspx?id=J4EK
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
• Let’s look at each step
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 1: Create a master key in the master database – A master key is a symmetric key used to protect the private keys of certificates and asymmetric keys – There is no “default” master key so you must either create it or use an existing key if already present
Encryption
• Step 1: Create a master key in the master database (cont.) USE master GO CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password„ -- Encrypted using Triple DES
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
• Let’s look at each step
Encryption
• Step 2: Create a certificate based on the master key – Required to create Step 3’s database encryption key
USE master GO CREATE CERTIFICATE LearnItFirstCert WITH SUBJECT = „Transparent Data Encryption Cert‟
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 3: Create a database encryption key – Based on the certificate from Step 2 – Can use AES, DES, Triple DES, RC4, and more USE [LearnItFirst.com] -- The db to protect GO CREATE DATABASE ENCRYPTION KEY WITH ALGORITHM = AES_256 ENCRYPTION BY SERVER CERTIFICATE LearnItFirstCert -- Cert from Step 2
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 4: Turn it on! USE [LearnItFirst.com] -- The db to protect GO ALTER DATABASE [LearnItFirst.com] SET ENCRYPTION ON GO
TDE
• A few tips for working with TDE: – Backup master and your DB before beginning – Configure on multiple test servers and databases until you are very comfortable before rolling into production – TDE requires time to do the encryption so schedule over a weekend or during planned downtime – Once you enable TDE, tempdb will be encrypted as well • This lowers the performance of SQL Server for non-TDE databases
TDE
• Backup and Restore – Take backups of your service master key, your master key and certificate(s) at the same time you turn on TDE – Store these backups in a separate location from your database backup – Your database cannot be attached or restored without these!
TDE
• Backing up TDE databases requires more… – The service master key will be covered later… BACKUP SERVICE MASTER KEY TO FILE = „F:\SMK_Timestamp.bak„ ENCRYPTION BY PASSWORD = „p@55w0rD!'
TDE
• Backing up TDE databases requires more… BACKUP MASTER KEY TO FILE = „F:\MasterKey.bak„ ENCRYPTION BY PASSWORD = „n3Wp@55!‟
TDE
• Backing up TDE databases requires more… BACKUP CERTIFICATE LearnItFirstCert TO FILE = „F:\LearnItFirstCert.bak‟
TDE
• Restoring TDE databases is exciting! – Make sure you take your heart medicine first… – Step 1: Restore Service Master Key (if needed) – Step 2: Restore Master Key (if needed) – Step 3: Re-create the certificate (if needed) – Step 4: Restore the database
TDE
• We will cover restoring service master keys in chapter 6 • Once all keys are restored and the certificate is created, just restore the database – No need to go through any hoops if restoring to the same server (all keys and certs are present!)
SQL Server 2008 Security: Linked Servers and Distributed Queries Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Distributed Queries Overview
• You are able to query additional data sources using Transact-SQL: – Linked servers are pre-configured for granular security and repeated use – OPENQUERY(), OPENROWSET() and OPENDATASOURCE() allow you to specify data source at run-time
• These are called distributed queries
Distributed Queries Overview
• Distributed queries often involve multiple data source types – You can JOIN a SQL Server table to an Oracle table, for example
Linked Servers
• Linked Servers: – Allow you to write easy Transact-SQL statements against multiple data sources without having to enter login credentials with each query – Login credentials are stored with the linked server’s definition – Used whenever you need to repeatedly access an external data source from within SQL Server and do not want to have to enter credentials each time
Linked Servers
• Linked Servers are configured by the DBA – In SSMS, go to Server Objects
Linked Servers
• Let’s view a demo!
SQL Server 2008 Security: Linked Servers and Distributed Queries Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Pass-through queries
• Pass-through queries are queries whose parsing is done completely by the remote data source – Are written in the language of the underlying data source – The SQL Server query parser does not even parse the statement
OPENQUERY()
• OPENQUERY() allows us to issue passthrough queries against linked servers – Anyone can execute – Permission is checked in the linked server – Can issue any SELECT, INSERT, UPDATE and DELETE
OPENQUERY()
-- If we have a linked server named -- JUPITER to a SQL Server 2008 -- instance SELECT * FROM OPENQUERY(Jupiter, „SELECT FirstName, LastName FROM dbo.Customer‟)
OPENQUERY()
-- OPENQUERY is not necessary here -- This is not a pass-through query: SELECT FirstName, LastName FROM Jupiter.TheDb.dbo.Customer -- Notice the four-part address -- Required for linked servers
OPENQUERY()
• OPENQUERY is generally used with nonSQL Server-based linked servers – Like Microsoft Access or Excel, Oracle, DB2 – For SQL Server-based linked servers, we generally just use “regular old four-part queries” SELECT FirstName, LastName FROM Jupiter.TheDb.dbo.Customer
OPENDATASOURCE() and OPENROWSET()
• OPENDATASOURCE() and OPENROWSET() are functions that allow ad-hoc queries to remote data sources – No linked servers – Require the server-level option “Ad Hoc Distributed Queries” to be enabled EXEC sp_configure „Ad Hoc Distributed Queries‟, 1 RECONFIGURE WITH OVERRIDE GO
OPENROWSET()
• OPENROWSET() is extremely flexible – Can bulk load data from text files – Can load data from Oracle, Access, et al -- Example query to MS Access database SELECT * FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0', 'C:\Program Files\Microsoft Office\OFFICE11\SAMPLES\Northwind.mdb';'admin ';'', Orders) AS x
OPENROWSET()
• Example showing OPENROWSET() -- Example query to a .txt file SELECT * FROM OPENROWSET(BULK 'c:\test\values.txt„)
OPENDATASOURCE()
• OPENDATASOURCE() allows four-part addressing without using a linked server – Not that common
OPENDATASOURCE () -- Example query to MS Access database SELECT * FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0', 'C:\Program Files\Microsoft Office\OFFICE11\SAMPLES\Northwind.mdb';'admin ';'', Orders)...[Customers] -- Notice the “...”
-- Example query to SQL Server instance SELECT * FROM OPENDATASOURCE('SQLNCLI', 'Data Source=London\Payroll;Integrated Security=SSPI') .AdventureWorks.HumanResources.Employee
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: SIDs and Users - What You Need to Know Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
What is a SID?
• SID is an acronym for Security IDentifier • When you create a login, it is given a SID: – Windows-based login SIDs are the same as the Windows account/group SID – SQL Server login SIDs are unique values for that instance
Example
• SIDs are important: – What happens if we want to restore a database on another server? Let’s look at a scenario…
Example
• Your production server has a SQL Server authentication login, “BigLogin” and a database, “OurDb” – BigLogin can access the OurDb database using BigLogin’s user account, “BigUser” – Your application is configured to use the BigLogin login to access SQL Server
Example
• You want to create an offsite copy of the database on another server – You take a backup of OurDb and restore it onto your new SQL Server – You configure your application to use this new server
• Will your application be able to connect?
Example
• No! – You have not created the BigLogin login on the new server
• Lesson learned… So you go create the BigLogin SQL Server authentication login on the new server • Will your application be able to connect?
Example
• Maybe! – It depends on whether or not you have set the default database for the connection string to be OurDb • If you set it, you cannot log in • If you did not set it (unlikely), then you can log in
• Right now, the login on the new server cannot access OurDb!
Example
• Here’s the key take away from this video: – When you create a SQL Server authentication login, it creates it with a random SID by default
Example CREATE LOGIN Scott WITH PASSWORD='p@55w0rD!' GO SELECT sid FROM sys.server_principals WHERE name = 'Scott' GO DROP LOGIN Scott GO CREATE LOGIN Scott WITH PASSWORD='p@55w0rD!' GO SELECT sid FROM sys.server_principals WHERE name = 'Scott„
Example
• When you create a SQL Server authentication login, it creates it with a random SID by default
Example
• So back to our scenario… – When you created the new login, it created it with a random SID – Main point: The user account in the OurDb database on the new server is not associated with your new login!
• Why not?
Example
• Because the SIDs for the login BigLogin are different on each server! • Enough for now… We’ll revisit this in the next chapter and show examples and how to resolve!
Example
• Speaking of the next chapter… • Up next: Chapter 6: Backup and Recovery
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Chapter Overview
• Lack of backups can break a company • As the DBA, it is your responsibility to create backup and recovery plans and test those plans – Perform these tests early and often for any critical project
Chapter Overview
• This chapter will teach you how to: – Develop backup plans – Understand the risks associated with any single plan – Take and restore backups
Chapter Overview
• This chapter will not teach you how to: – Implement Database Mirroring, Log Shipping, Replication, or Snapshots – Discuss creating or managing jobs and logs
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
Next up…
• Restoring User Databases • Restoring System Databases
Scenario Setup
• Before we get too deep, we need a real world example to use • For the scenarios in this chapter, you are the database administrator for LearnItFirst.com
Scenario Setup
• LIFWeb: Manages online website – Small-to-medium size (100GB) – Real-time data and transactions – Few bulk transactions (indexing only) – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFWeb: Manages online website – Real-time data and transactions – Few bulk transactions (indexing only) – Backup and restore goals: • Be able to restore quickly • Ensure no lost transactions • Be able to recover to “point of failure” and “point in time”
Scenario Setup
• LIFUtility: Internal support – Very small (100MB) – Static data (numbers tables, functions, procs) – No bulk transactions – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFUtility: Internal support – Very small (100MB) – Static data (numbers tables, functions, procs) – No bulk transactions – Backup and restore goals: • Backup/restore full database only • Point in time recovery is not important
Scenario Setup
• LIFReporting: OLAP database to feed SQL Server Analysis Services – Large (400GB) – Mainly bulk transactions loaded on weekends • indexing, text files, XML
– Not Mission Critical – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFReporting: OLAP database to feed SQL Server Analysis Services – Large (400GB) – Mainly bulk transactions loaded on weekends • indexing, text files, XML
– Not Mission Critical – Backup and restore goals: • Minimize backup size • No need for point in time or point of failure recovery
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
Coming up…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Types of SQL Server Backups
• The first thing to know is what you can back up – Entire database (all files - data and log) • Called a “Full” backup
– Log only • Called a “Transaction Log” backup
– Only the data that has changed since the last Full backup • Called a “Differential” backup
– Filegroups
Dispelling Myths
• Dispelling myths – You do not need to take a database offline to back it up – Backups will include open transactions – All backups include the transaction log – You cannot just backup a table by default • “All or nothing” • One exception: if that table is on a separate filegroup
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Recovery Models
• The next thing to know is what Recovery Models are • A database’s recovery model determines – What you can backup (and subsequently restore) – How the backup process works – How often the transaction log is cleared out
Recovery Models
• There are three types of recovery models – Simple – Full – Bulk Logged
• Each has specific scenarios for use • Can switch between them instantly
Recovery Models
Simple Can backup transaction log? Transaction log is “cleared out” automatically? Transaction log grows until manually cleared out? Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
Bulk
Full
Simple Recovery Model
• The Simple recovery model: – Automatically clears committed transactions from the log • By default, this happens about 1x per minute • Prevents the log from filling up and auto-growing
– Cannot backup the transaction log directly
Simple Recovery Model
• The Simple recovery model is appropriate when: – You do not need “Point in time” recovery – You want to minimize disk space – The default for the master, msdb, and tempdb – Perfect for static, test and development databases
Scenario
• Which of our databases is appropriate for using the Simple recovery model?
Scenario
• Which of our databases is appropriate for using the Simple recovery model? – LIFUtility because it is static and we do not need point in time recovery
Full Recovery Model
• The Full recovery model: – Allows you to restore to a specific point in time or to the point of failure • “I need to restore everything up until 12:15:35 - that’s when we know that Scott ran that nasty UPDATE statement that messed everything up!”
– For this reason, most production databases are Full recovery databases • Least risk of losing work!
Full Recovery Model
• The reason that you can restore to a point in time is that you can restore the log – You can only restore to point in time or point of failure from a transaction log backup, not from a Full or Differential
Scenario
• Which of our databases is appropriate for using the Full recovery model?
Scenario
• Which of our databases is appropriate for using the Full recovery model? – LIFWeb because it changes constantly and we do not want to lose any work
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Logged Recovery Model
• The Bulk Logged recovery model: – Is identical to Full recovery model except when dealing with bulk operations
Recovery Models
Simple Can backup transaction log? Transaction log is “cleared out” automatically? Transaction log grows until manually cleared out? Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
Bulk
Full
Bulk Operations in SQL Server 2008
• Certain statements and tools are considered bulk operations – CREATE INDEX and ALTER INDEX – SELECT INTO – BULK INSERT – OPENROWSET() – bcp
Bulk Operations in SQL Server 2008
• Bulk operations can be – Fully logged - transaction is logged first and then written to the data file – Minimally logged - minimal information is written to the log first and then entire transaction is written directly to data file
Bulk Operations in SQL Server 2008
• In the Full recovery model, ALL transactions are fully logged – This means that your bulk imports/loads/operations are incurring potentially unnecessary overhead
Bulk Operations in SQL Server 2008
• In the Bulk Logged recovery model, bulk operations are minimally logged – No performance overhead associated with logging means faster performance
• So if Bulk Logged is faster for bulk operations, why don’t we always use it?
Bulk Logged Recovery Model
• The Bulk Logged recovery model, while optimized for bulk operations, has a serious “gotcha” Simple
Bulk
Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
* *
Full
The “Gotcha”
• You cannot backup the log if: 1. You are using Bulk Logged recovery model 2. … and the data file is missing 3. … and bulk operations have occurred since the last backup
Bulk Logged Recovery Model
• Let me recap: – Bulk Logged and Full recovery models are identical provided no bulk operations have occurred since the last backup – Bulk Logged recovery model may prevent you from backing up your log
Bulk Logged Recovery Model
• Why? You need to know the architecture of the transaction log! – Let’s do that in the next video…
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
The Transaction Log, Part 1
• SQL Server uses “write ahead logging”
The Transaction Log, Part 1
• SQL Server allocates space an extent at a time – An extent is a collection of eight 8kb pages
The Transaction Log, Part 1
• When you have minimally-logged transactions, the only thing written to the log is the extents that were affected – Detailed row information is not written
The Transaction Log, Part 1
• Demo of a minimally logged transaction
Bulk Logged Recovery Model
• Name two advantages of using the Bulk Logged recovery model – _________________________ – _________________________
Bulk Logged Recovery Model
• Name two advantages of using the Bulk Logged recovery model – Faster bulk operations/loads – Minimize log space – Same protection as Full recovery model unless bulk operations have occurred
The Transaction Log, Part 1
• However… you do have to pay the price for the upload at some point – Detailed row information is not written during writes to the log thus minimizing overhead and log space – Detailed row information is written at log backup! • Expect your transaction log backups after a bulk operation to be large and time-consuming
Scenario
• Which of our databases is appropriate for using the Bulk Logged recovery model?
Scenario
• Which of our databases is appropriate for using the Bulk Logged recovery model? – LIFReporting because is loaded primarily using bulk operations – However, we likely will use Full recovery model most of the time and switch to Bulk Logged when using bulk operations • Remember that we can switch between them as needed
Switching Between Recovery Models
• You can switch between recovery models as needed – It is common to use FULL as your production model and switch to BULK LOGGED only during bulk operations – If you switch to SIMPLE, backup the log first – If you switch from SIMPLE, requires FULL or DIFF to actually take effect
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video… Demo!
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Scenarios
• Let’s walk through a few backup strategies using each recovery model • One caveat: we are not discussing the “how to” or the “why” of the restore yet
The Basics of Restoring Databases
• However you need to know a few basics: – You must always start with a FULL backup – You may only restore a single DIFFERENTIAL • Remember that a DIFFERENTIAL contains all changes since the last FULL backup • You do not need to restore any log backups taken before your DIFFERENTIAL
The Basics of Restoring Databases
• Basics of restoring LOG backups: – Logs must be restored in sequence – You cannot skip logs during the restore – You cannot skip transactions during a restore – It is possible to restore only part of the last log file you restore • Achieves “point in time” and/or “point of failure” recovery
The Basics of Restoring Databases
• Basics of restores: – The final backup you restore will, by default, take the database through the RECOVERY process and bring the database ONLINE • You decide which is the final backup file in your Transact-SQL code
– Any open transactions in the database will be rolled back during the RECOVERY process
The Basics of Restoring Databases
• One final wrinkle: – In the FULL and BULK LOGGED recovery model, you can back up the transaction log even if the data file is corrupt/suspect/missing • The only exception is when using BULK LOGGED recovery model and a bulk operation has occurred since the last backup
– This is how to do “point of failure” recovery!
The Basics of Restoring Databases
• Reminder: we will discuss all of this in detail later in the chapter
The Basics of Restoring Databases
• Example restore: – FULL backup: from Monday: 0800 – LOG backups: Monday: 0900, 1000, 1100 – DIFF backup: Monday 1200 – LOG backups: Monday: 1300, 1400, 1500
The Basics of Restoring Databases
• Example restore (cont.): – At 1430, user “Steve” notifies you that a recently applied application hotfix has “messed up the whole database” • You: “Okay, Steve - when was the hotfix applied?” • Steve: “We’re not exactly sure - sometime around 12:30”
– You and your team decide to restore up through 1220 (i.e. “point in time” restore)
The Basics of Restoring Databases
• Example restore (cont.): – What is your strategy to restore up through 1220? • Step 1: _____________________________ • Step 2: _____________________________ • Step 3: _____________________________
The Basics of Restoring Databases
• Example restore (cont.): – What is your strategy to restore up through 1220? • Step 1: Restore the most recent FULL • Step 2: Since the DIFF occurred before the stop time of 1220, restore the most recent DIFFERENTIAL • Step 3: Restore the 1300 LOG backup but tell SQL Server to STOP AT 1220
The Basics of Restoring Databases
• Example restore (cont.): – Since your last transaction in the restored database occurred at 1220, what happened to all transactions after 1220? ___________________ – What happened to any open transactions that were open as of 1220? _____________________
The Basics of Restoring Databases
• Example restore (cont.): – Since your last transaction in the restored database occurred at 1220, what happened to all transactions after 1220? Never occurred so “lost” – What happened to any open transactions that were open as of 1220? Rolled back in RECOVERY
Confused?
• It’s okay if you did not fully understand the scenario • We will thoroughly review restores including how to STOP AT a specific time later in this chapter
SQL Server Backup and Recovery
Presented by Scott Whigham
73
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Determining your backup strategy
• Let’s walk through backup strategies and restore scenarios • How do we determine the optimal backup strategy? What factors do we have to consider? ________________________________ ________________________________ ________________________________ ________________________________ ________________________________ ________________________________
Determining your backup strategy
• Factors to consider: 1. 2. 3. 4. 5. 6. 7.
Size of database Time it takes for each backup type to complete How much “drag” backups place on the system How much disk space our backups consume Where to put the backups? How much “loss” is acceptable How much downtime is acceptable
Factors to consider
1. Size of database – Large databases are problematic: • Backup time can be hours and hours – Therefore the restore time can be hours and hours
• Backup file sizes are huge – In SQL Server 2008 Enterprise Edition, backup files can be compressed (more on this later)
• Since the backups are so massive, they have an impact on system performance
– Smaller and mid-size databases do not have as much to worry about
Factors to consider
2. Time it takes for each backup type to complete – Some sites have downtime - they work business hours and therefore full system backups can occur during “off” hours with no impact – Many sites require maintenance windows to perform full system backups • Large database backups often do not “fit” your window!
Factors to consider
3. How much “drag” backups place on the system – A rule of thumb for most scenarios/databases: • FULL backups often have the most performance impact • DIFFERENTIAL backups have the next-most performance impact • LOG files, if taken often, will have the least impact
– This does not cover all situations
Factors to consider
4. How much disk space our backups consume – By default, SQL Server will do no compression on backups • Large databases means large backups • A SQL Server backup only contains the data, not the “empty” pages – If you have a 100GB database with only 10GB of data, your FULL backup will be ~10GB
– SQL Server 2008 Enterprise has the option to compress your backups (more on this later)
Factors to consider
5. Where to put the backups – You can backup to: • Disk - local and remote • Tape - local only
– Disks are fast and tapes are slow – Often the strategy is: • • • •
Backup to local or network disk Copy disk copy to tape Move/rotate tape offsite Delete disk backup according to a schedule
Factors to consider
5. Where to put the backups (cont.) – Backing up across a network often adds overhead • If time is a problem, backup local and do a network file copy later
– Do not store backups on same disks as data or log files • For performance reasons • For recovery reasons (more on this later)
Factors to consider
6. How much “loss” is acceptable – Strange factor, eh? – All backup strategies want to minimize data loss • Impossible to completely prevent the possibility that data loss can occur • Saying that “Data loss is impossible” is akin to saying, “My network is so secure no could ever hack into it”
– One technique for minimizing data loss is taking frequent backups • Some sites take LOG backups every 10 minutes
Factors to consider
6. How much “loss” is acceptable (cont.) – Backups are not the only strategy in play – High Availability solutions such as Database Mirroring, Log Shipping, and Replication are options we will discuss in a future chapter
Factors to consider
7. How much downtime is acceptable – All backup strategies want to minimize downtime • Just like with data loss, it is impossible to completely prevent the possibility that downtime can occur
– High availability solutions (including clustering) offer protection against downtime
Factors to consider
7.
How much downtime is acceptable (cont.) – If you do have to completely restore a database, “It takes as long as it takes” • A 1TB FULL backup is going to take a long time to restore…
– There are strategies that minimize restore time: • Frequent DIFFERENTIAL backups minimize the number of LOG backups that have to be restored, for example • Certain software packages minimize backup and restore times • SQL Server 2008 Enterprise Edition includes a feature called Fast Recovery
Factors to consider
7. How much downtime is acceptable (cont.) – SQL Server 2008 Enterprise Edition includes a feature called Fast Recovery • This allows users to access the database before the restore has rolled back uncommitted transactions (i.e. before the Undo phase of the restore has completed)
Determining your backup strategy
• The challenge is: “How do we create a backup strategy that protects us from data loss while also ensuring minimal downtime during a restore?”
Determining your backup strategy
• For a great article on specifically how the restore process works, visit http://msdn.microsoft.com/library/ms191455.aspx
• Article: “Understanding How Restore and Backups Work in SQL Server”
SQL Server Backup and Recovery
Presented by Scott Whigham
90
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Scenario Reminder
• Remember our three databases: – LIFWeb: Manages online website • Small-to-medium size (100GB) • Full recovery model
– LIFUtility: Internal support • Very small (100MB) • Simple recovery model
– LIFReporting: OLAP database to feed SQL Server Analysis Services • Large (400GB) • Full recovery model and switch to Bulk Logged when using bulk operations
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• If a problem occurs on Friday at 1200, can you back up the log so that you can achieve “point of failure” recovery? Explain your answer – ______________________________________________
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• If a problem occurs on Friday at 1200, can you back up the log so that you can achieve “point of failure” recovery? Explain your answer – No - Simple recovery model does not allow log backups
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• Since you cannot backup the log, what is your restore sequence? – ______________________________________________
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• Since you cannot backup the log, what is your restore sequence? – Restore the FULL backup from Saturday – Any data added or modified after Saturday night’s backup will be lost
Backup Strategy
• Are there any situations in which this is acceptable? – Read-only databases – Databases whose updates occur on Saturday mornings only – Test databases – “Junk” databases
Backup Strategy
• What are some strategies to employ if that much potential data loss is not acceptable? – More frequent backups – Changing the recovery model – Adding high availability options
Backup Strategy
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
Example #3
Backup Strategy:
Database: LIFUtility Recovery model: SIMPLE
• Full backups on Saturdays at 2300 • Differential backup Wednesday at 2300
• How does adding in differential backups each night change your restore sequence? – ______________________________________________
Example #3
Backup Strategy:
Database: LIFUtility Recovery model: SIMPLE
• Full backups on Saturdays at 2300 • Differential backup Wednesday at 2300
• How does adding in differential backups each night change your restore sequence? – Step 1: Restore the FULL from Saturday – Step 2: Restore Wednesday night’s DIFFERENTIAL – Any data added or modified after Wednesday night’s backup will be lost
In the next video…
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
SQL Server Backup and Recovery
Presented by Scott Whigham
103
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Backup Strategy
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
Simple Recovery Model
• Simple recovery model has a huge “hole”: – You cannot perform a “point of failure” restore – Why? • You cannot backup the transaction log • You cannot restore only part of a FULL or DIFFERENTIAL • You can restore only part of a LOG
Crash Procedure for Simple Recovery Model
• Your restore sequence in Simple recovery model is: • Step 1: Restore your most recent FULL backup • Step 2: Restore your most recent DIFFERENTIAL (if available) • Any changes after this will be lost
Simple Recovery Model
• Let’s take a look at what the Pros and Cons are when we change from Simple to Full recovery model
Example #1
Backup Strategy:
Database: LIFUtility Recovery model: FULL
• Full backups on Saturdays at 2300 • Differential backups each day at 2300
• In this scenario, a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. How does changing the recovery model change your restore sequence? – ______________________________________________
Full Recovery Model
• Remember that the full recovery model allows you to backup your transaction log – This is critical to remember during a crisis – Do not make the mistake of “forging ahead” with a restore before taking a backup!
Full Recovery Model
• Also remember that you can restore only part of your last transaction log backup – We can stop at 1129 to ensure minimal data loss!
Example #1
Backup Strategy:
Database: LIFUtility Recovery model: FULL
• • • • • •
• Full backups on Saturdays at 2300 • Differential backups each day at 2300
Step 1: Backup the current log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore the log but tell it to stop at 1129 All changes after 1129 will be lost Any open transactions as of 1129 will be rolled back
Backup Strategy
• Question #1: – What would happen if we accidently forgot to take the log backup in Step 1? • We would lose all changes since the DIFF
Backup Strategy
• Question #2: – What would happen if the disk(s) holding the data file crashed during the backup of the log? • Remember that we can still backup the log because changes are written to the log first, and then to the data file – This is a primary function of the transaction log
Backup Strategy
• Question #3: – What would happen if the disk(s) holding the transaction log crashed during the backup of the log? • We would lose all changes since the DIFF
Be Careful!
• Question 3 highlights a real problem: – If you do not backup the log frequently, you are at risk of losing all data since the last backup
• How do we mitigate this problem? – More frequent backups – Redundant disks
In the next video…
• We’ll cover more strategies and issues for the FULL recovery model
SQL Server Backup and Recovery
Presented by Scott Whigham
118
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Full Recovery Model
• Let’s take a look at a final example – This is a more likely scenario for a database with active changes (like LIFWeb)
Example #1
Backup Strategy:
Database: LIFWeb Recovery model: FULL
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
• Same scenario: a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. What is your restore sequence? – ______________________________________________
Example #1
Backup Strategy:
Database: LIFWeb Recovery model: FULL
• • • •
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore all log backups that have occurred since Thursday’s DIFF and stop at 1129 in the log backup from Step 1
Reminders
• You only restore the most recent DIFFERENTIAL – This is critical to understand for performance reasons • Restoring 300+ log files takes a long time whereas restore a single DIFF backup can be quite fast
Reminders (cont.)
• You will likely have a large number of log files – Do not lose them - you cannot skip logs during a restore! – Be cautious about who (or what) you allow to take log backups and manage the files – Keep redundant copies • Backup to disk and then copy to tape
Remember our strategy:
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
Database: LIFWeb Recovery model: FULL
Day
Type
Filename
Saturday
FULL
LIFWeb_Full_20100306.bak
Sunday
- LOG
LIFWeb_Log_20100307_0000.bak
Sunday
- LOG
LIFWeb_Log_20100307_0100.bak
Sunday
- LOG
LIFWeb_Log_20100307_0200.bak
(log backups each hour = 23 log backup files per day)
Sunday
- DIFF
LIFWeb_Diff_20100307.bak
Differentials
• It is possible to only use FULL and LOG backups – If you lose your DIFF backups, you can simply use the LOGs instead
Differentials
• Each DIFF contains all changes since the last FULL backup – Prepare that your DIFFs will take longer and longer (and be larger and larger) as you get farther away from the FULL backup • Saturday: FULL • Sunday: Diff (100MB) • Monday: Diff (200MB)
Differentials
• Example of a 1GB/day delta: Day
Type
Size in in GB
Saturday
FULL
300
Sunday
DIFF
1
Monday
DIFF
2
Tuesday
DIFF
3
Wednesday
FULL
304
Thursday
DIFF
1
Friday
DIFF
2
Crash Procedure for Full Recovery Model
• Restore sequence for Full recovery model: • • • •
Step 1: Backup your current transaction log Step 2: Restore your most recent FULL backup Step 3: Restore your most recent DIFFERENTIAL (if available) Step 4: Restore all transaction log backups taken since the last FULL or DIFFERENTIAL in sequence
– If you follow these steps, you will restore to “point of failure”
In the next video…
• How changing to BULK LOGGED recovery model affects your risk
SQL Server Backup and Recovery
Presented by Scott Whigham
131
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Logged Recovery Model
• Okay - the final one: Bulk Logged – This is a more likely scenario for a database with bulk operations like LIFReporting – Remember that BULK LOGGED is identical to FULL except when bulk operations occur
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same scenario: a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. What is your restore sequence? – ______________________________________________
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same as with the FULL: – – – –
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore log backup from Step 1 but stop at 1129 – “Point in time” restore
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• New scenario: At 1200 on Friday, the disk(s) storing your transaction log crash. What is your recovery sequence? – ______________________________________________
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Step 1: Restore Saturday’s FULL • Step 2: Restore Thursday’s DIFF • Step 3: All changes since the last backup will be lost
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• New scenario: At 1200 on Friday, the disk(s) storing your data file crash. What is your recovery sequence? – ______________________________________________
Aha!
• You actually do not have enough information to answer! – What question do you need to know the answer to before you can come up with a sequence? • “Have any bulk operations taken place since the last backup?”
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Let’s be more specific: At 1200 on Friday, the disk(s) storing your data file crash. There have been no bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same as with the FULL: – – – – –
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore all log backups in sequence “Point of failure” restore
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Let’s be more specific: At 1200 on Friday, the disk(s) storing your data file crash. There have been bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• We cannot back up the log – – – –
Step 1: Restore Saturday’s FULL Step 2: Restore Thursday’s DIFF All changes since the last backup are lost “Point of last backup” restore
Example #4
Backup Strategy:
Database: LIFReporting Recovery model: BULK LOGGED
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Hourly log backups
• Same as before: At 1200 on Friday, the disk(s) storing your data file crash. There have been bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #4
Backup Strategy:
Database: LIFReporting Recovery model: BULK LOGGED
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Hourly log backups
• We still cannot back up the log: – Step 1: Restore Saturday’s FULL – Step 2: Restore Thursday’s DIFF – Step 3: Restore all log backups that have occurred since last DIFF backup – All changes since the last backup are lost – “Point of last backup” restore • However, we’ve “lost less”
In the next video…
• We’ll cover a practical way to manage bulk operations that will minimize your risk
SQL Server Backup and Recovery
Presented by Scott Whigham
147
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Import How-to
• When you want to perform long-running bulk imports, you want to maximize performance – Example: you have a text file with rows of data that previously were stored in DB2 – You can use BULK INSERT, OPENROWSET(), SSIS, or bcp to load these rows into a SQL Server table
Bulk Import How-to
• There are two techniques to maximize performance: – Drop nonclustered indexes when doing bulk imports • Speeds up the import • Must rebuild after import
– Use BULK LOGGED recovery model
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• We backup the log first so that we have a “safe” restore point – Assuming we are loading data from a text file, if the import fails, we can always restore back to this log and re-do the import with no data loss
Bulk Import How-to
• If desired, we could switch Steps 1 and 2 around with no negative effects – Remember that we can switch between FULL and BULK LOGGED as necessary
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Use the DROP INDEX statement – Don’t forget to script out your index creation statements before you drop your indexes…
• Review BULK INSERT, OPENROWS, Integration Services, and, as a last resort, bcp.exe for options on importing rows
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup your log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• These two steps could also be switched around if you prefer • It is recommended that perform your log backup prior to adding indexes – Minimizes the amount of restore time in the event a catastrophe occurs during index creation
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Now that you have completed your bulk ops, switch back to FULL for normal, everyday operations – FULL recovery model provides the most complete coverage against data loss
Bulk Import How-to
• Question: if you forget to switch over to BULK LOGGED recovery model, can you still perform bulk operations? – Somewhat of a trick question… – Let’s break it into two parts…
Bulk Import How-to
• If you load a text file using BULK INSERT into a SQL Server database that uses the FULL recovery model, will the operation be fully logged or minimally logged? – Fully logged! – SQL 2000/7.0 allowed DBAs to prevent these operations but not SQL 2005/2008
Bulk Import How-to
• So if this is a fully logged operation, is it technically correct to still call it a “bulk operation”? – Yes, these are still bulk operations yet they are not minimally logged
Bulk Import How-to
• Question: What tool(s) will you use to perform the steps in this guide? – A SQL Server job is perfect – Integration Services is ideal as well – You can even schedule a SQL Server job to execute your Integration Services package!
SQL Server Backup and Recovery
Presented by Scott Whigham
165
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Checkpoints
• To understand the transaction log, you need to understand what a checkpoint is – Writes all dirty pages for the current database to disk. – Writes “maintenance” information in the event a recovery needs to occur – Checkpoints happen about once per minute • Configured via sp_configure and “recovery interval”
The Transaction Log
• We talked earlier about how the Simple recovery model “clears out the log” – The transaction log is “cleared out” at each checkpoint • This is why we cannot take “meaningful” log backups
– But what exactly gets removed from the log? How does SQL Server know what to keep and what to remove?
The Transaction Log
• The log has three basic parts: – Active log – Inactive log – Free space
The Transaction Log
• The active log (or “active portion of the transaction log”) is: – From the beginning of the oldest “open” transaction – From the beginning of any transactions marked for replication that have yet to be delivered to the distribution database
The Transaction Log
• The inactive log (or “inactive portion of the transaction log”) is: – From the beginning of the logical log up until the beginning of the active log
Visualizing the Transaction Log
• Visualization:
In the next video…
• We’ll learn more about the architecture of the transaction log
The Transaction Log
• Let’s take this backup strategy and work through it – FULL: Saturdays at 2300 – DIFF: Weekdays at 2300
SQL Server Backup and Recovery
Presented by Scott Whigham
175
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
The Transaction Log
• You can remove the inactive portion of the log – This is “clearing out the log” – Technically called truncating the log
• You cannot remove any part of the active portion
The Transaction Log
• The Simple recovery model automatically truncates the log on checkpoint • Why do we take log backups? – So that we can “replay” the events on the server in the exact order as they happened – About 1x/minute the inactive portion of the log is cleared out – Makes the log useless for recovery
The Transaction Log
• FULL and BULK LOGGED do not truncate the log – Do not become one of my examples to talk about – Your transaction log can grow to fill up the disk!
The Transaction Log
• By default, when you perform a transaction log backup, the inactive portion of the log becomes free space • FULL and DIFFERENTIAL backups do not clear out the log
The Transaction Log
• When you take a log backup, SQL Server: – Copies the entire log to the backup file – Marks the inactive portion of the log as “free space” (i.e. it can/will be overwritten)
• This is why your logs must be restored “in sequence”
The Transaction Log
• It is possible to backup the log without removing the inactive portion • Used in two instances: 1. When you want to take a log backup without affecting your “normal” backup routines •
Remember that you must restore “all logs since the last DIFFERENTIAL in sequence”
The Transaction Log
• Used in two instances: 2. When the data file is unavailable • •
What if the disk(s) holding the data file (.mdf) crash? We can still backup the log provided we do not truncate the log
In the next video…
• Let’s demo what happens when you never backup your log!
SQL Server Backup and Recovery
Presented by Scott Whigham
185
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Securing Your Backups
• You need to secure your backup files and tapes • Anyone can take a backup file and restore it on another server
Securing Your Backups
• Previous versions of SQL Server allowed passwords on backups – Deprecated in SQL Server 2005 due to being a weak protection mechanism BACKUP DATABASE master … WITH PASSWORD = ‘$hjg#907563’ -- required to restore
Securing Your Backups
• Best practice: backup your databases to disk and secure those files with proper Windows file permissions – Technically called “Access Control Lists” (ACLs) – Set the permissions on the root folder the backups are stored in
Securing Your Backups
• Best practice: secure your tapes – Again, if someone can get to a backup file, they can restore it…
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Securing Your Backups
• Remember that SQL Server does no compression on backups by default • For years, we’ve relied on 3rd party tools – WinZip, WinRAR, 7-Zip are unsupported
Securing Your Backups
• SQL Server 2008 debuts a feature called Backup Compression – Can only create compressed backups in Developer and Enterprise Edition – All editions can restore a compressed database
Securing Your Backups
• Compression actually increases backup speed – CPU intensive – If too many resources are consumed by backup, consider using Resource Governor to limit CPU usage by Backup Compression
In the next video…
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
SQL Server Backup and Recovery
Presented by Scott Whigham
198
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Reminders about the restore process
• Remember the basics: – You must always start with a FULL backup – You may only restore a single DIFFERENTIAL • Remember that a DIFFERENTIAL contains all changes since the last FULL backup • You do not need to restore any log backups taken before your DIFFERENTIAL • You can, however, skip the DIFFERENTIAL and restore the logs instead
The Basics of Restoring Databases
• Basics of restoring LOG backups: – Logs must be restored in sequence – You cannot skip logs during the restore – You cannot skip transactions during a restore – It is possible to restore only part of the last log file you restore • Achieves “point in time” and/or “point of failure” recovery
The Basics of Restoring Databases
• Basics of restores: – The final backup you restore will, by default, take the database through the RECOVERY process and bring the database ONLINE • You decide which is the final backup file in your Transact-SQL code
– Any open transactions in the database will be rolled back during the RECOVERY process
If you are lucky…
• Hopefully your backups are in the same place that SQL Server created them – When you take a backup, that information is stored in that instance’s msdb database • Database, time, name, user, and location • Used by SSMS to suggest restore sequence!
msdb
• msdb contains several tables for backups: – dbo.backupset - one row for each backup set • A backup set contains the backup from a single backup operation
– dbo.backupfile - one row for each backup file • A backup set contains the backup from a single backup operation
– There are others as well
SSMS is your friend
• If, during a restore, all backup files are in the same location they were when they were created: – You can breathe easy - SSMS will automatically create the restore sequence (correctly!) – You then can choose “point in time”, “point of failure”, or to restore up to a specific backup
SSMS is your friend
• It is highly recommended that you at least use SSMS to generate your restore sequence – SSMS reads the msdb tables to come up with: • Most recent FULL • Most recent DIFF (if available) • Each LOG in sequence
SSMS is your friend
• If you do use SSMS to recover, SSMS defaults to: – “Point of last backup”
• You can change this with ease
Transact-SQL is a pain…
• Now… If you want to restore using Transact-SQL, prepare for a lot of typing – Transact-SQL requires a separate RESTORE statement for each backup – Let’s learn more in the next video!
SQL Server Backup and Recovery
Presented by Scott Whigham
209
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Deeper into the restore
• Now… If you want to restore using Transact-SQL, prepare for a lot of typing – Transact-SQL requires a separate RESTORE statement for each backup – To get it “right the first time” requires a deeper understanding of the restore options and process
The RECOVERY process
• Remember the concept of “RECOVERY” – When a database goes through RECOVERY: • • • •
Committed transactions are rolled forward Open transactions are rolled back Database is brought ONLINE Only done on the last backup restored
The RECOVERY process
• When you have to restore multiple backups, you cannot go through RECOVERY until the last backup – Example restore sequence: • FULL: from Saturday • DIFF: From 2300 night before • LOG: 12 log backups to restore
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 1: Restore the FULL – Do you want to go through RECOVERY now? • No, otherwise open transactions that might be completed in a later backup to be restored would be rolled back
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 2: Restore the DIFF – Do you want to go through RECOVERY now? • No, for the same reason…
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 3: Restore 11 logs (but not the last log) – Do you want to go through RECOVERY during any of these logs? • No, for the same reason…
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 4: Restore the last log – Do you want to go through RECOVERY? • Yes! There are no more backups to be applied so we want any uncommitted transactions rolled back!
Restore options
• When restoring a backup file, you have three options: – RECOVERY (the default) – NORECOVERY • Don’t rollback or roll forward
– STANDBY • • • •
Allows read-only access to committed transactions Still able to continue with backup Requires an external file to store rolled back transactions This file must be available for subsequent restores
In the next video…
• We’ll go through restoring databases in SSMS and exploring the options…
SQL Server Backup and Recovery
Presented by Scott Whigham
220
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Point-in-time Restores are tricky
• Scenario: – “Mike” calls you and explains that, sometime around 11:30AM, a “bad” transaction was issued by a developer • Mission critical - must undo this transaction or restore to point prior to the transaction • Gotta do it now!
Point-in-time Restores are tricky
• Question: how can you read the transaction log to view the transactions? – Short answer: you can’t, by default – Well, you can but in a generally not-easy-to-reador-remember way
Point-in-time Restores are tricky
• There are third party applications that allow you to view the SQL Server log easily – Not cheap – Can be massive time savers – Lumigent Log Explorer, ApexSQL Recover among others
Point-in-time Restores are tricky
• If you don’t have access to the third party application, what are your options? – If you had Profiler running at the time, review! – Can review default traces – Can restore then “look and see if ‘bad’ transactions were gone
Enough lecture - let’s demo!
• Let’s go see this in action…
SQL Server Backup and Recovery
Presented by Scott Whigham
227
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Restoring Your Entire Server
• The Scenario: – The disk system that held all of your data files (.mdf) has just crashed and cannot be recovered – The sysadmins have just completed installing new drives and building the array – Now it’s your turn to get that SQL Server back up and running
Restoring Your Entire Server
• You are right in the middle of a catastrophe – You can’t make up for the past now • So you’d better have been backing things up!
– Right now, you need a plan…
Restoring Your Entire Server
• Since you cannot backup, your only option for a restore is “point of last backup” – Or is it??? We’ll just have to see about that later…
Restoring Your Entire Server
• You are awesome – Thankfully you have recent backups of all the right things • • • •
System Databases User Databases Service Master Key Certificates
Restoring Your Entire Server
• You are double awesome – You also have documentation: • • • •
Server collation Install locations of files Service pack/hotfix level of server Output of sqldiag.exe – sqldiag.exe is a general purpose diagnostic utility that makes keeping track of many servers easy
Restoring Your Entire Server
• We use the SQL Server setup.exe to perform a rebuild • The process is: 1. If there are system database files already in the target folder, delete them 2. Create new system database files from /Binn/Templates/ files •
Located in the folder that your program files are located
Restoring Your Entire Server
• The rebuild process is done via setup.exe: – NOTE: INSTANCENAME=MSSQLSERVER for default instance
setup.exe /QUIET /ACTION=REBUILDDATABASE /INSTANCENAME=instance_name /SQLSYSADMINACCOUNTS= accounts [/SAPWD=password] [/SQLCOLLATION=collation_name]
Restoring Your Entire Server
• Once you rebuild master, you will have an “empty” server – Remember: you are creating new master, model, msdb, and tempdb databases from the templates – The template master database does not know about your databases
Restoring Your Entire Server
• You will not see your: – Databases – Logins – Jobs – Server Objects (i.e. linked servers, backup devices)
• Server settings will not be your settings • model database is back to the default
In the next video…
• We’ll take a look at how to rebuild master
SQL Server Backup and Recovery
Presented by Scott Whigham
239
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Restoring Your Entire Server
• The Scenario: – The disk system that held all of your data files (.mdf) has just crashed and cannot be recovered – The sysadmins have just completed installing new drives and building the array – Now it’s your turn to get that SQL Server back up and running
Restoring Your Entire Server
• To get your SQL Server back: 1. Reinstall service pack and any hotfixes 2. Restore master •
Once complete, you databases, logins, server objects and server settings will be restored
3. Restore msdb •
Restores your jobs, database mail, and more
4. Restore model (optional) 5. Backup the tail of the log for any user databases affected 6. Restore any user databases affected
Restoring Your Entire Server
• Reminders: – No one can be in your database when you restore – You are overwriting a database during the restore • Conceptually think of it in three phases: 1. 2. 3.
Delete original database files Create new versions of database files Restore the backup data into the new files
Restoring Your Entire Server
• To restore master, your server must be in single user mode • Cannot restore other databases while in single user mode
Restoring Your Entire Server
• A word of caution: – Remember that, if you configure SSMS to open both Object Explorer and a query window, that is two connections
Restoring Your Entire Server
• Another word of caution: – Remember that, if your SQL Server Agent is configured to auto start with SQL Server, it logs in before you • When in doubt, stop the SQL Server Agent service
Restoring Your Entire Server
• To put your server in single user mode: 1. Stop the SQL Server Agent 2. Add the -m switch to your SQL Server service’s Startup Parameters via Configuration Manager 3. Restart the service 4. Restore master 1. SQL Server will automatically shut down
5. Remove the -m switch 6. Restart your SQL Server
Restoring Your Entire Server
Restoring Your Entire Server
• About restoring msdb: – SQL Server Agent logs in before you do – SQL Server Agent loves msdb – Make sure you stop the SQL Server Agent before trying to restore msdb
In the next video…
• We’ll take a look at how to rebuild master
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
1
p. 1
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 2
Chapter Introduction
• Chapter 8 covers several unrelated topics – All essential to being a DBA – None large enough to warrant a full chapter
p. 3
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 4
Moving and Copying Databases
• There are several techniques for moving and copying databases – BACKUP and RESTORE – DETACH and ATTACH – SQL Server Integration Services – Copy Database Wizard
p. 5
Moving and Copying Databases
• Depending on which technique you us, you can copy and move: – On the same server – Onto multiple servers – Multiple databases at the same time
p. 6
Moving and Copying Databases
• In the next few videos, we’ll see examples of how to move and copy databases using the following techniques: – BACKUP and RESTORE – DETACH and ATTACH – SQL Server Integration Services – Copy Database Wizard
p. 7
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
8
p. 8
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 9
Orphaned Users
• Way back in Chapter 5 (security) we talked about how user accounts are related to logins via the security identifier (SID) – Not by name
p. 10
Orphaned Users
• User accounts that previously had a login but now have no associated login are called orphaned users • Orphans can be created: – By deleting logins – By copying/moving database to new instance
p. 11
Orphaned Users
1. When you delete a login, the associated users are not deleted
p. 12
Orphaned Users
2. When you copy/move a database to a different instance, the user accounts may be orphaned –
–
If the destination instance does not have logins for the user account SIDs, the user accounts will be orphaned If the destination has logins with the expected SIDs, the user accounts will be mapped
p. 13
Orphaned Users
• Windows authentication logins: – If your user accounts are built on Windows user/group authentication logins, just create the Windows authentication login in the SQL Server • Remember from Chapter 5 that the SID is the actual Windows SID therefore the login/user will be mapped automatically
p. 14
Orphaned Users
• SQL Server authentication logins: – For SQL Server authentication login-based user accounts, simply creating the login with the expected name won’t work • CREATE LOGIN … generates a random SID
– Later on, we’ll cover how to fix!
p. 15
Orphaned Users
• You will encounter this problem often – The user accounts are stored in the database but logins are not – Any time you copy/move a database, you have to deal with this issue
p. 16
In the next video…
• We’ll talk about how to identify orphaned users and strategies for resolving
p. 17
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
18
p. 18
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 19
How to Identify Orphaned Users
• There are several ways to identify orphaned users 1. sp_change_users_login stored proc
2. Query the sys.database_principals view to locate users with no matching sid in sys.server_principals 3. Use SSMS’s graphical tools
p. 20
Orphaned Users
• There are two solutions to this problem: 1. Use sp_change_users_login stored procedure 2. Script out the login complete with SID from the source server and run script on destination server
p. 21
Let’s demo!
• Let’s take a look at using sp_change_users_login and scripting
techniques!
p. 22
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
23
p. 23
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 24
CLR Assemblies
• SQL Server 2008 allows you to load .NET DLLs (called assemblies) into your databases – Assemblies can be written in any language that implements that .NET Common Language Runtime (CLR) • C#, Visual Basic, etc • Often called “managed code” or “managed assemblies”
p. 25
CLR Assemblies
• An assembly loaded into SQL Server can contain one or more: – Stored procedures – User-defined functions – Triggers – and more
p. 26
CLR Assemblies
• Why would we need to use CLR assemblies? – Mostly because certain operations are more efficient in .NET than Transact-SQL • Examples: string manipulation, regular expressions
– Secondly because .NET can do things Transact-SQL cannot do • Example: create table results containing file and folder info
p. 27
CLR Assemblies
• What does this have to do with the DBA? – As the “gate keeper”, you must decide whether to allow and, if you do allow, to what extent you allow CLR integration – Remember the Principal of Least Privilege! • http://learnitfirst.com/go.aspx?id=LPSX
p. 28
CLR Assemblies
• CLR integration is disabled by default – Enabled using the sp_configure stored proc
p. 29
CLR Assemblies
-- To view setting: EXEC sp_configure 'clr enabled' -- run_value = 1: enabled
-- To enable: EXEC sp_configure 'clr enabled', 1 GO RECONFIGURE WITH OVERRIDE
p. 30
In the next video…
• We’ll talk about permission sets and how to configure security for CLR objects
p. 31
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
32
p. 32
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 33
CLR Assemblies
• Once you enable SQL CLR, you must decide which permissions to allow • When you install an assembly, you must determine it’s Host Policy Level – SAFE – EXTERNAL_ACCESS – UNSAFE
p. 34
SAFE
• SAFE is the default and most secure – Cannot go outside the SQL Server process – Can connect to SQL Server in-process
p. 35
EXTERNAL_ACCESS
• EXTERNAL_ACCESS is “SAFE plus go out of process” – Can do all that SAFE can do – Can also go outside the SQL Server process to access external resources • • • •
Read files and folders Read environmental variables Read registry more…
p. 36
EXTERNAL_ACCESS
• UNSAFE can do anything – Can take over the world! – Only sysadmins can assign – Has same permissions as EXTERNAL_ACCESS plus: • Can call unmanaged code (i.e. C++ DLLs) • Allows assemblies to perform “illegal operations against the SQL Server process space…” – Source: Books Online
p. 37
How do I know which to use?
• As a DBA, you cannot know every nuance of every language that can write managed code assemblies • Must coordinate with developer to understand security needs
p. 38
How do I know which to use?
• As a general guideline: – Use SAFE for assemblies that need only to do computations, manipulations, and table access – Use EXTERNAL_ACCESS for when assemblies need to go outside SQL Server space – Use UNSAFE when working with unmanaged code
p. 39
How do I know which to use?
• Each permission set has different requirements for creation: – SAFE: requires CREATE ASSEMBLY permission in the database – EXTERNAL_ACCESS: SAFE plus EXTERNAL ACCESS ASSEMBLY server permission – UNSAFE: sysadmins only!
p. 40
How do I know which to use?
• Create assembly in the [LearnItFirst.com] database USE [LearnItFirst.com] GO CREATE ASSEMBLY LIFRegEx FROM „C:\MyAssembly.DLL‟ WITH PERMISSION_SET = SAFE p. 41
How do I know which to use?
• Can change permission set with ALTER ASSEMBLY ALTER ASSEMBLY LIFRegEx WITH PERMISSION_SET = EXTERNAL_ACCESS
p. 42
The DBA’s Role
• With respect to the CLR, the DBA is expected, at a minimum, to be able to: – To be able to explain what the CLR is – To be able to make suggestions related to when to use the CLR for certain operations – To be able to stop developers from using CLR unnecessarily – To understand the various permission sets
p. 43
The DBA’s Role
• To reiterate: you are not expected to know every single language – Don’t be intimidated by the CLR – If a manager tells you that, “You have to know C# to be a good DBA” during a job interview, you do not want to work for that person! • ….. Well, how much does it pay again?
p. 44
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
45
p. 45
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 46
The Resource Governor
• The Resource Governor: – New in SQL 2008 – Allows you to limit “resources” based on certain criteria • CPU, memory, number of requests, more
– Only available in Enterprise and Developer editions
img by eliburd @ flickr.com
p. 47
Example Usage
• Examples uses of the Resource Governor – Can prevent expensive queries from consuming more than an allowed amount of memory or CPU – Limit the amount of CPU that backup compression can consume – Ensure that specific applications cannot issue queries or workloads that are unhealthy to the rest of the server – Web host wants to provide guaranteed levels of service to multiple clients
p. 48
Example Usage
• The basics of how it works: 1. DBA creates a resource pool 2. DBA assigns the resource pool one or more workload groups 3. DBA writes a classifier function that binds incoming sessions to the appropriate workload group
p. 49
The Resource Governor
• Source: Using the Resource Governor, white paper, Microsoft 2009
p. 50
Resource Pools
• Resource Pools are the highest layer of abstraction – Can define a MIN and MAX CPU – Can define a MIN and MAX memory – Contain n workload groups
p. 51
Workload Groups
• Workload groups allow more granular settings
p. 52
Workload Groups
• Settings available for workload groups: – 0 means unlimited for all – Maximum Requests • Max number of simultaneous requests
– CPU Time (sec) • Max amount of CPU time that a request can use
– Memory Grant % • Max amount of memory a single request can take from the pool
– Grant Time-out (sec) • Max time that a query can wait for a resource before the query fails
– Degree of Parallelism • Max degree of parallelism for parallel requests. Range is 0 to 64
p. 53
Classifications
• Classification is the process of determining the workload group for a given session – A user-defined scalar function that is registered using the ALTER RESOURCE GOVERNOR statement • Called the classifier function
– Evaluated for every new session except Dedicated Administrator Connections
p. 54
The Classifier Function
• The classifier function: – There can be only one – If it returns NULL or a non-existent group, the session is bound to the default workload group – Sessions are bound to the workload group for life
p. 55
Putting It All Together
• After enabling the Resource Governor and putting a classification function in place: 1. User logs in 2. The login authentication is verified 3. Upon successful authentication, any LOGON triggers are executed 4. Finally, classification occurs
p. 56
Words of Wisdom
• A final word of caution – Be ready… • If you mess up your classifier function, you’ve messed up the entire server • Only Dedicated Administrator Connection (DAC) sessions can bypass classification • If a problem arises, log in as DAC and remove classifier
p. 57
Let’s play!
• In the next video, let’s play with the Resource Governor!
p. 58
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
59
p. 59
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 60
Policy-Based Management
• Windows administrators have long been familiar with policy-based management – “All user accounts must change their password every 42 days and it must contain letters, numbers, and special characters”
p. 61
Policy-Based Management
• DBAs have always had standards and business rules – “All stored procedures must begin with usp_” – “No database may have AutoShrink turned on”
• However, what we haven’t had is a way to enforce them automatically
p. 62
Policy-Based Management
• SQL Server 2008 introduces Policy-Based Management (“PBM”) – Allows DBA to define business rules (i.e. policies) that can either be: • Automatically enforced by SQL Server (i.e. prevent noncompliant modifications) • Reported to the DBA as non-compliant
p. 63
Policy-Based Management
• Nearly all DBAs have a daily task list: – Verify backups – Look for anomalies (low disk space, unrecognized files or folders, etc) – Check the error logs – Check the job logs – and many more tasks…
p. 65
Policy-Based Management
• One such task is to verify that new servers are compliant – Example: “Have any new servers been installed?” • If “Yes”, are they up to our standards? – Evaluation or Developer Edition only unless installed by DBAs (to ensure licensing compliance) – Minimum build number – MYDOMAIN\SQLDBAs Windows group is sysadmin – Authentication mode is Windows-only (a.k.a. Integrated)
p. 66
Policy-Based Management
• Another task is ensuring no changes have occurred that result in databases/objects being non-compliant – Are all production databases • Online • Using correct recovery model
– Are there any new [tables/views/functions/procs]? • Do they meet naming conventions?
p. 67
Policy-Based Management
• Manual enforcement of these types of policies is error prone and a time sink – Spending just 10 minutes a day on manually enforcing policies adds up to more than 40 hours at the end of a year • An entire work week!
– How much time did you have to spend during the year fixing a problem that occurred because of non-compliance?
p. 68
In the next video…
• We’ll take a look at the terms and concepts behind SQL Server 2008’s PBM
p. 69
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
70
p. 70
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 71
Policy-Based Management
• Understanding SQL Server’s policy-based administration is – Easy because we are all familiar with the basic ideas behind policy-based management – Difficult due to having to learn new abstractionlayer-type terms
p. 72
Policy-Based Management
• First, let’s talk about what you can police: – Nearly anything that has properties
• “Okay, so what items in SQL Server have properties?” – Everything
p. 73
Policy-Based Management
• “Are you saying that I can create a policy on anything I want?” – Pretty much!
• “What can’t I police?” – Anything you would normally do in a constraint or trigger – Anything you would normally do with Resource Governor
p. 74
Policy-Based Management
• Let’s now talk about the terms and concepts of Policy-Based Management (PBM) in SQL Server 2008
p. 75
Policy-Based Management
• It only takes two steps to define your policies: – Step 1: Create a condition – Step 2: Create the policy built on the condition
p. 76
Policy-Based Management
• Conditions define what you want to police – “I want to ensure that all stored procedures in our WebMarketing database start with usp_” – “No database may have AutoShrink turned on”
• This may seem like the policy but…
p. 77
Policy-Based Management
• Policies define the action you want to occur when a set of conditions is met – “A developer should receive an error if they try to create a stored procedure in our WebMarketing database that does not start with usp_”
p. 78
Policy-Based Management
• A (weak?) analogy: – Conditions are like SELECT statements •
Rows that are returned from running the query are compliant
SELECT * FROM sys.procedures WHERE name LIKE 'usp_%'
p. 79
Policy-Based Management
• So far, so good, right? – Nothing too difficult yet… – Let’s make things a bit more complicated in the next video!
p. 80
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
81
p. 81
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 82
Policy-Based Management
• In truth, PBM is quite easy • The tricky parts are: – Understanding terms like facet, property, and target – Understanding which facets allow which evaluation modes – Writing conditions and always “being positive”
p. 83
Policy-Based Management
• Facets are confusing at first – Simple mode: facets are just another word for things you already know by other names • Table, database, server, login, etc
– Advanced mode: facets are an abstraction layer that models behavior or characteristics for targets. • A target is a table, a database, a server, a login, etc
p. 84
Policy-Based Management
• Each facet contains at least one property • Properties have values • To retrieve or query a property’s value, you write an expression
p. 85
Policy-Based Management
• Example facet: The Stored Procedure facet has several properties: – Name – CreateDate – Schema
p. 86
Policy-Based Management
• To continue our previous analogy in which a condition was similar to a SELECT statement – Think of the facet as what occurs after the FROM – Think of the property as the column
p. 87
Policy-Based Management
• Consider this query: SELECT * FROM sys.procedures WHERE name LIKE 'usp_%„
• In this example, ‘sys.procedures’ would represent the Stored Procedure facet and ‘name’ would represent the facet’s Name property p. 88
Policy-Based Management
• We write our conditions against facets – Our desired condition: “All stored procedures in the WebMarketing database must begin with usp_”
• Which two facets will we use? 1. _________________________ 2. _________________________
p. 89
Policy-Based Management
• We write our conditions against facets – Our desired policy: “All stored procedures in the WebMarketing database must begin with usp_”
• Which two facets will we use? 1. Stored Procedure 2. Database
p. 90
Policy-Based Management
• Explanation: – “Stored procedure” is the facet and we must find all stored procedures whose Name property starts with usp_ – “Database” is the facet and we must find only that database whose name is WebMarketing
p. 91
In the next video…
• We’ll learn more about expressions and the mindset of writing conditions and policies
p. 92
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
93
p. 93
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 94
Policy-Based Management
• We write conditions using expressions – Not the same as a Transact-SQL expression – Same syntax as SSIS and SSRS expressions – You need to know expression syntax to be a good DBA
p. 95
Policy-Based Management
• Expression syntax is not difficult – Operators are same as C#: ==, !=, etc – Functions are similar to C# and T-SQL: • GETDATE() returns current system date/time • ISNULL() returns true/false to indicate if an expression is null
– Conditions are similar to Transact-SQL • LIKE and NOT LIKE, IN and NOT IN
– Double-quotes around strings
p. 96
Policy-Based Management
• Expression syntax is used in the graphical tools only – If you want to script out your conditions, you must write XML!
p. 97
Policy-Based Management
• Misc info about conditions: 1. Cannot change the facet of a condition after creation • Can change the expression but not the facet
2. A single condition may only operate on one facet • Can combine conditions in a policy to create complex scenarios
3. Conditions can be used by multiple policies • Design your conditions so that they can be combined with other conditions (i.e. create reusable conditions)
p. 98
Policy-Based Management
• On a production server using PBM, you will likely have hundreds of conditions and policies – It is important to have a naming convention – Opt for longer, descriptive names • Conditions have an optional Description for more information • Policies have Descriptions and “Additional help links” allowing you to create web page links in your errors
p. 99
Policy-Based Management
• Policies also have Categories (a.k.a. Policy Groups) – Can run the entire category against a database/server instead of one-policy-at-a-time – Can mandate that new databases participate
p. 100
In the next video…
• We’ll cover Execution Modes
p. 101
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
102
p. 102
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 103
Evaluations
• Anytime a policy is evaluated and finds non-compliance, it logs the offender – In the SQL Server error Log – In the Windows Application log – In the msdb database
• Be prepared for msdb growth!
p. 104
Execution Modes
• Policies also have Execution Modes which determine – When the policy will be evaluated – The response to non-compliance
p. 105
Execution Modes
• There are four execution modes: – On demand – On schedule – On change: prevent – On change: log only
• Not all facets support all execution modes
p. 106
Execution Modes
• On demand: policies are not evaluated unless manually done so by admin – Useful for audits and non-critical policies – Supported by all facets – Actions taken: • Raises error #34052 • Logs infraction
p. 107
Execution Modes
• On schedule: a SQL Server job is created that periodically evaluates the policy – Useful when evaluation performance impacts system – Supported by all database-engine facets – Job does not error out if policy violations found • Writes errors to the logs and evaluation history
– Actions taken: • Raises error #34052 • Logs infraction
p. 108
Execution Modes
• On change: log: logs non-compliance only – Useful when you want a history of policy – Supported by few facets – Actions taken: • Logs infraction
p. 109
Execution Modes
• On change: prevent: non-compliance is not allowed – Useful to prevent critical changes – Expensive – Supported by fewest facets – Actions taken: • Rolls back user transaction • Raises error #34050 or #34051 • Logs infraction
p. 110
Summary
Facet
On change: log
On change: prevent
Application Role
a
a
Asymmetric Key
a
a
Database Option
a
Database Role
a
a
Login Options
a
a
Resource Pool
a
a
Schema
a
a
Server Configuration
a
Stored Procedure, Functions, Views
a
a
Table Options
a
a
Workload Group
a
a p. 111
In the next video…
• We’ll cover whether or not policies replace permissions!
p. 112
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
113
p. 113
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 114
Writing Policies and Conditions
• Consider this security requirement: “All databases must be owned by a sysadmin” – Should this be done via policy or other means? • Answer: It depends on what action you want!
p. 115
Writing Policies and Conditions
• If you want to prevent someone who is not a sysadmin from adding a database, then this is a permission • If you want to report on the databases that are currently not owned by sysadmins, then you want a policy
p. 116
Writing Policies and Conditions
• “Do policies take the place of permissions in SQL Server 2008?” – No! – Policies and permissions actually have nothing to do with one another
p. 117
Writing Policies and Conditions
• Generally speaking, actions that can be prevented by permission are not preventable using On change: prevent – Example: preventing someone from creating databases, tables, procs, etc
p. 118
Writing Policies and Conditions
• Prevention policies are “things you have permission to do but, for some reason, management doesn’t want you doing them” – Such as creating stored procs in the WebMarketing database that do not start with usp_
p. 119
Writing Policies and Conditions
• Remember our policy: “We want all stored procedures in WebMarketing to start with usp_” – Is this a permission or a policy? • A policy however the user still needs the CREATE PROC permission in the database!
p. 120
In the next video…
• We’ll discuss how to write effective conditions and policies
p. 121
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
122
p. 122
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 123
Writing Policies and Conditions
• Lastly, writing effective policies requires practice and patience – Writing conditions can be a confusing mess • It is easy to create two policies that contradict each other
– Expect to spend a few hours “getting it right” for your first several policies and conditions • You will mess up!
p. 124
Writing Policies and Conditions
• Example policy requirement: “All databases must be owned by the sa” – This requires one condition since there is only one facet (Database facet)
p. 125
Writing Policies and Conditions
• How would you write the condition? 1. To return databases owned by the sa 2. To return databases not owned by the sa
p. 126
Writing Policies and Conditions
• How would you write the condition? – To return databases owned by the sa – To return databases not owned by the sa
• Your conditions define the ideal, not the exception • Since we want only sa-owned databases, our condition is where @Name=“sa” p. 127
Writing Policies and Conditions
• Example policy req’t: “We want all stored procedures in WebMarketing to start with usp_” – This requires two conditions since there are two facets: • •
Stored Procedure facet Database facet
p. 128
Writing Policies and Conditions
• We have the choice of writing our Stored Procedure condition so that it returns: – Stored procs that start with usp_ – Stored procs that do not start with usp_
• Which do we choose?
p. 129
Writing Policies and Conditions
• We have the choice of writing our Stored Procedure condition so that it returns: – Stored procs that start with usp_ – Stored procs that do not start with usp_
• Remember – we want our conditions to define what we do want, not what we don’t want
p. 130
Writing Policies and Conditions
• To understand why it is this way, consider our options for the second facet (Database facet): – Database named WebMarketing – Databases not named WebMarketing
• Clearly we choose the first option
p. 131
Writing Policies and Conditions
• The final step is to create a policy on these two conditions
p. 132
In the next video…
• Enough talk - let’s demo!
p. 133
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
134
p. 134
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 135
PowerShell!
• PowerShell support for SQL Server 2008 is included in SSMS • PowerShell is a very extensible and powerful scripting shell that allows admins to script (“automate”) administration functions
p. 136
PowerShell!
• If you know DOS, you can easily get started with PowerShell (“PS”) – Commands like dir, del, cd perform the same function in PS
• PowerShell can do everything you can do at the command line and more
p. 137
PowerShell!
• So why talk about PowerShell in a SQL admin course? – SQL DBAs can automate SQL Server via PowerShell scripts! – This allows for uniformity across the enterprise • Anyone who knows PowerShell can write scripts • Allows integration between systems - can load Windows Event Logs into SQL Server tables for analysis via PS
p. 138
PowerShell!
• You can manage your SQL Server by – Issuing PS commands directly into the PS shell (powershell.exe) – Executing PS files that contain PS scripts • C:\Scripts\DeployNewServer.ps1
– Creating SQL Server jobs that execute PowerShell steps
p. 139
PowerShell!
• Full coverage of PS is beyond the scope of this course however there are several concepts you will need to know • Let’s start with the basics…
p. 140
PowerShell!
• PowerShell and the OS: – Included in Windows 2008 and Windows 7 – Download for Windows 2003, Vista, XP, etc
• Versions of PowerShell – 2.0 - Windows 2008 R2 and Windows 7 – 1.0 - Windows 2008, Windows 2003, Vista, XP
• http://www.microsoft/powershell
p. 141
PowerShell!
• SQL Server 2008 ships with the sqlps.exe, a minishell – Can drop to command line and just run sqlps to manage servers – Integrated into SSMS
p. 142
PowerShell!
• The sqlps minishell incorporates several cmdlets for managing SQL Server – Pronounced “commandlets”
• cmdlets are .NET classes invoked by PowerShell at runtime • Common cmdlets included in sqlps: – Invoke-Sqlcmd – Invoke-PolicyEvaluation
p. 143
In the next video…
• PowerShell is easier to demo than talk about!
p. 144
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
145
p. 145
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 146
Unstructured Data
• Let’s compare data storage/access of 1990 to 2010 for a health insurance company • 1990: – Storage: DB2 – Tables to store an insured family: 25 – Type of data stored: row data only – Export formats: CSV, text
p. 147
Unstructured Data
• 2010: – Storage: SQL Server 2008 – Tables to store an insured family: 50 – Type of data stored: row data, xml, images, audio – Export formats: CSV, text, xml, images, audio
p. 148
Unstructured Data
• Today’s companies have to deal not just with row data but “unstructured data” as well – Legal firms need transactional support for MS Word, Excel documents – Health industry needs transactional support for imaging – Music industry needs transactional support for audio
p. 149
Unstructured Data
• Unstructured data and relational databases have a history – Most common technique: store unstructured data (“files”) in file system; store pointers to file locations in the database • Pros: fast, easy to understand, easy to implement, easy to separate storage • Cons: No transactional support, backup/restore complicated, requires separate permissions, links can become stale
p. 150
Unstructured Data
• Files are stored in a folder in the file system that requires additional permissions
p. 151
Unstructured Data
• Beginning in SQL Server 2005, we could store unstructured data in the form of files in a new data type, VARBINARY(MAX) – Replaced the older image data type – Pros: transactional support, simplify backup/restore, native SQL Server security, never stale, can separate storage using filegroups – Cons: speed, data must be converted in/out to binary format, limited to 2GB files, explosion of database growth, buffer pool memory used to access data
p. 152
Unstructured Data
• Files are stored in a VARBINARY(MAX) column as a Binary Large Object (BLOB)
p. 153
Unstructured Data
• SQL Server 2008 introduces FILESTREAM access – Sort of a blend of file system pointers and VARBINARY(MAX) techniques – Data is stored in the NTFS filesystem • Improved speed, separation of storage
– All data access is through SQL Server • Single point of security, full transactional support, fulltext indexing, no stale links
p. 154
Unstructured Data
• FILESTREAM is not for everything BLOB… – For small BLOBs, a standard VARBINARY(MAX) will offer better performance • Consider “small” to mean around 1MB in file size
– No backup compression – No Transparent Data Encryption (use Encrypted Files System instead) – Not supported in database mirroring
p. 155
Unstructured Data
• More about FILESTREAM: – Works in all editions – Allows full ROLLBACK support – Supported on clusters and in log shipping – No limit on file size • VARBINARY(MAX) is limited to 2GB • VARBINARY(MAX) FILESTREAM has no such limit
p. 156
In the next video…
• We’ll cover more DBA-related details about FILESTREAM
p. 157
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
158
p. 158
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 159
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 160
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 161
FILESTREAM
• FILESTREAM requires both SQL Server DBA and the Windows admin – The SQL Server Service must be configured for FILESTREAM access first using the SQL Server Configuration Manager
p. 162
FILESTREAM
• Inside SQL Server, FILESTREAM is an instance-level option – Disabled by default; sysadmins only can change
p. 163
FILESTREAM
• To enable FILESTREAM using TransactSQL: EXEC sp_configure ‘filestream access level’ -- 0: disabled (default) -- 1: Transact-SQL access only -- 2: Full access enabled (a.k.a. -T-SQL and streaming access -via Windows APIs) -NOTE: Local disks only for now…
p. 164
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 165
FILESTREAM
• Once you enable FILESTREAM at the instance level, you must now: 1. Create a new filegroup specifically for FILESTREAM 2. Add files to the filegroup
• You can do this with CREATE DATABASE or ALTER DATABASE
p. 166
Enable FILESTREAM on a Database
p. 167
Enable FILESTREAM on a Database
p. 168
Enable FILESTREAM on a Database
p. 169
FILESTREAM
• To create the filegroup at creation: CREATE DATABASE BigInsurance ON PRIMARY (NAME = BigInsurance, FILENAME = „D:\BigInsurance.mdf'), FILEGROUP BIDocs CONTAINS FILESTREAM( NAME = BigInsuranceDocs1, FILENAME = ‘F:\BigInsuranceFiles') LOG ON ( NAME = BigInsurance_log, FILENAME = „E:\BigInsurance.ldf')
p. 170
FILESTREAM
• To add FILESTREAM to an existing database: ALTER DATABASE [LearnItFirst.com] ADD FILEGROUP FilestreamFG CONTAINS FILESTREAM GO ALTER DATABASE [LearnItFirst.com] ADD FILE(Name=„File1‟, FileName=„F:\File1‟) TO FILEGROUP = FilestreamFG
p. 171
FILESTREAM
• To add FILESTREAM to an existing database: ALTER DATABASE [LearnItFirst.com] ADD FILEGROUP FilestreamFG CONTAINS FILESTREAM GO ALTER DATABASE [LearnItFirst.com] ADD FILE(Name=„File1‟, FileName=„F:\File1‟) TO FILEGROUP = FilestreamFG
p. 172
In the next video…
• We’ll take a look at the other requirements to use FILESTREAM
p. 173
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
174
p. 174
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 175
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 176
FILESTREAM
• You can add FILESTREAM support to new tables or existing tables – Use ALTER TABLE to add to an existing table
USE [LearnItFirst.com] GO CREATE TABLE dbo.ClaimImage ( ClaimId INT NOT NULL PRIMARY KEY , TheImage VARBINARY(MAX) FILESTREAM NOT NULL , RowId UNIQUEIDENTIFIER NOT NULL ROWGUIDCOL UNIQUE DEFAULT NEW_ID() ) -- Tables with FILESTREAM data must have a -- non-null UNIQUEIDENTIFIER column
p. 177
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 178
FILESTREAM
• Writing/reading binary data is for developers – Requires accessing Win32 APIs – .NET is popular but can write Transact-SQL directly
INSERT dbo.ClaimImage (ClaimId, TheImage) VALUES (1, 0x01)
p. 179
FILESTREAM
• DBAs needs to know about security and backup/recovery – Security permissions are done inside SQL Server, not on the file system – The physical files/folders are secured by SQL Server service account – Backup/restore will include all files
p. 180
FILESTREAM Best Practices
• Best Practices for FILESTREAM – Do not make your ROWGUIDCOL a clustered index – For best performance, FILESTREAM files should be on separate physical volumes when possible – NEWSEQUENTIALID() will perform better than NEWID() when replicating databases – Read Paul Randall’s column: • http://learnitfirst.com/go.aspx?id=WBLX
– Have your network admins (or whoever configures the physical Windows Server 2008/2003 machine) read the top part of this article: • http://msdn.microsoft.com/library/dd206979.aspx
p. 181
FILESTREAM
• Final thoughts… – Transact-SQL’s DELETE statement causes the physical file to be deleted • Physical file deletion is not performed until a CHECKPOINT operation
– Do not attempt to manage files via the file system • Cannot change names, attributes or delete files
– There are other options… • Search for “SQL Server 2008 remote BLOB storage”
p. 182
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
183
p. 183
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 184
Database Snapshots
• A database snapshot is a read-only “copy” of a database that “stops” at the point the snapshot is taken – Example: If you take a snapshot of the LearnItFirst database at 0800, the snapshot would have the entire database yet would not reflect any changes that occurred after 0800
p. 185
Database Snapshots
• About database snapshots: – Must exist on same instance as source – Available for Enterprise & Developer editions only – Debuted in SQL Server 2005 – Multiple snapshots can exist of same source db • Some sites create a snapshot for each day of the week
p. 186
Database Snapshots
-- At 0800, we execute: CREATE DATABASE LIFSnapshot_0800 ON(NAME=„LearnItFirst‟ ,FILENAME=„D:\LIF_0800.ss‟) AS SNAPSHOT OF LearnItFirst -- Logical file name must be -- same as source. Also note -- that we do not define a log p. 187
Database Snapshots
USE LIFSnapshot_0800 GO -- Returns data up through 0800: SELECT * FROM dbo.MyTable GO -- Changes after 0800 are unseen
p. 188
Database Snapshots
• Reads against the snapshot: – Will use the source database if the data requested has not changed since snapshot creation • Acquire SHARED locks at source to prevent blocking
– Will return the snapshot copy if the data has changed • This provides a “at creation” level of consistency to data
p. 189
Database Snapshots
• Database snapshots are generally used for three purposes: – Reporting/History: reads against a snapshot are consistent to the point-in-time of creation – Restore point: can restore a database using the snapshot; transactions after the snapshot was taken are lost – Testing: can test effect of changes and revert back to snapshot in the event of disaster/error
p. 190
Database Snapshots
• Example restore: -- Only allowed if: -- (1) there is only one snapshot -- (2) all source db files exist
RESTORE DATABASE LearnItFirst FROM DATABASE_SNAPSHOT = „LIFSnapshot_0800‟
p. 191
Database Snapshots
• Database snapshots use sparse files to minimize disk space – Feature of NTFS that allows NTFS to allocate disk space on an “as needed” basis – When you create a snapshot, database pages are not copied to the snapshot – As pages become dirty, the “original” data page is copied to database snapshot • When that happens, the sparse file’s size stays the same yet the “Size on disk” increases
p. 192
Database Snapshots
• A 1.1GB database with a “fresh” snapshot
p. 193
Database Snapshots
• After performing UPDATEs to source:
p. 194
Database Snapshots
• Snapshots get stale – Have to plan to delete old snapshots – If they stick around, the can become larger than source!
p. 195
Database Snapshots
• Final thoughts… – Pros: • Great way to provide users with a point-in-time view of database • Snapshots are quick and easy to create thanks to sparse files
– Cons: • No way to create using SSMS graphical tools; T-SQL only • Cannot change user permissions for snapshot vs. source – A snapshot is a read-only copy of the source
• Added overhead to source database whenever changes occur – Transaction is not complete until snapshot is up to date
p. 196
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
1
p. 1
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 2
Chapter Introduction
• Chapter 9 is about performance and exception monitoring your server • We’ll start with the basics (using Profiler) and end with new features in SQL Server 2008 (Data Collector)
p. 3
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 4
Tools of Performance Monitoring
• There are many built-in tools to monitor performance and health: – Activity Monitor and sp_who • Covered in Chapter 5!
– SQL Server Profiler • Detailed view of statements issued against instance
– Dynamic Management Views (DMVs) • “Live” view of server health and activity
– Performance counters and objects (“PerfMon”) • “Live” view of server health and activity
– Data Collector • Historical view of server health and activity
p. 5
Tools of Performance Monitoring
“But Mr. Scott, how do I know when to use each one?” Profiler
DMVs
Perf Mon
Data Coll.
Requires manual load
Requires manual load
Requires manual load
Detail
Depends…
Summary
Summary
Low
None
Negligible
Low-Hi
100-300
200-400
200-300
300-500
Provides “live”, real-time reporting? Can do trend analysis and historical reporting? Can view SQL issued? Provides detail-level view or summary? Can be used to diagnose performance problems? Level of impact on server Difficulty level
p. 6
In the next video…
• Let’s play with Profiler!
p. 7
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
8
p. 8
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 9
SQL Server Profiler
• SQL Server Profiler is a tool for monitoring real-time queries issued against your server – “Old Faithful” - been around for years – Requires sysadmin or ALTER TRACE server permission
p. 10
SQL Server Profiler
p. 11
SQL Server Profiler
• DBAs use traces to monitor statements sent to SQL Server – We’ve already seen how SQL Server uses “default traces” in Chapter 7
• Running traces will have an impact on server performance – How much? It depends on how much you trace
p. 12
SQL Server Profiler
• SQL Server Profiler is the GUI for defining, viewing, and replaying traces – It does not have to be running to create a trace – Many DBAs create their traces via Profiler and then script out to jobs
p. 13
SQL Server Profiler
• Running Profiler on a “live” system adds to an already stressful situation – Particularly if run interactively on the server
• If you must run Profiler against a “live” server, run it remotely • Adds network overhead but this is almost never an issue today
p. 14
SQL Server Profiler
• Trace data can be stored: – In a .trc file (fastest) – In a SQL Server table
• .trc files can be configured to rollover to a new file after a certain threshold • Can load .trc file contents into SQL Server table after trace for maximum flexibility!
p. 15
SQL Server Profiler
• After trace is complete, you can export results
p. 16
SQL Server Profiler
• Tips on working with Profiler – Traces can have filters to help minimize the noise • The better you filter, the less of an impact you put on the server
– Being able to write queries against the trace results can help you identify many common trends and problems • This requires you load the trace data into a table!
p. 17
In the next video…
• Let’s play with Profiler!
p. 18
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
19
p. 19
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 20
Dynamic Management Views
• DBAs use Dynamic Management Views (“DMVs”) and Dynamic Management Functions (“DMFs”) to peek under the covers – In the sys schema and begin with dm_ – Used to get up-to-the-second performance information during critical time – Often need to JOIN many together to come up with desired results p. 21
Tools of Performance Monitoring
Reminder… Profiler
DMVs
Perf Mon
Data Coll.
Requires manual load
Requires manual load
Requires manual load
Detail
Depends…
Summary
Summary
Low
None
Negligible
Low-Hi
100-300
300-500
200-300
300-500
Provides “live”, real-time reporting? Can do trend analysis and historical reporting? Can view SQL issued? Provides detail-level view or summary? Can be used to diagnose performance problems? Level of impact on server Difficulty level
p. 22
Dynamic Management Views
• There are too many DMxs to cover each one so we’ll cover concepts and core DMxs – Over 130 DMVs and DMFs in SQL Server 2008!
• It is important for DBAs to maintain a script library
p. 23
Dynamic Management Views
• The easiest way to get to know the DMVs/DMFs is to categorize them – Below are the more popular categories for performance monitoring Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 24
Dynamic Management Views
• Intellisense can help you greatly!
p. 25
Dynamic Management Views
• Object Explorer can also help
p. 26
DMV and DMF Query
• You can also query system views for full list SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_%‘ ORDER BY name p. 27
DMV and DMF Query Results
p. 28
Database Category
• Database category: – 16+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_db%'
p. 29
In the next video…
• We’ll take a look at a few categories and examples
p. 30
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
31
p. 31
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 32
Dynamic Management Views
• Reminder: the more popular categories for performance monitoring Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 33
Database Category
• Database category: – 16+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_db%‘ ORDER BY name p. 34
Database Category
• From a performance standpoint, you will spend a lot of time in the Database category – Primarily looking at table, index, and partition information • Size, fragmentation • Missing indexes • Index use
p. 35
Database Category
• Common index-related DMVs in the Database Category – sys.dm_db_missing_index_details returns information about indexes that could potentially help speed up the system – sys.dm_db_index_usage_stats can tell you whether an index is being used and, if so, in what way
p. 36
Database Category
• Common index-related DMFs in the Database Category – sys.dm_db_index_physical_stats is one way to identify fragmentation (see Chapter 8!) – sys.dm_db_index_operational_stats is similar to the DMV sys.dm_db_index_usage_stats yet only includes actual execution events instead of plan generation events
p. 37
In the next video…
• We’ll take a look at writing queries using the DMVs and DMFs in the Database category
p. 38
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
39
p. 39
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 40
Dynamic Management Views
• Let’s look at the other categories now… Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 41
Execution/Session Category
• Execution/Session category: – 20+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_exec%'
p. 42
Execution/Session Category
• The Execution/Session category will help you spy on people – That’s what you really want to do, isn’t it? – Can also view specific query information • Plans, text, stats, caching information
p. 43
Execution/Session Category
• There is at least one DMF in this category that you definitely want to know – sys.dm_exec_sql_text allows you to see the query text a user submits • By itself, it is not that useful but combined with other Execution/Session DMVs it is very powerful • Accepts one parameter - the memory location of the query plan – Populated via a column found in other sys.dm_exec_ DMVs
p. 44
Execution/Session Category
• A common DMV in this category is: – sys.dm_exec_query_stats allows you to see
metrics about queries submitted to server • Reads, writes, waits, execution count, etc • plan_handle column is the column to pass to sys.dm_exec_sql_text to return SQL
p. 45
Return 50 most-run queries
SELECT TOP(50) sql.text , qs.last_execution_time , qs.execution_count , qs.last_elapsed_/1000000.0 as ElapsedTimeInSecondstime FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS sql ORDER BY qs.execution_count DESC
p. 46
Return 50 most-run queries
p. 47
Execution/Session Category
• Other common DMVs in this category are – sys.dm_exec_procedure_stats same as query_stats yet only returns stored procedure
info – sys.dm_exec_cached_plans gives you info on
the cached query plans • Is it compiled or adhoc? • How many times has it been used?
p. 48
Execution/Session Category
• Spying on users is easy! – sys.dm_exec_sessions is one row for each
authenticated session
-- Returns all non-internal sessions SELECT * FROM sys.dm_exec_sessions WHERE session_id>50
p. 49
In the next video…
• We’ll write several queries and see the Execution/Session category in action
p. 50
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
51
p. 51
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 52
Dynamic Management Views
• Let’s look at the IO and OS categories now to help diagnose disk pressure Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 53
IO Category
• IO category: – 4+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_io_%‘ ORDER BY name p. 54
IO Category
• The most common item in this category is – sys.dm_io_virtual_file_stats() returns information about usage of the database files • Reads, writes, IO waits, size • Accepts two parameters: – database_id - use NULL for all databases – file_id - the file_id from sys.database_files; use NULL for all files
p. 55
IO Category
• sys.dm_io_virtual_file_stats
returns several columns: Column
Description
sample_ms
Milliseconds since server was restarted
num_of_reads
Physical reads of objects in database file
num_of_bytes_read
Large textual-based columns will affect this greatly
io_stall_read_ms
Total time a user process waited for IO to this file
num_of_writes, num_of_bytes_written, io_stall_write_ms
Database write activity
io_stall
Total amount of waits reading /writing to this file
p. 56
IO Category
• Here are some things to monitor with dm_io_virtual_file_stats() – Look at total reads+writes to find “most active” – Look at io_stall to identify files that might benefit from new disks • All databases have stalls; the important metric is “What percentage of the uptime of the server am I having stalls?” • Compare read/write stalls on log and data file to identify which could benefit from new/faster disks
p. 57
IO DMF -- View IO stats for all DBs on instance: SELECT DB_NAME(mf.database_id) AS DB , mf.physical_name, mf.type_desc , vfs.* FROM sys.dm_io_virtual_file_stats(null,null)vfs JOIN sys.master_files mf ON mf.database_id = vfs.database_id AND mf.file_id = vfs.file_id ORDER BY DB_NAME(mf.database_id)
p. 58
IO DMF
p. 59
IO DMF -- Aggregate total IO since last restart: SELECT DB_NAME(vfs.database_id) AS database_name, CAST(SUM(vfs.num_of_bytes_read + vfs.num_of_bytes_written) / 1048576. AS DECIMAL(12, 2)) AS IO_MB FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs GROUP BY vfs.database_id ORDER BY IO_MB DESC
p. 60
OS Category
• The OS category is one of the largest – 36+ DMVs (no DMFs!)
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_os_%‘ ORDER BY name p. 61
OS Category
• You can use this category to find waiting tasks: – sys.dm_os_wait_stats - aggregated info on dozens and dozens of wait types – sys.dm_os_waiting_tasks - detail info on waits for a specific resource • Hint: use dm_os_wait_stats to identify the areas of contention and then dm_os_waiting_tasks to view task information
p. 62
OS Category
• These two DMVs are accumulative – Their statistics are the totals since SQL Server service has been started
• To track effectively over time, it is necessary to reset periodically – Common to store in a table for historical perspective – DBCC SQLPERF(‘’, CLEAR)
p. 63
OS Category -- Current wait info: SELECT wait_type, wait_time_ms / 1000. as wait_time_s FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC -- Clear out stats: DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR) -- Run this at a later time to view waits since CLEAR SELECT wait_type, wait_time_ms / 1000. as wait_time_s FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC
p. 64
OS Category
p. 65
OS Category
• To identify disk pressure, you can look in sys.dm_os_wait_stats for specific wait types – Order by wait_time_ms DESC to identify most often waits – If anyone of these are in the top two positions, you have disk I/O pressure: • • • • •
ASYNCH_IO_COMPLETION IO_COMPLETION LOGMGR WRITELOG PAGEIOLATCH_x
p. 66
In the next video…
• Let’s look further at the OS category and others
p. 67
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
68
p. 68
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 69
OS Category
• The OS category has lots of info – sys.dm_os_sys_info - info such as number of CPUs, start time of service – sys.dm_os_tasks - lists each running task – sys.dm_os_loaded_modules - lists each loaded DLL that SQL Server is using • • • •
Cryptography Manager Kerberos Security Package Microsoft SQL Server Native Client 10.0 SQL Server Windows 95 Lite SCM
p. 70
OS Category
• Many OS category DMVs can help diagnose memory pressure – Performance Counters can help too – We’ll cover Perf Counters in an upcoming section of this chapter
p. 71
OS Category
• The main areas for memory pressure are: – Database page cache (used to cache table/index) – Query Workspace memory (for Hash and Sort ops) – Plan cache – There are others…
p. 72
OS Category
-- Find out what has the most entries -- in memory SELECT TOP (20)[Name], COUNT(*) AS 'Count' FROM sys.dm_os_memory_cache_entries GROUP BY [Name] ORDER BY COUNT(*) DESC;
p. 73
OS Category
p. 74
OS Category -- Top 10 consumers of Buffer Pool SELECT TOP 10 [type] , SUM(single_pages_kb) AS memory_in_kb FROM sys.dm_os_memory_clerks GROUP BY type ORDER BY SUM(single_pages_kb) DESC;
p. 75
OS Category
p. 76
OS Category
• One area that many servers have trouble with is the TokenAndPermUserStore cache – An internal security system that generates security tokens for sessions – Particularly troublesome area for DBs with lots of adhoc SQL – The problem can be that the TokenAndPermUserStore cache takes up so much memory and is so frequently accessed that when there are a high number of items in the cache, access takes a long time
p. 77
OS Category
1. Queries for diagnosing TokenAndPermUserStore cache
issues: -- What is the size? The larger, the slower! SELECT SUM(single_pages_kb + multi_pages_kb) AS ‘CurrentSizeOfTokenCache(kb)’ FROM sys.dm_os_memory_clerks WHERE name = 'TokenAndPermUserStore'
p. 78
OS Category
2. Queries for diagnosing TokenAndPermUserStore cache
issues: -- How many entries in the -- TokenAndPermUserStore? SELECT TOP (20)[Name], COUNT(*) AS 'Count' FROM sys.dm_os_memory_cache_entries GROUP BY [Name] ORDER BY COUNT(*) DESC;
p. 79
Scary!
p. 80
OS Category
• There are multiple ways to prevent TokenAndPermUserStore issues 1. Minimize ad-hoc SQL 2. Log in as a sysadmin (no tokens need to be generated) 3. Limit the size of the cache via sp_configure using • • •
access check bucket count access check cache quota http://support.microsoft.com/kb/955644
p. 81
OS Category
• There is one way to fix a “live” TokenAndPermUserStore issue – Free the system cache -- WARNING: Do not use on production without -a full understanding! -- After running, there will not be an entry -in sys.dm_os_memory_clerks for TAPUS!
DBCC FREESYSTEMCACHE('TokenAndPermUserStore')
p. 82
In the next video…
• Let’s look at blocking and locking…
p. 92
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
93
p. 93
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 94
Performance Monitor
• To maximize coverage, you want to use both DMVs/DMFs along with Performance Monitor – a.k.a. PerfMon and System Monitor
• PerfMon gathers stats about SQL Server as well as the rest of the environment – Disk subsystems, non-SQL resource usage
p. 95
Performance Monitor
• PerfMon uses objects and counters – Objects are for categorization – Objects contain one or more counters
• When you install SQL Server, it installs a set of instance-specific objects
p. 96
Performance Monitor
• You can view the SQL Server objects and counters using the DMV sys.dm_os_performance_counters – Careful - more than 1,100 rows
p. 97
Performance Monitor
• A few things to know about
sys.dm_os_performance_counters – Only displays SQL Server instance’s counters – Any counter that has “/sec” in the name is cumulative since time SQL Server service was started • Transactions/sec • Bulk copy rows/sec
– To accurately read these values, compare intervals • Compare 0800 reading to 0900 reading, for example
p. 98
Performance Monitor
• Example query to return all deprecated features usage – Many feature usages are actually SQL Server internals and tools! SELECT * FROM sys.dm_os_performance_counters pc WHERE object_name LIKE '%Deprecated Features%' AND cntr_value > 0 ORDER BY counter_name p. 99
Performance Monitor
p. 100
Performance Monitor
• Querying for specific counters allows adhoc views of point-in-time information – But no graphs, charts, pretty colors
• For trend analysis, you have a few choices: – Store the query results in a table for later review – Use PerfMon to store logs for later analysis
p. 101
Performance Monitor
• Some DBAs will store certain values in a table for later querying or for historical perspective – Watch the rest of the chapter prior to attempting this! -- Create a new table (”PerfCounters”) -- and store current counters
SELECT * INTO PerfCounters FROM sys.dm_os_performance_counters
p. 102
Performance Monitor
• All DBAs will use PerfMon
p. 103
In the next video…
• Let’s take a look at specific objects and counters
p. 104
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
105
p. 105
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 106
Performance Monitor
• There are a few specific areas you will want to monitor – Sometimes you can find that “Aha!” moment using ad-hoc monitoring – Sometimes you can only find that “Aha!” moment using trend analysis
p. 107
Performance Monitor
• A few topics before we go further: – PerfMon, counters with “/sec” in the name are not cumulative • Makes it easy to do ad-hoc monitoring
– PerfMon’s GUI defaults to taking measurements every one second • Too often for trend analysis • 30-seconds is generally “good enough” for ad-hoc troubleshooting
p. 108
Performance Monitor
• A few areas DBAs want to monitor: – Disk usage – CPU usage – Memory usage
• Let’s look at each!
p. 109
Performance Monitor
• A few areas DBAs want to monitor: – Disk usage – CPU usage – Memory usage
p. 110
Disk Usage
• Disk usage monitoring – Disk I/O is commonly a performance stealer – While SQL Server manages the “when” and “how often” of disk I/O, it is the operating system that performs I/O – When monitoring disk activity, you need to monitor both SQL Server’s disk activity and the disk activity of the server as a whole
p. 111
Disk Usage
• Disk I/O cannot be isolated as a metric – Example: You are monitoring disk I/O and notice that you have waits for disk reads/writes. Does this always mean that you need faster/more disks? • No! Could be that you are paging constantly and memory is the real bottleneck
p. 112
Disk Usage
• Important system objects and counters: Object
Counter
When to worry
PhysicalDisk
% Disk Time
Consistently > 90%
PhysicalDisk
Current Disk Queue Length
Particularly helpful when “% Disk Time” is high; allows to see how many I/O operations are waiting. Start worrying when this value is more than 1.5x the number of spindles
PhysicalDisk
Avg. Disk Queue Length
In many configs, a disk has 1 spindle. Start worrying when this value is more than 1.5x the number of spindles
Memory
Page faults/sec
“Some” page faults are normal; look for consistently high numbers
p. 113
Disk Usage
• PhysicalDisk: % Disk Time – Tells us how “busy” your array is – With this counter, you want to worry about sustained activity, not bursts – A % Disk Time of more than 75% for several continuous minutes occurring several times a day can likely be resolved: • By adding more disks to the array • By using faster drives
p. 114
Disk Usage
• PhysicalDisk: % Disk Time (cont.) – When you do notice this is high, start looking at the PhsyicalDisk: Current Disk Queue Length • How many disks are in my array? – Tells you how many spindles you have
• How many I/O tasks are waiting?
– If the number of waiting tasks is 1.5 or higher per spindle, you need better I/O performance!
p. 115
Disk Usage
• PhysicalDisk: Avg. Disk Queue Length – Monitor this over time • Business hours • 24 hour period • Peak times
– If you find that it exceeds more than 1.5 waiting tasks per spindle for continuous periods, increase disk performance
p. 116
Disk Usage
• Important SQL Server objects & counters to help diagnose I/O trouble – Object: • SQL Server: Buffer Manager
– Counters: • Page reads/sec • Page writes/sec
p. 117
Disk Usage
• Page reads/sec and Page writes/sec measure physical reads, not logical reads – Not logical reads! – Object: • SQL Server: Buffer Manager
– Counters: • Page reads/sec • Page writes/sec
p. 118
One Caveat
• High disk usage is not always a sign that you need more/faster disks – Sometimes poorly optimized queries can masquerade as disk/memory/CPU issues – Proper indexing and better queries can help eliminate slow queries as the culprit
p. 119
A Final Caveat
• PerfMon is great for monitoring disks directly attached to the server – Good but not great for monitoring SANs – Check with your SAN vendor for suggestions on monitoring disk activity
p. 120
In the next video…
• Let’s play with PerfMon and look at Disk, CPU, and memory counters
p. 121
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
122
p. 122
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 123
Data Collector
• While it is fun and educational to learn the various DMVs and DMFs along with the relationships between various system views, it is a huge time sink – Pro tip: download the SQL Server 2008 System Views map – http://learnitfirst.com/go.aspx?id=5GR5
p. 124
Data Collector
• How to be an unsuccessful DBA: – Try to remember all the DMVs, DMFs, and Performance Counters – Perform all of your performance data collection by running 100+ line queries daily • Bonus points if you are the only person who knows where on the network these queries/files are located
– Spend 20 hours a week writing and running DMVbased queries
p. 125
Data Collector
• How to be an successful DBA: – Learn the most important DMVs, DMFs and Performance Counters for your situation • Every job is different and every database is different • OLTP databases need different metrics than OLAP • DBAs of a “single database only” server are often concerned with different things than DBAs of “20 database” servers
– Make your reporting easy to digest and easy to run • No more 20 hours a week manually writing/running DMVbased queries!
p. 126
Data Collector
• Life as a DBA has traditionally been spent doing a lot of manual collection of performance data 1. 2. 3. 4.
Create a tables to store performance data Write the queries to collect the data Create a job to store query results in tables Write queries against tables to determine trends, etc
p. 127
Data Collector
• This was how things were done for more than a decade – As recent as SQL Server 2000… – SQL Server 2005 debuted “Custom Reports” to help us • Allowed integration of SSMS with Reporting Services to create professional quality reports based on our queries • “SQL Server 2005 Performance Dashboard” reports
p. 128
Data Collector
• SQL Server 2008 includes the Data Collector to save you time – Creates a Management Data Warehouse (MDW) that stores critical tuning, planning, and activity history – Used daily by DBAs to do capacity planning, mitigate risks, free up time otherwise spent trying to write complex DMV/DMF queries!
p. 129
Data Collector
• The Data Collector is turned off by default – Enabling it is not a trivial decision… – Performance data collection will impact your system • It’s the Heisenberg uncertainty principle in action! • How much of an impact depends on what you collect and how often
p. 130
Data Collector
• A disclaimer: – Please test, test, test this prior to deploying in a production environment – This is version 1.0 software – Easy to “over collect” • 30GB of data collection in three days!
p. 131
Data Collector
• Q& for Data Collector – “What type of information can DC collect?” • Traces, PerfMon counters, SQL queries
– “Can I use DC to collect SQL 2005/2000 servers?” • No, SQL Server 2008 only
– “Is this another one of those ‘Enterprise Edition only’ features?” • No - works great in Standard and Workgroup Editions too!
p. 132
Data Collector
• Q& for Data Collector (cont.) – “How much of an impact will this have on my server?” • Microsoft says “5% of CPU and 250-350MB” for the built-in collection sets. Custom collection sets that you create may consume more – More on this later…
p. 133
In the next video…
• Let’s learn a bit more about the Data Collector
p. 134
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
135
p. 135
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 136
Data Collector
• The Data Collector (“DC”) is built into SSMS
p. 137
Data Collector
• You can enable the DC via the Configure Management Data Warehouse Wizard
p. 138
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 139
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 140
Data Collector
• The MDW is a relational data warehouse to store all of your metrics for later analysis
p. 141
Data Collector
• When you create the MDW database, you define the name and physical file location • Placing the MDW database on either a dedicated server or dedicated disks will minimize performance impact to monitored servers
p. 142
Data Collector
• The MDW data can be shared with nonsysadmins – All sysadmins have access – Often want network admins to have access to performance data without having access to specific databases
p. 143
Data Collector
• There are three databases roles you can assign:
p. 144
Data Collector
• Once you decide security, you are ready to build the MDW
p. 145
In the next video…
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 146
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
147
p. 147
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 148
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 149
Data Collector
• Back through the wizard…
p. 150
Data Collector
• The Cache directory is local storage on the monitored machine – Remember that this can be run from a central server
p. 151
Data Collector
• Data collector will create SQL Server jobs to perform data collection • The job has two main components: 1. Collect the info 2. Upload the info to the MDW
• Where you collect the info locally is the cache directory – Generally the %temp% folder
p. 152
Data Collector
• Ready, set, go!
p. 153
Data Collector
• What it should look like
p. 154
Data Collector
• Now comes the work… – You can use the System Data Collection Sets – You can create your own collection sets
• There is no graphical way to create your own collection sets – Must write Transact-SQL
p. 155
In the next video…
• We’ll take a look at enabling the Data Collector and viewing the results
p. 156
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
What We’re Going to Cover
• Our final chapter is an overview of the various areas related to high availability – Log Shipping – Database Mirroring – Clustering – Peer-to-peer replication* • Not exactly “high availability” but often used in conjunction with the above
p. 3
High Availability
• High availability is a design that attempts to ensure users always have access to the database – Unplanned unavailability in some systems is not acceptable • Flight systems, emergency response teams, military operations
– Planned unavailability is allowed for maintenance
p. 4
High Availability
• Availability and uptime are not the same thing – A system can be up yet, if the network is down, it is not available
• Downtime could be a combination of things – – – –
Time spent identifying problem Time spent repairing problem Restore time Service or OS restarts
p. 5
High Availability
• To calculate total availability of a system, summarize: – Network downtime – Application downtime – Database downtime – Operating system downtime
p. 6
High Availability
• To increase a system’s availability – Decrease the amount of downtime – Decrease the frequency of downtime
p. 7
High Availability
• Applications and services often feature service level agreements (“SLAs”) that list minimum availabilities: – 90% availability means allowing 36.5 days of unplanned unavailability – 99% availability means allowing 3.65 days of unplanned unavailability – 99.999% (the “five nines”) availability means allowing ~00:05:15 of unplanned unavailability
p. 8
High Availability
• As a SQL Server DBA, your job is to ensure the availability and uptime of your – Server/service – Database(s)
p. 9
High Availability
• In this chapter, you will learn about various built-in techniques to help you achieve high availability – Some are expensive to implement – In all cases there are 3rd party solutions which may offer better alternatives – Often you will combine techniques to increase both uptime and availability • Example: A clustered instance with database mirroring
p. 10
In the next video
• Let’s get started with log shipping!
p. 11
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Log Shipping Best Practices
• There are a few categories of Best Practices as relate to Log Shipping – Backup and Restore – Network – Security
p. 3
Log Shipping Best Practices
• Backup and Restore Best Practices for Log Shipping: 1. Give enough time for your backups to complete •
If you schedule backups every 5 minutes and a backup takes 7 minutes to complete, what happens to the next backup? –
Answer: The next backup will not take place until the next scheduled time
p. 4
Log Shipping Best Practices
• Backup and Restore Best Practices: 2. The more frequently you backup, the larger MSDB becomes •
Check up on sp_delete_backuphistory stored proc for clearing out msdb periodically
3. Use backup compression if you can •
Makes for faster backups, faster network copies, and faster restores - just make sure you have the CPU power to do it!
p. 5
Log Shipping Best Practices
• Backup and Restore Best Practices: 4. The more frequently you backup, the less possible data loss at the target •
SQL Server 2008 allows for as frequent as 10 seconds!
p. 6
Log Shipping Best Practices
• Network Best Practices for Log Shipping 1. Fast networks are good; slow networks are bad
p. 7
Log Shipping Best Practices
• Network Best Practices for Log Shipping 2. If you can’t do native backup compression (available in Enterprise only) and speed of file copy is an issue, consider using 3rd party compression • • •
It does no good to take backups every 5 mins if they take 15 mins to copy 7-Zip is an industry favorite despite it’s being “unsupported” Depending on the type of data, can compress 1:1, 5:1, 20:1 or more
p. 8
Log Shipping Best Practices
• Security Best Practices for Log Shipping 1. Secure all folders that participate in log shipping • •
Having an unauthorized person get to backups is terrible Having a sysadmin who is trying to “clean up disk space” delete critical backup files means you cannot restore
p. 9
Log Shipping Best Practices
• Security Best Practices for Log Shipping 2. Use your log shipping backups in your normal disaster recovery scenario •
•
When you take a log backup for log shipping, that is your log backup for your local disaster recovery plan as well Make sure you copy to the same location as your FULL/DIFF backups!
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Database Mirroring Best Practices
• There are a few categories of Best Practices as relate to Database Mirroring – Architecture – Network – Security is not generally in this list • All data transfer is encrypted if using same domain
p. 3
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: – “Architecture” is an all encompassing term that can include • • • •
Transaction safety levels Editions used Size of database(s) Usage scenario(s)
p. 4
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 1. Transaction safety levels can influence the performance of database mirroring • Safety OFF (i.e. asynchronous mirroring) means that transactions are committed at principal without waiting for confirmation from witness/mirror – If transactions cannot be sent, it forms a queue at principal
• Safety FULL (synchronous mirroring) requires confirmation from mirror that the log has been hardened
p. 5
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 1. Transaction safety levels can influence the performance of database mirroring • Because of how differently applications use a database, it is impossible to just say, “Always use Safety ‘X’ because it performs better” • Only one thing you can do: read the whitepaper! – http://technet.microsoft.com/library/cc917681.aspx
p. 6
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 2. Use Enterprise edition for maximum configurability • Standard edition can only run Safety FULL transaction safety level – Safety FULL will have an impact on transaction throughput at the principal
• Thankfully log compression is enabled in both Standard and Enterprise editions – Log compression is a new feature in SQL 2008
p. 7
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 3. Get creative when it comes to bulk imports • Database mirroring requires FULL recovery model • You can: – – – – –
Disable mirroring Perform the steps to bulk import Switch back to FULL recovery Perform a log backup and restore the log backup at the target Turn on mirroring again
p. 8
Database Mirroring Best Practices
• Network Best Practices for Database Mirroring: 1. Consider using database mirroring for local or metro-area networks •
The farther away the servers are, the longer it takes to stream the transactions – Also introduces higher potential for communication issues
• Network roundtrip times can be 100x faster on a LAN than on a WAN (Wide Area Network) – 1ms vs. 100ms
p. 9
Database Mirroring Best Practices
• Network Best Practices for Database Mirroring: 2. Faster networks mean promotions and plaudits •
Slow networks mean updated resumes and m-e-e-t-in-g-s to explain why it doesn’t work like the vendor said it would…
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
SQL Clustering Best Practices
• There are a few categories of Best Practices as relate to SQL Clusters – Hardware – Installation – Configuration
p. 3
SQL Clustering Best Practices
• Hardware best practices involve: 1. Do not use the quorum disk for any SQL Serverrelated files 2. Use identical network adapters with identical software/drivers 3. Manually set the speed and duplex mode of any adapters that allow multiple speeds •
Setting to “Auto Select” may result in dropped packets
p. 4
SQL Clustering Best Practices
• Hardware best practices involve (cont.): 4. Since the clustered disks are a single point of failure, ensure they are sufficiently redundant •
RAID 10 is quite popular
p. 5
SQL Clustering Best Practices
• Installation best practices involve: 1. Start the installation from the node that owns the disk resources 2. If you have to install SQL 2008 onto a cluster with SQL 2000 already on it, install SP4 for SQL 2000 on all nodes first, then install SQL 2008 onto the cluster 3. For a 2008 install onto a pre-existing 2005 cluster, install SQL 2005 SP2 first and then SQL 2008 p. 6
SQL Clustering Best Practices
• Configuration best practices involve: 1. On Active/Active clusters, make sure to set max server memory for each instance so that you allow for multiple instances running on one node at the same time 2. Consider using a slipstreamed installation DVD •
Slipstreaming is the process of creating an installation media that already contains a service pack and/or hotfix
p. 7
SQL Clustering Best Practices
• Configuration best practices involve (cont.): 3. Integration Services is not installed as a clustered resource •
Although you could make it a clustered resource, it is better to install it locally to each node
p. 8
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
SQL Server Replication
• Replicating data is the process of copying data and table definitions from database to database – Example: • User enters data at the Los Angeles server • Replication copies the change servers in Sydney, London, and Mumbai • After copying the change, all servers have the same data
p. 3
SQL Server Replication
• Basic terms: – Publisher: the source database – Publication: the tables, views, and stored procedures that the publisher is publishing – Subscriber: the database receiving the changes – Subscription: how the subscriber receives the data – Distributor: the server responsible for delivering the publication to the subscriber
p. 4
SQL Server Replication
Publisher (source)
Distributor
Subscribers
p. 5
SQL Server Replication
• When the data is copied to a database, it is considered synchronized – Synchronization can be instant or scheduled • Instant - as the change is entered at Publisher it is instantly replicated; Subscribers and Publisher always have the same data • Scheduled - allows latency between when data is entered at Publisher and when Subscribers have data
p. 6
SQL Server Replication
• There are three basic types of replication: – Snapshot – Merge – Transactional
• Let’s look at each!
p. 7
Snapshot Replication
• Snapshot replication: – Publisher generates a snapshot - a point-in-time copy of the publication – Distribution then replaces copy of subscribers’ data with the snapshot • Typically one-way replication (data entered only at publisher)
– Analogy: Newspaper industry
p. 8
Snapshot Replication
p. 9
In the next video…
• Let’s continue our tour of the types of replication with Merge and Transactional
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Merge Replication
• Merge replication: – Basic premise: Anyone can change the data – Transactions are replicated • Unlike Snapshot replication which replicates entire snapshot
– Changes are synchronized between all servers
p. 3
Merge Replication
• Merge replication: – Setup flow: 1. Create the publication and generate the snapshot 2. Distribute snapshot to subscribers –
At this point, Publisher and Subscribers have same data
3. Someone changes the data (could be Pub or Sub) 4. Merge Agent handles merging the data so that, in the end, everyone has the same data
p. 4
Merge Replication
• Conflicts can occur in Merge Replication – What happens when two sites change the same data at the same time?
• You can help lessen the chance of a conflict: – You can specify that changes to the same row by different sites is allowed provided that they change different columns – At synchronization, the Merge Agent merges the row and synchronizes all servers so that everyone has the same data
p. 5
Merge Replication
• You can define how conflicts are resolved in several ways: – By setting priority levels for each site • In the event of a conflict between two sites, the highest priority site’s changes “win”
– By setting, “First in wins” or “Last in wins” • Whoever made the change first/last “wins”
– By writing custom business rules
p. 6
Merge Replication
p. 7
Transactional Replication
• Transactional replication: – Transactions are replicated between servers – Transactional can: • Be one-way: only the Publisher can replicate changes • By multi-way: changes can occur at any site
– Transactional may or may not have conflict detection • Depends on topology • Rule of thumb: “Last in wins” unless you configure differently p. 8
Transactional Replication
p. 9
P2P Transactional Replication
• Peer to Peer (P2P Transactional replication: – Allows both scale-out and high availability • Easily scales since each site acts as Publisher and Subscriber (called “Nodes”) • Every site has the same data so clients can connect to any site thus allowing a degree of load balancing • If one site goes down, users just connect to another site
– Debuted in SQL 2005
p. 10
P2P Transactional Replication
p. 11
P2P Transactional Replication
• Conflicts can occur in P2P – What happens when a user connected to the Dallas node updates a row and, at the same time, a user on the Houston node deletes that row? • Default: Last in wins • Can enable “conflict detection” - default setting is to stop distribution when a conflict is found
p. 12
Let’s demo!
• Let’s demo P2P!
p. 13
Presented by Scott Whigham
It All Began in the 80s
• In 1989, Microsoft (“MSFT”) was a $200 million a year company with no database product • Ashton-Tate had the popular dBASE package • Sybase had DataServer, a new product
It All Began in the 80s
• MSFT signed deal to license Sybase’s DataServer on OS/2 and other MSFTbased operating systems • MSFT convinced Ashton-Tate to help them market the product • Ashton-Tate/Microsoft SQL Server – Version 1.0 shipping in May, 1989
It All Began in the 80s
• Microsoft SQL Server 1.1 shipped in 1990 – No more Ashton/Tate – Now for Windows 3.0
The 1990s
• Microsoft SQL Server 4.2 release timeline – There was no 2.x or 3.x – 16-bit originally for OS/2 released in 1992 – Windows NT 3.1 32-bit release in 1993
The 1990s
• Original license with Sybase was restrictive – MSFT could not add or change features without Sybase’s approval – Sybase and MSFT parted ways in 1994
The 1990s
• Next up: SQL95 – SQL95 was released in Spring, 1995 • Technically Microsoft SQL Server 6.0
– Replication is here! – MSSQL was now one of The Big Guys in the database world • Sybase, Informix, Oracle, and IBM still strong
The 1990s
• SQL Server 6.5 in 1996 – Short release cycle due to pressure from other companies – Can you say ”ANSI Standard Compliant”? – Still built on the “old” Sybase code
The 1990s
• SQL Server 7.0 in 1998 – Total rewrite – no more Sybase code! – Language is still Transact-SQL but it starts to fork – Tons of new features • Debut of Data Transformation Services • Row-Level locking • OLAP Services debuts
The Millenium
• SQL Server 2000 in 2000 – Originally planned as SQL Server 7.5 – Not that many new features but far more powerful – Built to work with Windows Server 2000 – OLAP Services becomes SQL Server Analysis Services – Reporting Services debuts a few years later as a ”free” add-on
The Millenium
• SQL Server 2005 in 2005 – Delays caused by integration with .NET team – Made to work with Windows Server 2003 – Many new features: • • • • • •
SQL CLR Integration Services Full integration of Reporting Services Database Mirroring Tighter XML integration Schemas!
The Millenium
• SQL Server 2008 in 2008 – Not a huge step but built for Windows Server 2008 • This allows for more memory, better 64-bit support
– Some new features but primarily enhancements • • • • •
Transparent data encryption Management policies Geo-spatial data and datatypes Filestream Backup compression
The Millennium
• SQL Server .NEXT – What’s next? – Business Intelligence (“BI”) continues to grow – Storing images is hot – Will we ever see an O.S. built on SQL Server?
Editions of SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Two Main Pain Points
• Microsoft SQL Server® 2008 comes in two flavors: – Server Editions • Enterprise, Standard
– Specialized Editions • Express, Developer, Workgroup, Web, Compact
Processor Architecture
• Microsoft SQL Server® 2008 is further segmented by processor architecture: – x86 • All
– x64 • All except Compact
– IA64 • Enterprise and Developer only
Server Editions • Enterprise (x86, x64, IA64) • Standard (x86, x64)
Specialized Editions • Developer (x86, x64, IA64) • Workgroup (x86, x64) • Web (x86, x64) • Express (x86, x64) • Compact (x86)
Compact Edition
• SQL Server 2008 Compact Edition – Compact is meant to run on phones/portable devices – Prior names include “SQL Server for Windows CE” and “SQL Server Mobile Edition”
• Targets the .NET Framework 3.5 • Free! • Not discussed further in this videobook
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Server Editions
• The Server Editions target companies running Windows Server who need: – Large databases – Lots of users – Scalability and availability
• Enterprise and Standard Editions are also the most expensive!
Enterprise Edition
• Enterprise Edition is the most powerful edition available – No hard limit for CPUs or Memory (OS maximum) • This allows for VLDBs (Very Large Databases)
– Expensive to buy – Used for mission-critical databases and data warehouses where uptime is paramount (even across distances)
Enterprise Edition
• Enterprise Edition is: – Expensive to buy • Averages around 5x Standard Edition cost
– Used for mission-critical databases and data warehouses where uptime is paramount (even across distances)
Standard Edition
• Standard Edition is the next most powerful edition – 4 CPU limit but no Memory limit (OS maximum) • This allows for KLDBs (Kinda Large Databases) – Okay, so that’s not technically an industry term…
– Used when needs are similar to Enterprise Edition but budget isn’t available • Even with just 4 CPUs you can potentially handle 100s of GBs of data
Standard Edition
• Standard Edition is the next most powerful edition – 4 CPU limit but no Memory limit (OS maximum) • This allows for KLDBs (Kinda Large Databases) – Okay, so that’s not technically an industry term…
– Used when needs are similar to Enterprise Edition but budget isn’t available • Even with just 4 CPUs you can potentially handle 100s of GBs of data
Comparing the Server Editions
Enterprise Edition Business Critical Large scale online transaction processing (OLTP) Large scale reporting Data warehousing
Advanced analytics Server consolidation Departmental Small-to-medium scale OLTP Reporting and analytics
Standard Edition
Comparing the Server Editions
Enterprise Edition
Standard Edition
O.S. maximum
4
50
16
Number of clustered nodes
16
2
Database mirroring
Full
Single-thread
Number of CPUs Number of Instances Hypervisor support Database snapshots Parallelism, index, and DBCC enhancements
Hot-add CPU, memory Fast recovery
In the Next Video…
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Editions of SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Server Editions • Enterprise (x86, x64, IA64) • Standard (x86, x64)
Specialized Editions
• Developer (x86, x64, IA64) • Workgroup (x86, x64) • Web (x86, x64) • Express (x86, x64) • Compact (x86)
Specialized Editions
• The Specialized Editions are for specific situations – Developer Edition: for testing and development • Same feature-set as Enterprise Edition but different license, O.S. & hardware requirements/limits
– Workgroup: for branches, small depts • Great for replication to remote sites
Specialized Editions
• Web: ideal for web hosts – Licensing allows for low Total Cost of Ownership (TCO)
• Compact: for mobile devices – Usually sync up with Enterprise/Standard to upload/download new data
Specialized Editions
• Express: free; great for learning or redistributing – Perfect for very small databases with small numbers of users – The most confusing edition since there are three “versions” of Express
The Express Editions
• There are three “flavors” of Express: – SQL Server Express • The SQL Server database but without tools, reporting
– SQL Server Express with Tools • Same as above but with feature-limited versions of management and development tools
– SQL Server Express with Advanced Services • Same as SQL Server Express with Tools but with reporting, more admin
What We’re Going to Cover
• Editions • Processors • Compact Edition
• Server Editions • Specialized Editions
• Reference Tables
Comparing the Editions
Enterprise/ Developer
Standard
Workgroup
Web
Express
O.S. maximum
4
2
4
1
50
16
16
O.S. maximum
O.S. maximum
4GB
O.S. maximum
1GB
Number of clustered nodes
16
2
Database mirroring
Full
Single-thread
Witness only
Witness only
Witness only
Replication Support
Full
Full
Subscriber only
Subscriber only
Subscriber only
Number of CPUs
Number of Instances Memory Database snapshots Log shipping
Data Compression Transparent Data Encryption
Pricing and Licensing SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Two Ways to License
• When you purchase Microsoft SQL Server® 2008, you purchase: – A license to run SQL Server on the server and one of the following: • CPU license • Client Access License (“CAL”) for a user • Client Access License (“CAL”) for a device
CAL Licensing
• CAL Licensing Model Explained – User model: Requires single CAL “… for each user who accesses a system running SQL Server.” • One user can use the same CAL to access different SQL Servers separately • Source: “Database Pricing” whitepaper, MSFT 2008
CAL Licensing
• CAL Licensing Model Explained – Device model: Requires single CAL “… for each device that accesses a system running SQL Server.” • Devices can be PC, workstations, terminal, PDA, mobile phone, etc
CAL Licensing
• User model or Device model? – First question: Should you use CPU licensing instead? – If you cannot/should not use CPU licensing: • Use device model when there are multiple users per device • Use user model when there are multiple devices per user • When in doubt, consider user model
CAL Licensing
• Example scenario: Your CRM application has 50 users worldwide but only 10 use at the same time all using their individual workstations – Need a minimum of 10 user CALs
CRM: Customer Relationship Management
CPU Licensing
• CPU Licensing Model Explained – Unlimited number of connections, users for a single CPU – License only needed for physical CPU • A server with one quad-core CPU would require only a single CPU license
How MSFT Determines Usage
• Consider this scenario: – DBA installs SQL Server Enterprise Edition on web server – One web application makes a single connection to SQL Server – Web application is used by 500 people worldwide
• How many licenses do you need?
How MSFT Determines Usage
• Answer: “Not enough information to answer!” – Many people think the answer is “One CAL” since there is only one app accessing the database server! – You need to ask a follow-up question • What is the follow-up question you should ask?
How MSFT Determines Usage
• “How many of those 500 users will be using the system at the same time (concurrently)?” – Answer: 10 • Then you need 10 user CALs
– Answer: 100 • Then you need 100 CALs or 1 CPU license
CPU or CAL?
• At some point it makes sense to switch from CAL to CPU – 10 CALs or 1 CPU? – Depends on cost of each – Likely to be somewhere between 60 and 90 concurrent users that you save money by using CPU – Don’t forget to add in growth
In the Next Video
• Licensing Model • Virtualization
• Cost per edition
Pricing and Licensing SQL Server 2008
Presented by Scott Whigham
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Virtualization
• What about licensing when you install SQL Server inside VMware or another virtualized environment? – Same basic rules – For Enterprise Edition, if all processors are licensed, customers can run “unlimited instances of SQL Server on an unlimited number of virtual operating environments on that same machine” • Source: “Database Pricing” whitepaper, MSFT 2008
What We’re Going to Cover
• Licensing Model • Virtualization
• Cost per edition
Costs (Retail only)
CPU License
CAL
Enterprise
Standard
Workgroup
Web
$24,999
$5,999
$3,899
$15/CPU/month
$8,487 for server license plus $162 per CAL
$885 for server license plus $162 per CAL
$730 for server license plus $162 per CAL
Not available
Developer Edition: $50 total cost
Prices are in U.S. dollars Source: http://www.microsoft.com/sqlserver/2008/en/us/pricing.aspx, Nov, 2008
What’s New in SQL Server 2008
Presented by Scott Whigham
What to Expect
• This is an overview, not a detailed discussion • Expect to see “Here’s what it is and here’s 1 or 2 sentences about it” • Many if not all will be covered later in detail • Not all new features are covered
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
What We’re Going to Cover
• SSIS • SSRS • SSAS
SQL 2008 Is About…
• SQL 2008 is likely to be interesting to most people because of it’s ability to scale rather than because of new features – While there are lots and lots of new features, only a few of them are major new features
SQL 2008 Is About…
• Many of the new features target “5%ers” – Geospatial support is one such example
• Some of the new features are simply “better ways” – Geospatial support and FILESTREAM are examples
SQL 2008 Is About…
• If you were creating a first-time app for storing/working with geospatial data, SQL Server 2008 would be ideal – What if your app already uses a technique for storing/working with geospatial data that was different from the way SQL Server 2008’s internal datatypes and use?
SQL 2008 Is About…
• A SQL Server 2005 DBA can administer a SQL 2008 box without learning anything new – Obviously wouldn’t be aware of new features and enhancements but it is very possible (and likely) – SQL Server Management Studio (“SSMS”) has been augmented but not drastically changed
SQL 2008 Is About…
• SQL Server 2008 new features further segment “The Talent” – It’s not possible for one person to know it all • This was possible as recently as SQL Server 7.0 or even SQL Server 2000
SQL Server 2005 Changed Everything
• In SQL Server 2005, The Business Intelligence side of SQL Server matured thus adding (needed) complexity to existing products – SQL Server – Integration Services (“SSIS”) – Reporting Services (“SSRS”) – Analysis Services (“SSAS”)
Changing Roles
• Today, it’s easier to be a “Specialist” in one area and a “Generalist” or even a “Deep Generalist” in others – You might specialize in SQL Server DBA but have deep, general knowledge of SSIS, a general knowledge of SSRS and scant knowledge of SSAS
SQL Server 2008
• In SQL Server 2008, the DBA must work harder than ever to be a Specialist – Fairly easy to be a generalist – Tough to be a deep generalist – Very hard to be a specialist • What size databases do you specialize in? • At your last job, did you implement encryption? • Did you do database mirroring and use snapshots at your last job? • How many active/active clusters have you built? • What’s the best way to do document/file storage?
Why is this discussion here?
• Why have this discussion now? – Don’t get intimidated into thinking you have to know it all – Don’t be afraid to say, “That’s not my specialty” – Pick the areas you like and will need to continue employment and master those
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Performance Improvements
• Compression Enhancements: – Backup compression • Are your backups taking up too much space?
– Log Stream compression • Are you using database mirroring? If so, log stream compression can offer significantly better performance
– Data compression • Do you have a lot of data that is similar or bloated? • Easy to use: BACKUP DATABASE … WITH COMPRESSION
Performance Improvements
• Enhancements and improvements for databases of all sizes – Enhanced/Improved Plan Guide Support • Freezing plan guides, pull plans directly from plan cache, SSMS integration
Performance Improvements
• Enhancements and improvements for databases of all sizes – Resource Governor • Allows you to manage workload and resources by setting limits to incoming requests • Can limit certain users/groups/roles to fewer resources
Performance Improvements
• Enhancements and improvements for databases of all sizes – Performance Data Collection • Includes a central repository for storing performance data as well as tools and reports
Performance Improvements
• Enhancements and improvements for larger databases: – Partition-aligned indexed views • Are you using indexed views or partitions already? If so, this is great. If not, probably unnecessary for you at the moment
– Partitioned Table Parallelism • In SQL Server 2005, each partition was limited to a single thread
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Security Improvements
• Lots of new features that are potentially “better ways” – Transparent Data Encryption • Extends SQL Server 2005 encryption • Easier to turn on/off encryption for the entire database without having to modify calling applications
Security Improvements
• Lots of new features that are potentially “better ways” – Extensible Key Management (EKM) • Allows the keys that protect the database files to be stored externally
Security Improvements
• Some are just changes… – New database roles • MSDB database: – db_dts… roles have been renamed to be db_ssis… – Server group roles: ServerGroupAdministratorRole, ServerGroupReaderRole – Policy-Based Management role: PolicyAdministratorRole – Data Collector: dc_admin, dc_operator, dc_proxy
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Management Improvements
• New Feature: – Policy-Based Management • DBAs can set policies that cover everything from naming conventions to CLT integration • One of the most exciting new features for DBAs
Management Improvements
• New Feature: – SQL Server Audit • Allows customizing what you want audited • Allows for auditing at server and database level – – – –
SELECT, INSERT, UPDATE, DELETE, EXEC DBCC Login/logout attempts State changes
• Scripts can be generated from audits
Management Improvements
• New Feature: – Change Data Capture (CDC) • Allows you to record changes to data into another table without triggers – INSERT, UPDATE and DELETE
• Perfect for applications that update external databases • Example: you need to synchronize two 200GB databases nightly in a one-hour window
Management Improvements
• New Feature: – Data Profiling Task • New SSIS Task allows DBAs to profile for things like: – – – –
Number of rows Number of unique values Percentage of NULL values in column Duplicate values in potentially key columns
Management Improvements
• New Feature: – Extended Events (Xevents) • Capture, filter, and act upon events generated both by SQL Server, the operating system, and other applications • Can trace events at any level • Can choose aggregated data or event details
Management Improvements
• Enhancements – Central Management Servers & Server Groups • DBAs can now designate one SQL Server a Central Management Server and create a Server Group – – – –
Queries can be executed on the servers in the server group Policies can be evaluated against the group Changes made in Object Explorer can be propagated Windows authentication required
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Availability Enhancements
• Hot pluggable CPUs • Automatic recovery of suspect pages • Add nodes to peer-to-peer replication without stopping system activity • Enhanced database mirroring
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
T-SQL Improvements
• Intellisense is finally here! • Table-valued parameters are here! – Goofy syntax and requirements but you can now pass table results to stored procedures • Not to user-defined functions though
T-SQL Improvements
• Compound operators are finally here! SET @i += 1
• Declaration and assignment in one statement is finally here! DECLARE @i INT = 1
T-SQL Improvements
• XML enhancements – Better and easier schema validation – Better XQuery support • Use of “let” clause to assign values
– Full xs:dateTime support
T-SQL Improvements
• FILESTREAM – Allows DBAs and Developers to store documents as part of the operating system yet to be able to access those as part of the database • COMMIT/ROLLBACK aware • Uses new VARBINARY(MAX) FILESTREAM storage type • Users can use regular SELECT, INSERT, UPDATE, and DELETE statements to query and modify FILESTREAM data
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Datatypes and Geospatial
• New datatypes for working with date/time data Datatype
Accuracy
Used when…
DATETIME
3/1000th of a second
Need to store milliseconds
SMALLDATETIME
Minute
Need to store minutes
DATE
Day
Need to store only date
TIME
100 nanoseconds
Need to store only time
DATETIME2
100 nanoseconds
Need more accuracy than 3/1000th of a second
DATETIMEOFFSET
100 nanoseconds
Same as DATETIME2 with TimeZoneOffset capability
New datatypes
• HierarchyId – Used to represent the position in a hierarchy – Has it’s own functions to work with hierarchical data • GetAncestor() • IsDescendant()
Geospatial Datatypes
• SQL Server 2008 adds native support for storing and working with geospatial data • New datatypes: Datatype
Used when…
More…
GEOMETRY
Represents a flat 2D surface with X and Y coordinates for points
GEOGRAPHY
Same as GEOMETRY but for Earth
The GEOMETRY data type doesn’t really work as expected when you pass in latitude and longitude pairs. For GEOMETRY, the distance between the POINTS (90 0) and (90 180) is 180 but in GEOGRAPHY, since both points refer to the same location (the North Pole), the distance is 0.
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
The Debugger is finally in SSMS
• In SQL Server 2005, the T-SQL Debugger was not in the SQL Programmer’s native environment – SQL Server 2000 included it in the Query Analyzer – SQL Server 2005 forced us to Visual Studio
• SQL Server 2008 features the T-SQL Debugger in SSMS
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In the next video
• SSIS • SSRS • SSAS
What’s New in SQL Server 2008
Presented by Scott Whigham
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
SSIS Enhancements
• New SQL Server Integration Services (SSIS) features and enhancements include: – C# is finally here! – Scripting is VSTA (Visual Studio Tools for Applications) instead of old SQL Server 2005 way: VSA (Visual Studio for Applications) • Easier to use custom assemblies
– Performance improvements such as better threading and pipeline scalability
SSIS Enhancements
• More (SSIS) features and enhancements: – Lookups are way better • SQL 2005 had many restrictions and inefficiencies – Limited to ~3GB of memory – OLEDB only – Binary matching mean case+accent sensitivity (with workarounds)
• SQL 2008 makes life easier – Persistent Lookups store a cached copy locally – OLEDB, ADO.NET, and flat file sources – More memory
SSIS Enhancements
• New and Improved – Data Profiling Task (already covered) – Change Data Capture makes ETL (Extract, Transform, Load) easier – New SSIS datatypes for working with new SQL date/time datatypes
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Reporting Services Improvements
• IIS no longer required to be on same machine to host the Reporting Services web service – Http.sys now handles HTTP requests just like IIS
• Better caching and better memory management
Reporting Services Improvements
• Tablix data regions for free-form reporting • Reports now can be rendered in Microsoft Word® • Able to respond to “3-4 times the number of users and their requires on the same hardware” compared to SQL Server 2005 Reporting Services1 1 – Source: Carl Rabeler, Program Manager on the SQL Server Customer Advisory Team
Reporting Services Improvements
• Prettier, nicer, better – Microsoft purchased Dundas Data Visualization technology to use inside SSRS and we now have • Gauges • Better calendar controls • Easier to manage charts
– Report Builder 2.0 – Report Designer
• T-SQL Improvements • Datatypes • Geospatial • Debugging
Business Intelligence
• Overview of new features and how it relates to DBAs • Performance • Security • Management • Availability
Developers
DBAs
In this video
• SSIS • SSRS • SSAS
Analysis Services
• Star JOINs • MERGE statement • Various enhancements
Deprecated and Discontinued in SQL Server 2008
Presented by Scott Whigham
What this video covers
• This is an overview, not a complete list – Check Books Online for complete list
Deprecated
• “Deprecated” items are slated for removal in a “future version, hot fix, or service release” • “Discontinued” items are “things that don’t work that way anymore” – Usually Discontinued items were once Deprecated
Discontinued Items that may affect you
• Some Discontinued items that may affect you are: Discontinued item
Replacement
BACKUP LOG WITH NO_LOG
None. Switch to SIMPLE recovery model instead
BACKUP LOG WITH TRUNCATE_ONLY
None. Switch to SIMPLE recovery model instead
60, 65, and 70 compatibility models
Databases must be at least 80 (SQL Server 2000)
Northwind and Pubs
Gone – use AdventureWorks instead
Surface Area Configuration Tool
Lasted only one version – SQL Server 2005
Deprecated Items that may affect you
• Some Deprecated items that may affect you are: Deprecated item
Replacement
BACKUP WITH PASSWORD =
None
DATABASEPROPERTY
DATABASEPROPERTYEX
sp_dboption
ALTER DATABASE
FASTFIRSTROW hint
OPTION(FAST n)
sp_dbcmptlevel
ALTER DATABASE… SET COMPATIBILITY LEVEL
BACKUP … TO TAPE
Backup to disk only
SET ANSI_NULLS OFF and ANSI_NULLS OFF
None – they will always be one
Deprecated Items that may affect you Deprecated item
Replacement
SQLOLEDB is not a provider
Use SQL Native Client
TIMESTAMP syntax for ROWVERSION datatype
Use ROWVERSION datatype
text, ntext, and image datatypes
Use VARCHAR(MAX), NVARCHAR(MAX) and VARBINARY(MAX)
sp_attach_db
CREATE DATABASE … FOR ATTACH
sp_change_users_login
ALTER USER
DBCC DBREINDEX
REBUILD option of ALTER INDEX
DBCC INDEXDEFRAG
REORGANIZE option of ALTER INDEX
DBCC SHOWCONTIG
sys.dm_db_index_physical_stats
Deprecated Items that may affect you Deprecated item
Replacement
sp_configure ‘allow updates’
No effect
sp_configure option ‘priority boost’
No effect
sp_addapprole
CREATE APPLICATION ROLE
sp_addlogin
CREATE LOGIN
sp_adduser
CREATE USER
sp_grantdbaccess
CREATE USER
sp_addrole
CREATE ROLE
sp_changeobjectowner
ALTER SCHEMA or ALTER AUTHORIZATION
sp_defaultdb
ALTER LOGIN
Deprecated Items that may affect you Deprecated item
Replacement
sp_grantlogin
CREATE LOGIN
sp_denylogin
ALTER LOGIN DISABLE
sp_revokelogin
DROP LOGIN
GRANT ALL DENY ALL REVOKE ALL
Must specify permissions independently
Lots of system tables
Compatibility views and Catalog views
WRITETEXT UPDATETEXT READTEXT TEXTPTR()
None.
Deprecated Items that may affect you Deprecated item
Replacement
3 and 4-part column names in queries
Use 1 or 2-part names only such as SELECT t.ProductName
GROUP BY ALL
Use custom case-by-case solution with UNION, CTE, or derived table
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
This Chapter
• This chapter is all about installing and upgrading to SQL Server 2008 • Included in this chapter are: – Installing SQL Server 2008 for the first time – Upgrading an entire SQL Server 2000 and 2005 to SQL Server 2008
This Chapter
• Related topics not covered in this chapter but covered later: – Upgrading a single database to SQL Server 2008
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
First Time Install of SQL Server 2008
• There are several items you need to know before you install SQL Server 2008 for the first time 1. 2. 3. 4. 5. 6. 7.
What type of server are you installing? Hardware, software requirements 32-bit or 64-bit? Which machine(s) need to run the install program? What security model will you use? Where will you put the files? What default server collation will you use?
• We’ll look at each one individually
Things You Need to Know Before Installing SQL Server 2008
1. What type of server are you installing? • •
The type of server determines the hardware What is the database profile of your server?
• 1 small database (DB) • Multiple small DBs • 1 medium DB • 1 medium DB and multiple small DBs • Multiple medium DBs • 1 Large DB • 1 Large DB and multiple small DBs
• 1 Large DB and 1 medium DB • 1 Large DB and multiple medium DBs • 1 Very Large DB (VLDB) • 1 VLDB and multiple medium DBs • 1 VLDB and 1 Large DB • 1 VLDB and multiple large DBs
Things You Need to Know Before Installing SQL Server 2008
• What other functions will this server perform? • SQL Server and Reporting Services on same machine • SQL Server and Analysis Services on same machine • Standalone Reporting Services server • Reporting Services web farm
• Standalone Analysis Services • Standalone SQL Server • File server • Domain controller • Mail server • SharePoint server
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
2. Hardware, software requirements –
–
The more you know about the type of server you are installing, the more you can accurately predict server needs Disk space is cheap but make sure you plan for growth
Things You Need to Know Before Installing SQL Server 2008
2. Hardware, software requirements – – –
Small and medium size databases need RAID 1 (OS, logs) and RAID 5 (data files) sets Large and VLDBs need RAID 10 SANs (Storage Area Networks) Don’t forget about backups!
Things You Need to Know Before Installing SQL Server 2008
3. 32-bit or 64-bit? –
Did you know? Windows Server 2008 will be the last 32-bit server from Microsoft The advantage of 64-bit architectures are that you can address more physical memory In a 32-bit system with 16GB of physical RAM, you can only directly address 4GB
–
– •
You can still use the rest but it uses Address Windowing Extensions (AWE) to “map” to the memory whereas 64-bit can address the memory directly
Things You Need to Know Before Installing SQL Server 2008
3. 32-bit or 64-bit? – –
64-bit supports larger numbers of processors and more linear scalability per processor This may mean lower Total Cost of Ownership (TCO) since you can “do more with less hardware”
Things You Need to Know Before Installing SQL Server 2008
4. Which machine(s) need to run the install program? – – –
One install on the server One install on each workstation that needs the management/development tools Possible need to install SQL Server Native Client on clients
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Two choices: • •
Windows Authentication only Combination of Windows Authentication and SQL Server Authentication (“Mixed Mode”)
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Windows Authentication only •
Can grant or deny Windows user or Windows group access to SQL Server
Things You Need to Know Before Installing SQL Server 2008
5.
Windows Authentication Example: – – –
•
Chad is the user “chadw” chadw belongs to two Windows groups: Users and SQLDevelopers The DBA adds “SQLDevelopers” group as a login to the SQL Server
Question: Can Chad now login to the SQL Server? –
•
Yes
Question: Who else can log in? –
Anyone else who is a member of the SQLDevelopers group
Things You Need to Know Before Installing SQL Server 2008
5.
Windows Authentication –
PROs: •
No passwords are stored in SQL Server or sent across the network Centralized management of users and groups (in Active Directory)
•
–
CONs •
The SQL Server DBA is no longer in control of security –
• •
Windows admins can now add/remove users from Windows groups thus elevating/denying privileges in SQL Server
Non-Windows machines have trouble logging in Anyone without a Windows account cannot use SQL Server
Things You Need to Know Before Installing SQL Server 2008
5. What security model will you use? –
Mixed Mode •
Can grant or deny Windows user or Windows group access to SQL Server DBA can also create SQL Server logins
• –
Have a login name and a password
Things You Need to Know Before Installing SQL Server 2008
5. Mixed Mode Example – – –
Scott’s iPhone runs an application that uses SQL Server Scott is a consultant who not have a Windows account The DBA adds a SQL Server login “scottw” with a password
Things You Need to Know Before Installing SQL Server 2008
5. Mixed Mode –
PROs: • •
–
Best of Windows Authentication mode SQL Server can control security more granularly
CONs •
Passwords are passed across network –
•
Can be encrypted
Microsoft says this is less secure
Things You Need to Know Before Installing SQL Server 2008
5.
Account Provisioning –
Depending on what features you install, you must determine how they log in The following features are require a service account:
– • • • • • • •
SQL Server Database Engine SQL Server Agent SQL Server Browser SQL Server Analysis Services * SQL Server Reporting Services * SQL Server Full-text Daemon Launcher * SQL Server Integration Services *
* Only if you install this feature
Things You Need to Know Before Installing SQL Server 2008
• • •
Always run SQL Server services using the lowest possible user privilege No need to run any SQL Server service as Administrator Use separate accounts for all SQL Server services (no account sharing)
Things You Need to Know Before Installing SQL Server 2008
5. SQL Server Database Engine –
Usually a domain account • •
–
Do not use NETWORK SERVICE account LOCAL SYSTEM is a bad idea since it has elevated privileges
May need to assign additional permissions to the user account to allow SQL Server to access directories for backups or import/export
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Things You Need to Know Before Installing SQL Server 2008
6. Where will you put the files? –
During the installation you are asked: • • •
Where do you want the “Program Files” to go? Where do you want the database and log files to go? Where do you want your backups to go?
Things You Need to Know Before Installing SQL Server 2008
6. Default locations: Item
Path
Can Be Changed?
SQL Server Program Files
\Program Files\Microsoft SQL Server\MSSQL10.
Yes
Reporting Services Program Files
\Program Files\Microsoft SQL Server\MSRSL10.
Yes
Analysis Services Program Files
\Program Files\Microsoft SQL Server\MSASL10.
Yes
SQL Server Data Files
Yes
SQL Server Backups
Yes
Each installation is assigned an INSTANCE_ID. How this is named is covered later
Things You Need to Know Before Installing SQL Server 2008
6. Where should you put the files? – –
–
Program files should likely be in default location Data files and log files should be on RAID 5 and RAID 1 respectively for small/medium installations Backups should be on RAID 1 for small/medium installations
Things You Need to Know Before Installing SQL Server 2008
• Customize your installation:
Things You Need to Know Before Installing SQL Server 2008
7. What server collation should you use? – –
During setup you must decide the default server collation Collation is for working with character data
•Things You Need to Know Before Installing SQL Server 2008 SELECT * FROM Customer WHERE Lastname=‘Pérez’
–
Which of the following results would you want returned? • • • •
Perez PEREZ perez Pérez
• • • •
PÉREZ pérez Péréz PÉRÉZ
• • • •
péréz Peréez PERÉZ peréz
Things You Need to Know Before Installing SQL Server 2008
7. Collation defines: –
Sort order •
–
Does “A” sort before “a”?
Whether accents, binary, and case-sensitivity matter •
–
Are “Pérez” and “perez” the same?
The characters available in a column •
More on this later...
Things You Need to Know Before Installing SQL Server 2008
7. There are two types of Collations –
Windows Collation • •
What is the collation of your Windows machine? Used when: –
You know that all of your Windows machines have the “correct” collation settings
Things You Need to Know Before Installing SQL Server 2008
7. There are two types of Collations –
SQL Server Collation •
Who cares what the collation of my Windows machine is! Used when:
• – –
You want to install multiple instances with different collations Your Windows machines have different collations but you want all SQL Servers to have the same collation
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
In the next video…
• What to Do Before • Upgrade • What to Do After
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Instances
• Instances are “installations of SQL Server” • There are two types: – Default – the name of the computer is the name of the InstanceId – Named – you must provide a unique InstanceId
Instances
• Most sites usually have a single instance – Whether it is a default or named instance is hard to assume
• Instances compete for: – CPU – Memory – Disk space
Instances
• Single instances are appropriate for: – Situations where having a single server for security and management is best – Serving multiple databases to a single user – All databases should be on the same version/edition of SQL Server
Instances
• Multiple instances are appropriate for: – Situations where having a single server for all databases would be a security breach – Databases need be on the different versions/editions of SQL Server – Same database needs to exist on different versions/editions of SQL Server (for testing)
Instances
• Multiple instances can be: – Mixed versions • SQL Server 2000, 2005, 2008
– Mixed Editions • Enterprise, Developer, Standard, Express
– Mixed Releases • SQL Server 2008 RTW, SQL Server 2008 SP1
Instances
• There can be only one default instance • Instance names must be: – Unique – First character must be a letter – Not contain underscores or # – 16 characters or less – Cannot use the word “DEFAULT” anywhere – No funny business
Instances
• Instance Miscellany: – Enterprise Edition: up to 50 instances – Other editions: up to 16 instances
Instances
• Licensing – All editions now allow CPU license to cover “unlimited instances per operating environment”
Things to Know After You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Service Packs and Hotfixes
• Like all software, your SQL Server needs to be updated • You need updates for any and all features you’ve installed – SQL Server – Analysis Services – Reporting Services – And all the rest that don’t fit on this page
Service Packs and Hotfixes
• Primarily, SQL Server is updated using the Incremental Service Model (ISM) – The ISM is meant to be a standardized approach to keeping SQL Server up to date – All ISM releases can be requested by any customer regardless of their support offering
Service Packs and Hotfixes
• ISM is a way to distribute “hotfixes” – Critical on-demand (COD) • You must be vetted by Microsoft to qualify
– On-demand (OD) • Same exact requirements as COD…
Service Packs and Hotfixes
• ISM model continued – Cumulative Update (CU) • Update released every two months • Incremental numbers (CU1, CU2, CU3) • Contains all previous COD and OD hotfixes rolled into one update • May contain other fixes as well • All previous CUs since last service pack
Service Packs and Hotfixes
• ISM model continued – General Distribution Release (GDR) • “A GDR addresses an issue that has a broad customer impact, that has security implications, or that has both” – “An Incrementing Service Model…” – Microsoft Product Support, 2008
• Cannot be requested – Released when MSFT decides
Service Packs and Hotfixes
• Service Packs – Usually rollup every COD, OD, CU and GDR since the last service pack – Service packs are cumulative • SP3 includes all of SP1 and SP2
– Often include new features
Service Packs and Hotfixes
• ISM model:
Service Packs and Hotfixes
• We use “Product Level” to determine the service pack level – RTM = “release to manufacturer” • The initial release
– SP1 = “Service pack 1”
Service Packs and Hotfixes
• We also use “Build Numbers” to track what has been installed – Tell us which hotfixes, COD, OD, service packs, et al have been installed – Most hotfixes increment the build number
Service Packs and Hotfixes
• Build Numbers – To determine the service pack level, you can check the build number – To determine hotfix, COD, OD, et al, sometimes you need to look at the file version
• Build numbers are built around initial release and service packs
Service Packs and Hotfixes
• Sometimes you need the file version and the build number Version
Build Number File
File Version
SQL 2005 SP1
9.00.2047.00
9.00.2047.00
SQL 2005 SP2
9.00.3042.00
Microsoft.SqlServer.Mainten ancePlanTasks.dll
9.00.3042.00
SQL 2005 SP2 refresh
9.00.3042.00
Microsoft.SqlServer.Mainten ancePlanTasks.dll
9.00.3043.00
SQL 2008 RTM
10.0.1600.22
SQL 2008 CU1
10.0.1763
Service Packs and Hotfixes
• To determine your SQL Server build number: SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')
Service Packs and Hotfixes
• “How do I get the hotfix/service pack/etc?” – Some are publicly available (from microsoft.com) – Others require you filling out an “I agree that, if this breaks my system, it is my own fault” type of form – Others require contacting Microsoft and convincing them that you are eligible for the hotfix
Service Packs and Hotfixes
• Remember: A CU will be released every two months – Well, not exactly. But close enough – Put a recurring task to search for “SQL Server 2008 cumulative update”
Final Thoughts
• Be wary – COD are least tested – Service Packs are most tested – Install what you need – Use a test box
Final Thoughts
• Be prepared – Often, removing a hotfix/service pack/etc requires an uninstall/reinstall – Be sure you back everything up before installing – Be prepared to support multiple versions/builds
Things to Know Before You Install SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
What Is Installed • Management Tools – – – – –
SQL Server Management Studio (SSMS) SQL Server Configuration Manager sqlcmd Import and Export Data PowerShell
• Development Tools – SSMS – Business Intelligence Development Studio – Visual Studio 2008
• Performance Tools – SQL Profiler – Database Engine Tuning Advisor
• Documentation and Tutorials – Books Online (BOL)
What Is Installed
• SQL Server Management Studio (SSMS) – The tool you’ll likely spend the most time with since it does both DBA and development • Backups, security, jobs • Stored procedures, queries, creating/modifying objects
– Deserves its own video!
What Is Installed
• SQL Server Configuration Manager – Allows DBAs to determine network configuration and aliases of servers – Used sparingly – often lots right after install and then 1x per month thereafter
What Is Installed
• sqlcmd.exe – Command line query and script execution engine – Can do most of what SSMS can do – Powerful scripting capabilities
What Is Installed
• Import and Export Data – A wizard for importing and exporting….. DATA! – Uses SQL Server Integration Services engine – What type of data? • SQL Server, Oracle, DB2, Sybase • Excel, Access, text/flat files • Any OLEDB or ODBC
Upgrading to SQL Server 2008
Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
Upgrading
• First thing: decide what needs to be upgraded – Can upgrade an entire instance* – Can upgrade just one user database – Can upgrade all user databases
Upgrading
• This chapter only looks at upgrading an entire instance – Other chapters look at upgrading individual DBs
What Can You Upgrade?
• Upgrades only support 32-bit to 32-bit or 64-bit to 64-bit – No 32-bit to 64-bit native upgrade path – If needed, you will need to move individual DBs
What Can You Upgrade?
• What does the term “upgrade” mean? – There are two different ways to view: • Upgrading from a prior version (2000, 2005 only) • Upgrading from a lower edition of SQL 2008
What Can You Upgrade?
• Upgrading from a lower edition of SQL Server 2008 is usually: – Upgrading from Express to a “paid” version – Upgrading from Workgroup to Standard – Upgrading from Standard to Enterprise
What Can You Upgrade?
• Requires least amount of planning and prep-work
What Can You Upgrade?
• Upgrading from a different version of SQL Server requires significant planning – Are you upgrading to same edition? – Are you upgrading 32-bit to 64-bit? – Will file paths stay the same or change? – Will CPU or memory requirements change? – Is your SQL going to cause problems?
What Can You Upgrade?
• Upgrading from SQL Server 2000 to SQL Server 2008 is a challenge – So many architecture changes
• Upgrading from SQL Server 2005 is relatively simple
Things That Will Block Your Upgrade
• Basic stuff generally – New server doesn’t meet hardware/software requirements – Will be blocked if there is a pending restart – Windows Installer service must be running
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
The SQL Server Upgrade Advisor
• Regardless of type of upgrade, install and run the SQL Server Upgrade Advisor – Ships with SQL Server – Reports on incompatibilities and problems
• Before upgrading – Correct any errors reported – Note and act upon the warnings – Note the informational messages
Upgrading to SQL Server 2008
Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
The SQL Server Upgrade Advisor
• Regardless of type of upgrade, install and run the SQL Server Upgrade Advisor – Ships with SQL Server – Reports on incompatibilities and problems
• Before upgrading – Correct any errors reported – Note and act upon the warnings – Note the informational messages
Upgrading SQL Server 2000 DTS Packages Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
SQL Server 2000 DTS
• SQL Server Integration Services (“SSIS”) is the replacement for SQL Server Data Transformation Services (DTS) • SSIS is radically different from DTS and requires special consideration when upgrading
SQL Server 2000 DTS
• In an in-place upgrade, DTS packages will remain DTS packages in SQL Server 2008 – The upgrade will not “upgrade” your DTS packages – You can continue to edit the DTS packages if you install the SQL Server 2000 DTS Designer – You can even use the Microsoft SQL Server Backward Compatibility feature to run DTS packages as-is • 32-bit only!
SQL Server 2000 DTS
• To migrate DTS packages, run the Package Migration Wizard (“PMW”) – Right click on the package and select “Package Migration Wizard” – Can upgrade one package at a time or all packages – Complete the steps to upgrade
SQL Server 2000 DTS
• After migrating to SQL 2008, you might be surprised • SSIS is similar to DTS just like Microsoft is similar to Apple SSIS / DTS
Microsoft / Apple
Both are ETL tools
Both write operating systems
Both installed with SQL Server
Both sell MP3 players
Different ways of doing the same things
Different ways of doing the same things
SQL Server 2000 DTS
• Be prepared to do a lot of rewriting and re-architecting – Refactoring may take more time up-front but will likely save hours and hours in the long run – The ActiveX Script Task has been deprecated • Often the functionality is replaced by a built-in SSIS task
– Complex data transfers will often need to be rewritten
After You’ve Upgraded to SQL Server 2008 Presented by Scott Whigham
• Installing Developer Edition • Working with Instances • Things to Know After You Install
Upgrading to SQL 2008
• Overview of Chapter • Things to Know Before You Install
Installing for the 1st Time
Overview
What We’re Going to Cover
• What to Do Before • Upgrade • What to Do After
After You’ve Upgraded
• Life is good, right? • Check to make sure everything came over as expected – It probably did
• It’s a good time to change service account passwords – If you don’t already have a policy for doing so
After You’ve Upgraded
• Things you should test – “The Obvious” – Do your DTS packages still work? – Do your jobs still run? – Your pre-upgrade Profiler trace queries – do they still return same results? – Does your application still work? – Do the things that the Upgrade Advisor warned you about fail?
After You’ve Upgraded
• Things you should test – “The Not-SoObvious” – Do your datetime queries/applications still run as expected? • Failures using the Enterprise Library 4.x with .NET 3.5 and SQL 2008 for datetime and smalldatetime
After You’ve Upgraded
• Next steps – Understand the configuration parameters – Check your database compatibility – Review “Suggested Changes After Upgrade” video
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
This Chapter
• What this chapter covers – What has been installed – An overview of each of the tools • Answering The Three W’s: – What it is, What it does, Who Uses It
– An overview of configuring the server
This Chapter
• What this chapter does NOT cover – An overview of configuring databases – In-depth coverage of all tools – Every single server option available – Cover every tool available
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
An Overview of What Is Installed
• There are tools available for: – Administrators • • • •
SQL Server Database Administrators (DBAs) Reporting Services Administrators Analysis Services Administrators Systems Administrators
– Developers • • • •
SQL Programmers Report developers Integration Services developers Analysis Services developers
An Overview of What Is Installed
• There are tools available for: – Command line only – GUI (“Graphical User Interface”) – APIs (“Application Programming Interface”)
• Let’s look at each
Command line tools
• Command line tools primarily target DBAs Tool
Who uses?
Description
sqlcmd.exe
DBAs, Developers Command line queries/batches
dtexec.exe
DBAs
Execute SSIS packages
sqldiag.exe
DBAs
Diagnostics, system info, monitor, troubleshoot
bcp.exe
DBAs
Import/export text files
profiler.exe
DBAs
Trace server activity
GUI tools
• GUI tools are for everyone! Tool
Who uses?
Description
SQL Server Management Studio (SSMS)
DBAs, Developers Queries, management, security
Business Intelligence Development Studio (BIDS)
Developers
Develop/edit reports, SSIS packages, SSAS
Visual Studio 2008
Developers
It is the “real” name of BIDS
SQL Server Configuration Manager
DBAs
Manage services, aliases, and network configuration
GUI tools
• More GUI tools Tool
Who uses?
Description
SQL Server Profiler
DBAs
Trace server activity
Database Engine Tuning Advisor
DBAs
Performance tuning, optimizations
Data Profile Viewer
DBAs
View stats on column/table usage
Command line tools
• APIs are for developers and integrators API
Who uses?
Description
SQL Server Management Objects (SMO)
Developers
Manage SQL Server via programming interfaces
SQL Server Replication Developers Management Objects (RMO)
Manage replication via programming interfaces
Data Collector
Developers
Capture, store and report on performance/health stats
Native Client
Developers
Combines both ODBC and OLE-DB into one DLL
WMI
Developers
Use WMI for server events and configuration
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• You can configure SQL Server at the instance and database level • This chapter covers the instance level • Next chapter covers the database level
Configuration
• Instance-level configurations affect all connections and all databases for that instance • You can change settings using – – – –
SQL Server Management Studio SQL Server Configuration Manager sqlcmd.exe SQL Server Management Objects (SMO) API
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Service Accounts
• Service accounts are not static – Change passwords – Change accounts
• Changing anything here can be an exciting experience • To change, use the SQL Server Configuration Manager
Configuration: Service Accounts
• To modify the service accounts, use the SQL Server Configuration Manager
Configuration: Service Accounts
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Network Protocols
• Network protocols define what connections SQL Server will accept
Configuration: Network Protocols
• If SQL Server has TCP/IP enabled but all others disabled, only clients using TCP/IP for connections would be allowed – Both the server and client must use the same protocol
Configuration: Network Protocols
• Shared Memory – Connections and data are not transferred across IP stack; done in memory – Clearly this only works when the client and server are on the same machine – For remote connections, must use one of the other network protocols
Configuration: Network Protocols
• How to Enable Remote Connections – Default for Express and Developer editions is to turn off remote connections • This is done by disabling all protocols except Shared Memory
• To enable remote connections: 1. Enable TCP/IP or one of the other network protocols 2. Restart SQL Server instance
Configuration: Network Protocols
• TCP/IP is most common – Requires a unique port per instance – Default port for default instance is 1433 – Named instances use a random-first-time port
Configuration: Network Protocols
• Change/view settings in the Configuration Manager – Any changes here require restarting instance
Configuration: Service Accounts
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration: Memory, CPU
• Memory and CPU usage can change – When new hardware is added – When the application’s usage changes
• Some changes require restarting the SQL Server service but most are dynamic • Let’s look at memory in this video
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Memory, CPU
• Memory is used the most in two areas – Buffer cache • Stores the data users are working with in memory to avoid reading disk on commonly-requested data
– Procedure cache • SQL Server caches the “how” of running queries and routines to avoid having to figure out the fastest route to the data on subsequent calls – The “how” is the “Execution Plan”
Configuration: Memory, CPU
• Sizing the Buffer Cache – Too small means that users have to wait for SQL Server to access hard disk – Too large means that the operating system and/or other applications do not have enough memory
Configuration: Memory, CPU
• Buffer cache as a percentage of the data file size – The larger the database, the smaller the percentage of the size • A 10TB system with 10 concurrent users likely needs a 1-2% buffer cache • A 10GB system with 100 concurrent users may need as much as a 25% buffer cache
Configuration: Memory, CPU
• The Buffer Cache should be expected to be between 5 and 25% of the data file size for an online transaction processing (OLTP) system – Should be a bit larger percentage for reporting
Configuration: Memory, CPU
• Sizing the Procedure Cache – On a 32-bit system, the procedure cache is limited to 1GB • Cannot use Address Windowing Extensions (AWE)
– On a 64-bit system, procedure cache can be larger • This is necessary on larger systems
Configuration: Memory, CPU
• Memory options you can change – Max server memory • Always, always set this to avoid a too-large buffer cache
– Min server memory – Using Address Windowing Extensions (AWE) • Only used in 32-bit installations when there is > 4GB of RAM
Configuration: Memory, CPU
• Memory options you can change (cont.) – Index create memory • Specifies the amount of memory to use in index creation sorts • 0 means dynamic • Have a good reason to change
Configuration: Memory, CPU
• Memory options you can change (cont.) – Minimum memory per query • Specifies the amount of memory to use for execution of a query • 0 means dynamic – 1024KB is default • Have a good reason to change
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Memory, CPU
• CPU settings that can be changed – Enable/disable processors • Licensing needs • System performance
– Affinity mask options • Processor affinity: binds processors to specific threads to prevent switching mid-execution • I/O affinity: binds disk I/O to specific processors
Configuration: Memory, CPU
• CPU settings that can be changed (cont.) – Maximum worker threads • CPUs do work on threads • Do you need to set a maximum? – Default is “0” which means dynamic
– Boost SQL Server priority – specifies whether SQL Server process should execute at a higher priority • Advanced option
Configuration: Memory, CPU
• Parallel queries are queries whose work is split among CPUs • “Cost” is a metric of time – A query with a cost of 1 says that theoretically will take 1 second • This was determined pre-release in the Microsoft labs therefore it will not likely match your system performance
Configuration: Memory, CPU
• You can configure how SQL Server handles parallel queries – Cost threshold for parallelism • SQL Server does not run a parallel plan for queries that “cost” less than this amount • Default value is 5 – This means that a query has to be estimated to run for 5 seconds before SQL Server will consider using a parallel plan – 5 is the default for a very good reason
Configuration: Memory, CPU
• You can configure how SQL Server handles parallel queries (cont.) – Max degree of parallelism • Controls how many processors are available to run a single statement • Default value is “0” which means to use all available CPUs • Can be overridden at query level
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
Configuration: Security
• Security settings that can be changed: – Authentication mode • Requires restarting SQL Server service to take effect
– Login auditing • Bare auditing – doesn’t record too much information – None, Failed only, Successful only, Both
– Server Proxy Account • Used with xp_cmdshell • Not likely to use but, if you do, be careful
Configuration: Security
• Security settings that can be changed (cont.): – Enable Common Criteria Compliance • Forces SQL Server to adhere to Common Criteria compliance • Developer and Enterprise edition only • More info: http://www.commoncriteriaportal.org/
– Enable C2 audit tracing • Audits all attempts to access statements and objects to file(s)
– Cross database ownership chaining • Coming up in later chapter
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
• Let’s look at each one
Configuration: Connections
• Connection options affect all connections to the instance – Some can be overridden at connection level
• Triple check this in development/testing before deployment – One setting change can cause critical parts of an application to fail
Configuration: Connections
• Connection options you can change: – Maximum number of concurrent connections • Default is “0” which is no limit
– Use Query Governor to prevent long-running queries • Great idea in theory; better to use Resource Governor
Configuration: Connections
• Connection options you can change (cont.): – Allow remote server connections • Do you want remote servers to run stored procedures on this server?
– Require distributed transactions for server-toserver communication • Uses Microsoft DTC to ensure that transactions completely succeed on all servers
Configuration: Connections
• Common connection options – All are “Off” by default – This allows client to set it’s own defaults Setting
Description
Default
ansi nulls
Does the statement “WHERE LastName = NULL” return any rows?
Statement would return 0 rows; requires ISNULL to test for NULL values
no count
“34 rows affected” messages are returned to client
quoted identifier
Do double quotes identify object names or query criteria?
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
Configuration
• Commonly changed configurations are: – Service accounts – Network protocols – Memory, CPU – Authentication mode and auditing – Connection options – Default file locations, compression settings
Configuration: File locations & Compression
• Defaults for new databases: – Default index fill factor • How full should a “page” be? • Default is to fill page completely
– Backup compression • Sets whether a backup is compressed by default • Any backup can be compressed or uncompressed regardless of this setting • For Enterprise and Developer edition
Configuration: File locations & Compression
• Defaults for new databases (cont.) – Database default locations • Configure the default locations for data and log files individually • For new databases only • Default locations are only used in statements that do not specify file locations
CREATE DATABASE ScottsDB
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008 • Demo
A Look at the Tools and Configuring SQL Server 2008 Presented by Scott Whigham
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
What We’re Going to Cover
• Configuring SQL Server 2008
How to Change Configuration Settings
How to Change Configuration Settings
• How to change configuration settings – Use SSMS for interactive or jobs – Use sqlcmd.exe for scripts
• Can use the GUI and Transact SQL scripts
Configuration
• GUI is great for learning • Scripts are better for – Change tracking – Undoing – Sharing across machines/users
• Changes require two things: – The actual change – Acknowledgement that the change should be applied
Configuration
• Tips – Have a “backup” or “back out” plan – Understand that SQL Server will often clear out portions of its memory after configuration changes • This can have severe short-term impact!
Configuration
• “Configured values” are what SQL Server will change to once you click “OK” • “Running values” are what SQL Server is currently using – Running values is a read-only view of what the system is using right now
Configuration
• To change configurations in the GUI: 1. Launch SQL Server Management Studio 2. Login as a sysadmin 3. Right click on server and select Properties
• After changes, click “OK” to save •
Try, try, try not to do this during production hours!
Configuration
• sp_configure is the Transact-SQL way to modify server configurations • To use sp_configure, you must first enable the “advanced” options EXEC sp_configure ‘show advanced’, 1 RECONFIGURE
Configuration
• sp_configure requires use of RECONFIGURE or RECONFIGURE WITH OVERRIDE to take effect without a restart of SQL Server service – Otherwise changes will not take effect until you stop/start
EXEC sp_configure ‘show advanced’, 1 RECONFIGURE
• An Overview of the Tools • Demo
Configuration
• Overview of Chapter
The Tools
Overview
In the next video…
• Configuring SQL Server 2008
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
This Chapter
• What this chapter covers – Databases, data bases and DBs • Creation and management
– Database file structure – An overview of configuring the database
This Chapter
• What this chapter does NOT cover – Every single database option available – Performance tuning and optimizations – Monitoring
This Chapter
• Pre-requisites to this chapter – You should already know how to create a database using the GUI
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
SQL Server Databases
• SQL Server databases have at least two files – Data files – Transaction Log files (commonly called “log files”)
• A database has at least one of each – We’ll look at when to have multiples of each later
SQL Server Databases
• Data files store everything that a user needs such as – The data – The indexes – Source code for routines (stored procedures, functions) – Security information (users, roles, permissions)
SQL Server Databases
• Transaction log files are used to protect SQL Server databases from “failure” • Failure could be – User error – Disk failure – Software failure – Dreaded BSOD (Blue Screen of Death)
SQL Server Databases
• Every write in a database is done inside of a transaction • All transactions are written first to the log and then to the data file • This is known as “Write Ahead Logging” – http://en.wikipedia.org/wiki/Write_ahead_logging
SQL Server Databases
• You begin a transaction with the BEGIN TRAN statement • A transaction may include many statements
SQL Server Databases
• A transaction remains “open” until either you: – COMMIT TRAN – ROLLBACK TRAN
(called “rolling back”)
• Once “closed”, the transaction may never be re-opened
SQL Server Databases
DECLARE @ProcessingHistoryId INT DECLARE @PayAmount DECIMAL(19,2) = $1.94 BEGIN TRAN INSERT PayrollProcessingHistory VALUES (‘Whigham’, ‘2009-04-01’, @PayAmount) SELECT @ProcessingHistoryId = SCOPE_IDENTITY()
INSERT ChecksToWrite VALUES (@ProcessingHistoryId , @PayAmount) COMMIT TRAN
SQL Server Databases
• Transactions in SQL Server are durable – Once the transaction is committed, the transaction will remain in the database even in the event of a system crash
• Demo of write-ahead logging
*More on transactions in our “SQL Server 2008 Programming” course
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
System Databases
• SQL Server uses SQL Server for configuration – Some configuration is in the registry – Some configuration is in XML files
• The majority of database and instance configurations are stored in the five system databases
This Chapter
• The five system databases: – master – model – msdb – tempdb – Resource
• Let’s look at each one
The master Database
• The master database: – Is the first database “loaded” by SQL Server service on start – Stores instance-level configuration such as memory, CPU, and the info set by sp_configure – Stores information about each database on the server – Stores the logins into the server – Stores routines that are shared among all databases
The model Database
• The model database: – Is a template for new databases – Whatever objects are in model will be created when a new database is created – Useful for standardization
The msdb Database
• The msdb database: – Stores any and all automation configuration and information • Log shipping, mirroring, jobs, alerts, mail • History of all of the above
– May optionally store SSIS packages – Stores history about backups – Used heavily by SQL Server Agent
The tempdb Database
• The tempdb database: – Is what the name implies: temporary storage – Temp tables are created here – Row versioning is done here – Is “wiped clean” at shutdown and created at startup • You can actually delete the database files on the hard drive after shutdown and SQL Server will create them anew
The Resource Database
• The resource database: – Hidden from view; must view at the disk level – A read-only database that contains all of the “system objects” in SQL Server • Each database has “pointers” to these system objects
– Makes it easy to upgrade • During upgrade, MSFT can just replace the resource database and immediately all databases on server have latest source code and objects
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Databases in SQL Server 2008
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Types of Databases
• Databases are either: – System databases – shipped with SQL Server; from Microsoft – User databases – everything else; not an official term
• A typical SQL Server has many user databases
Types of Databases
• User databases can fulfill many different functions – Customer Relationship Management (“CRM”) – Payroll – Document storage – Website analytics package – Configuration – Project management – Instrument measurement & Quality Control
Types of Databases
• Each of these databases has multiple needs – A CRM database needs to provide • Heavy read and reporting capability • Light to Moderate write capability • Could be anywhere from 5 reads for every write (5:1) up to 50:1 reads to writes
Types of Databases
• An instrument measurement database needs extremely fast write capability – 250+ writes per second – Reads are done at reporting level • Defect reports, abnormalities
– Could be 1000:1 writes for every read
Types of Databases
• A website analytics package might use multiple databases – One database needs extremely fast write capability to capture real time data – Another database needs extremely fast read capability for reporting
Types of Databases
• Database needs change with time – End of month reporting – End of year reporting – Tax season
Types of Databases
• Databases are classified as having one or more of these attributes: – Online transaction processing (“OLTP”) – Online analytical processing (“OLAP”)
• A database might serve both needs • It is generally better to serve a single purpose
Types of Databases
• A data warehouse is an example of an OLAP database • Data warehouses are “fed” by the OLTP systems on a scheduled basis – This ensures that the reports will be up to date – Often called a “batch load” or a “bulk load”
Types of Databases
• SQL Server Analysis Services (“SSAS”) stores a multidimensional data warehouse – Requires installing SSAS – Stores data in “cubes” – Use SSIS to load data from the SQL Server OLTP and/or OLAP database into the cubes * More information can be found in our SQL Server 2008 Analysis Services course
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases
• You can create databases: – In the GUI (SSMS) – Transact-SQL scripts – Using SQL Server Management Objects (SMO)
Creating Databases
• The GUI is great for one-time use • Scripts and SMO are great for standardization and re-creation – A software vendor that wants to create a SQL Server database during setup – A DBA needs the same database installed on multiple servers
Creating Databases
• Transact-SQL scripts can be executed in – SSMS – sqlcmd.exe – .NET, Java, or any language that can connect to SQL Server – 3rd party query tools
Creating Databases
• SQL Server Management Objects (SMO) can be used – .NET, Java, or any language that can reference the necessary assemblies
Creating Databases
• Databases are just files – Data files – Transaction log files
• Files must be local – No mapped drives or UNC paths
Creating Databases
• Data files come in two flavors: – Primary data file: the first data file created • Generally has an “.mdf” extension
– Secondary data files • Generally have an “.ndf” extension
Creating Databases
• There is only one transaction log regardless of how many physical log files • Log files generally have an “.ldf” extension
Creating Databases
• “When do I need more than one data file and/or more than one log file?” – Two reasons: • Too much data to store in a single data file or extreme logging needs mean that too much logging for one single file • Performance
Why should I use multiple files?
• Space and file management – May not have space available for growth on current drive; add new drive and new file – Data or Log file may be too large for one file – Data or Log file may not be easily re-created on another machine if it gets too large • When restoring a database, you cannot split files!
Why should I use multiple files?
• Performance – The more disks, the faster performance – SQL Server can read two files on two separate disks in parallel • Can nearly double read and write access
Why should I use multiple files?
• Files can be added dynamically – Instant access – Essential at a time when you’ve run out of log space!
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases
• After the database has been created, all file manipulation is done by referring to the logical file name – Adding space to a file – Shrinking a file – Removing a file
Creating Databases
• The logical file name is an abstraction layer – It need not refer to the database by name
• Physical file names cannot change while database is online – Logical file names can change at any time without affecting database status
Creating Databases
• Autogrowth is a safety net to prevent your database from preventing users from doing their jobs • The file will not grow until the file is 100% full – It then grows according to the file growth parameters
Creating Databases
• Autogrowth parameters: – Grow the file by a percentage of current size or a fixed amount (in MB) – Set maximum size
• Hint: Always set a maximum size – Do not let the file grow until it fills up the disk
Creating Databases
• Best Practices for Creating Databases – Create the database the size that you expect it to be in 12-18 months • This prevents autogrowth from happening during peak hours
– Do allow autogrowth • Keep parameters small and easy so that database can grow quickly and be usable quickly
Creating Databases
• Best Practices for Creating Databases (cont.) – Have a plan for growing the database in 12-18 months (or before autogrowth kicks in) – Always, always keep an eye on the file sizes • Particularly the transaction log
Creating Databases
• Expanding a file is easy and dynamic • We’ll take a look at working with multiple file databases in an upcoming video
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Filegroups
Filegroups
• A data file belongs to a filegroup • A filegroup contains n data files • A database may have n filegroups – A data file belongs to only one file group
• Filegroups allow you to do data placement and backup/restore on an entire filegroup
Creating Databases
• Each database has a PRIMARY filegroup – The primary data file belongs to the PRIMARY filegroup
Filegroups
• Transaction logs do not belong to filegroups
Multiple File Databases
• One advantage of filegroups is data placement – Tables can have “Regular Data” on one filegroup and “Text/Image” data on another CREATE TABLE dbo.MyTable ( MyColumn nchar(10) NOT NULL, MyVarColumn varbinary(MAX) NOT NULL ) ON RegularDataFG TEXTIMAGE_ON ImageDataFG GO
Multiple File Databases
• Filegroups can also be marked as read only – Any objects on the filegroup are read only while the rest of the database remains read/write ALTER DATABASE LearnItFirst MODIFY FILEGROUP [Sales2001] READONLY
Filegroups
• When there are multiple files in a filegroup, SQL Server uses a proportional file strategy – SQL Server does not fill up one file and then fill up the next file • Doing so would kill performance on a system that had files on separate drives
Filegroup Example
• LearnItFirst database has two data files in the MyFG filegroup – DataFile1: 100MB free – DataFile2: 200MB free
• SQL Server will write one extent to DataFile1 and two extents to DataFile2 – An extent is eight pages
Filegroup Example
• Once all files in the filegroup are full, it will expand the first file and write until it is full before expanding the second file
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Creating Databases in SQL Server 2008 Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Multiple File Databases
Multiple File Databases
• Adding a log file is easy – The log will now be “striped” across all log files – Can be done using GUI, TSQL or SMO
• Adding a data file is just as easy – The data will now be ”striped” across all files in the filegroup – Can be done using GUI, TSQL or SMO
Multiple File Databases
• When you add a new file to the database, it defaults to the PRIMARY filegroup – Best practice for multiple data file databases is to create a separate filegroup for user data and leave the primary data file as the sole file in the PRIMARY filegroup
Multiple File Databases
• Removing a file is hard – Requires “emptying” the file – Emptying the file requires you to move the data on the file to other data files in the same filegroup
• Demo of – A database filling up – Adding/removing files
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring SQL Server Databases
• Instance configuration affected every database on the instance – A single SQL Server instance may serve 1,000+ databases
• Each database may serve a different purpose and may benefit from individual configuration
Configuring SQL Server Databases
• SQL Server databases are autonomous; any configuration to one database has no effect on other databases • Different functions require different configurations – OLTP, OLAP, BLOB storage, Historical data only, mix
BLOB: Binary Large Object
Configuring SQL Server Databases
• This section focuses on the individual database configuration • We’ll cover the main options only – There are more than 30 different options
• We’ll cover upgraded databases in an upcoming section
Database Configuration
Database Configuration
• Database options are set by choosing between a predefined set of values – Most common is True/False
• Some database options override instance options – ANSI NULLs, Quoted Identifiers, et al
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
• Let’s take a look at each of these in the next video!
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Automatic” Category control how SQL Server proactively manages your database – “Auto Close” – Takes the database offline when the last user leaves the database • Defaults to FALSE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Create Statistics” • Statistics are used by SQL Server to compute the fastest route to data • If TRUE, SQL Server will create statistics on columns the first time they are referenced. • Defaults to TRUE; Do not change unless you know this will help your situation
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Shrink” • If your database has free space, SQL Server periodically will remove the free space and give it back to the operating system • Defaults to FALSE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Update Statistics” • Statistics can get out-of-date (“stale”) due to large amounts of UPDATEs and INSERTs • When SQL Server encounters a query with out-of-date statistics, the statistics are updated prior to the query execution – This can cause a lag
• Defaults to TRUE
Database Configuration
• Settings in the “Automatic” Category (cont.) – “Auto Update Statistics Asynchronously” • Statistics are updated when encountered but the query does not wait for the statistics update command to complete • Defaults to FALSE
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Cursor” Category control how SQL Server behaves with cursors – Few applications use cursors today – Be careful about testing before/after you change
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Some settings in the “Miscellaneous” Category override Instance-level defaults – Can set “Default Connection Options” at instancelevel – Can be override at both database-level and connection-level WARNING: Some apps depend on certain settings so do thorough testing when making changes
Database Configuration
• Other settings in the “Miscellaneous” Category are database-specific – Date Correlation Optimization Enabled • Used for foreign keys on DATETIME columns • Defaults to FALSE SELECT * FROM Parent p JOIN Child c ON p.DateTimeKey = c.DateTimeKey
Database Configuration
• Other settings in the “Miscellaneous” Category are database-specific – Cross-database Ownership Chaining Enabled • Covered in next chapter
– Trustworthy • Covered in next chapter
– VarDecimal Storage Format Enabled • In SQL Server 2008, all tables use the VarDecimal Storage Format • Was a changeable setting in 2005
Database Configuration
• More settings in “Miscellaneous” – Parameterization • • • •
Specifies how SQL Server works with query plan re-use Two choices: SIMPLE or FORCED Defaults to SIMPLE Can be overridden at query level
Database Configuration
• Parameterization continued… – When parameterization is SIMPLE and SQL Server recognizes a query that it thinks will be used in the exact same way but with different SARGs (Search ARGuments), it will attempt to create a reusable execution plan • Previously called “auto-parameterization”
Database Configuration
SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 1
SQL Server recognizes that this query will likely have multiple variations SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 1 SELECT * FROM AdventureWorks.Production.Product WHERE ProductID = 5
SQL Server creates a reusable, parameterized query plan that satisfies both queries
Database Configuration
• In SIMPLE parameterization, SQL Server will only create reusable plans for queries that it believes it can be reused • In FORCED parameterization, any literal in any SELECT, INSERT, UPDATE or DELETE statement is parameterized
Database Configuration
• FORCED parameterization can offer a tremendous performance gain for applications in which the database is not auto-parameterizing queries
Database Configuration
• FORCED parameterization can slow down databases as well due to a bloated plan cache or reusing sub-optimal plans • See this article for more: – http://msdn.microsoft.com/en-us/library/ms175037.aspx
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Configuring Databases
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Recovery” Category control how SQL Server verifies the contents on a “page” – SQL Server stores data and indexes on “pages” – The “Page Verify” option tells SQL Server how to check that the data/index page is not corrupt – Data can become corrupt due to incomplete I/O writes and/or disk errors
Database Configuration
• “Page Verify” has three choices: – CHECKSUM (the default) • SQL Server computes a checksum of what the page’s checksum would be if the wrote was successful • After the write, SQL Server compares the checksum values • If they are not the same, the page is logged in msdb’s suspect_pages table
Database Configuration
• “Page Verify” has three choices (cont.): – TornPageDetection • SQL Server writes a “flag” bit for every 512 bytes • By counting the “flags”, SQL Server can determine if the page has the correct number of bytes • However, it cannot determine that the data is corrupt inside of the 512 bytes – It just counts that 512 bytes were written
Database Configuration
• “Page Verify” has three choices (cont.): – None • No page verification is done
• Suggestion: Leave the default setting asis (CHECKSUM)
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• Settings in the “Service Broker” Category control… well, the Service Broker • Service Broker allows for messaging and queuing applications
Database Configuration
• There are 6 categories of configuration – Automatic – Cursor – Miscellaneous – Recovery – Service Broker – State
Database Configuration
• “State” controls whether SQL Server database is – Online or Offline • i.e. accepting connections or not
– Read-Only or Read-Write
• Databases default to Online, Read-Write
Database Configuration
• “State” also controls whether SQL Server is – Single-user, Multi-user or “Admins Only” • “Admins Only” is technically called RESTRICTED_USER
– Whether “Transparent Data Encryption” is enabled • More on this later (requires more setup)
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Indexes
• SQL Server has three types of indexes • (or “indices” if you prefer)
– Clustered indexes – Nonclustered indexes – Full-Text indexes • Not covered in this course
Indexes
• Most tables have both clustered and nonclustered indexes – Only tables that store large/wide text columns that need frequent searching would use Full-Text
• The purpose of an index is to speed up searches
Indexes
• Queries use “Search Arguments” or SARGs SELECT PhoneNumber FROM PhoneBook WHERE LastName=‘Whigham’ AND FirstName=‘Scott’
• To satisfy the SARGs, it helps if the data is sorted in LastName, FirstName order
Indexes
• A clustered index is really how the actual data is stored – A clustered index on “LastName, FirstName” means that the actual data on the page is sorted in alphabetical order by LastName, FirstName
• When you create a primary key, the default is to create a clustered index on that key
Indexes
• Consider this query: SELECT LastName, FirstName FROM PhoneBook WHERE PhoneNumber = ‘+1(214) 555-1212’
• Would having the data sorted by LastName, FirstName order help this query?
Indexes
• Solution: create a nonclustered index on PhoneNumber! – A nonclustered index is a copy of the index keys and the primary key sorted by the index keys – Now a copy of the PhoneNumber column is created along with a pointer back to the primary key
Indexes
• Indexes use a fill factor to determine how full to create the page – A fill factor of 50% means that the page will be created 50% full (thus leaving 50% for new rows to be written to the page)
Indexes
• A full index/data page cannot have any more data written to it – Imagine a phone book page full of “Smith” entries – No room to write any more – Next page is a full page of “Sullivans”
• A new “Smith” moves to town – How does this record get added to the PhoneBook table?
Indexes
• Solution: A Page Split! – When a page split occurs, a new page is created (somewhere on the disk) – Half of the original page’s data is “cut and pasted” onto this new page along with the new row
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Indexes
• Page splits cause poor performance in multiple ways – The time it takes to allocate a new page (and maybe even having to grow the database) – The time it takes to “cut and paste” (and log) – The pages are not contiguous any longer thus causing more work at the disk level
• This is called “Index Fragmentation”
Indexes
• Solution: You need to Rebuild or Reorganize Your Indexes Periodically – Called “Defragmentation” – Speeds up performance
Indexes
• An index “REBUILD” creates the best performance • SQL Server re-creates the index internally again • Once complete, the existing index is dropped • There is a period of time where the index/data is unavailable (“locked”) • No pages are split after running – all are contiguous
Indexes
• An index “REORGANIZE” does a “best effort” defragmentation but leaves the data online • Reorganizes the “leaf nodes” of the index • Some fragmentation may still exist since it does not fix page splits
Indexes
• To rebuild an index, use the ALTER INDEX command -- Rebuild all indexes on table ‚Members‛ ALTER INDEX ALL ON Members REBUILD WITH(ONLINE = ON) -- ONLINE=ON keeps the table available -- during the rebuild process
Indexes
• To rebuild an index, use the ALTER INDEX command -- Rebuild only the primary key index ALTER INDEX PK_Members ON Members REBUILD WITH(ONLINE = ON, FILLFACTOR=50) -- Keeps data available but uses a new -- fill factor of 50%
Indexes
• To reorganize an index, use the ALTER INDEX command -- Rebuild only the primary key index ALTER INDEX PK_Members ON Members REORGANIZE -- Reorgs are always online operations
Indexes
• Index defragmentation needs to be part of your normal maintenance – How often you do each requires a thorough understanding of your table data and usage
• You will create jobs and maintenance plans to do this work using Transact-SQL scripts
Indexes
• Scenario: “Members” table adds 10,000 new rows per week and is heavily read – Primary key is an auto-incrementing IDENTITY – Fill factor for all indexes is 75% – One primary key (on IDENTITY column) – Six nonclustered indexes to cover common queried columns
Indexes
• Should we rebuild or reorg? – Wrong question! – Start with the idea that you want to rebuild but will “fall back” to a reorg in the event that the table is too heavily used • In this event, scheduled maintenance windows (if possible) can be the time that you do rebuilds
Indexes
• Frequently Asked Questions – How often should I defrag? • As often as possible without affecting users • When the table becomes fragged!
– Should I rebuild/reorg all indexes at once or do them individually? • It’s easier management-wise to do all at once • Large indexes/tables may require separate operations
Indexes
• Let’s do a demo!
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Partitions
• Large tables often have data that is used in different ways – An “Orders” table might have 2-10 years of data – Some data is heavy read/write while other data is read-only
Partitions
• With large tables – Index maintenance can take hours – Queries against large tables can take a long time • In a table of 200,000,000 rows, how long would it take to find all orders placed on Jan 1, 2001 at 23:43:23.243?
Partitions
• Large tables can be partitioned into smaller, more manageable subsets – An “Orders” table can be partitioned into one partition per year – A query for rows WHERE OrderDate = ‘Jan 21, 2001’ now only needs to access a single partition
• Indexes can be partitioned as well
Partitions
• Partitions allow you to create subsets of the data and to place them strategically – Partitions can be spread across multiple filegroups – Queries only need to access the partition(s) that serves their data
Partitions
• Partitions can be spread across multiple filegroups – Some partitions can be read-only while others are read-write – Can backup more heavily-written-to partitions more frequently
Partitions
• Rows are partitioned horizontally – This means that the entire row is in a single partition
• Time is a common partitioning strategy – One partition for each year – One partition for each quarter – One partition for each month
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
An Overview of Indexes and Partitioning Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Partitions
• Creating partitions requires: – A Partitioning Function – A Partition Scheme – A table/index
Partitions
• Partitioning Functions define “what goes where” – You define boundaries and ranges
• Boundaries define the number of partitions • Ranges define how the boundaries are defined
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008)
Partitions
• This partition function can be used for any table/index that has an INT column • Three partitions will be created above • RANGE defines how the partitions are defined
Partitions
RANGE RIGHT FOR VALUES (2007, 2008) Partition 1
Partition 2
Partition 3
WHERE TheYear < 2007
WHERE TheYear >= 2007 WHERE TheYear => 2008 AND TheYear < 2008
TheYear is an INT column in the above table
Partitions
RANGE LEFT FOR VALUES (2007, 2008) Partition 1
Partition 2
WHERE TheYear <= 2007 WHERE TheYear > 2007 AND TheYear <= 2008
Partition 3 WHERE TheYear > 2008
Partition Schemes
• A Partition Scheme maps the partition function(s) to a filegroup(s) – Partition functions can belong to multiple partition schemes – One partition scheme can only contain one partition function
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008) GO CREATE PARTITION SCHEME MyScheme AS PARTITION ScottsPF TO (myFilegroup1, myFilegroup1, myFilegroup2)
Partition Schemes
• The preceding sample mapped the three partitions from “ScottsPF” to two different filegroups TO (myFilegroup1, myFilegroup1, myFilegroup2)
• Partition schemes can also map all partitions to the same filegroup ALL TO (myFilegroup1)
Partition Schemes
• You can partition tables and/or indexes – If you partition the table, the indexes are partitioned by default (unless otherwise specified during index creation)
Partitions
CREATE PARTITION FUNCTION ScottsPF (INT) AS RANGE RIGHT FOR VALUES (2007, 2008) GO CREATE PARTITION SCHEME MyScheme AS PARTITION ScottsPF ALL TO (‘default’) GO CREATE TABLE MyTable (MyID INT, TheYear INT) ON MyScheme(TheYear)
Partition Schemes
• Let’s do a demo!
Managing Database Files
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Managing Database Files
• We’ve already seen how to grow databases – By adding space to an existing file – By adding additional data or log files
• Now it’s time to see how to make our “footprint” smaller!
Managing Database Files
• Up first: Removing a Database • When you remove a database, you are removing – Users – Data (tables, indexes) – Stored procedures, functions, triggers, etc
Managing Database Files
• Removing a database deletes the files – Once a database has been deleted, it cannot be un-deleted! • Edit -> Undo will not help!
• You cannot delete a database that is in use
Managing Database Files
• To delete a database in the GUI, right click on the database and select Delete • To delete a database using Transact-SQL: DROP DATABASE MyDb
Dropping a database
Managing Database Files
• To protect ourselves from dropping databases, we employ two strategies: – Backups – Scripts
• We’ll look at backups in an upcoming chapter • Let’s look at scripts now!
SQL Server Scripts
• Sql scripts can do almost anything – Create, alter, drop databases – Create, alter, drop tables, functions, stored procs – Insert, update, delete rows
• All “real” DBAs use scripts • Scripts are typically stored in .sql files
SQL Server Scripts
• Scripts can be executed – In SSMS – Using sqlcmd.exe – Inside of any custom or 3rd party app that can connect to SQL Server – In batch files (that call sqlcmd) – SQL Server Management Objects (SMO) – PowerShell • Which can handle it’s own scripts and sql scripts!
SQL Server Scripts
• You can also generate scripts – Create, alter, drop databases – Create, alter, drop tables, functions, stored procs – Insert, update, delete rows
• Script generation can be done – In SSMS – “By Hand”
SQL Server Scripts
• Let’s demo!
Managing Database Files
Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
Managing Database Files
• Shrinking databases is a sometime necessity • You can shrink – Individual data files or log files – An entire database (all files)
• Shrinking can be done with the GUI, TSQL, or any of the other methods
Managing Database Files
Shrinking a Database
• There are two options for shrinking an entire database 1. Remove all free space after the last page • PROs: Fastest, least impact • CONs: Not likely to remove as much as the 2nd option
Shrinking a Database
• There are two options for shrinking an entire database (cont.) 2. Reorganize the entire database to compress the pages at the “front” of the file, then remove free space • PROs: Removes maximum amount of free space, can specificy a “Target Percentage” of free space (“I want 10% free space left in file”) • CONs: Slow, performance hog
Managing Database Files
Shrinking a Database
• To shrink a database using Transact SQL, use DBCC SHRINKDATABASE DBCC SHRINKDATABASE ( database_name [ , target_percent ] [ , { NOTRUNCATE | TRUNCATEONLY } ] )
Shrinking a Database
• NOTRUNCATE – Compress pages to “front” of file – Does not release freed space to OS – Only available in T-SQL
• TRUNCATEONLY – Does not compress the pages to the front of the file – Releases freed space after the “last” page to the OS – Default in SSMS
Shrinking a Database
-- TRUNCATEONLY: DBCC SHRINKDATABASE(N'LearnItFirst.com') -- TARGET SIZE of 10% free space: DBCC SHRINKDATABASE(N'LearnItFirst.com‘,10) -- NOTRUNCATE: DBCC SHRINKDATABASE(N'LearnItFirst.com', NOTRUNCATE)
Shrinking a Database
• Shrinking individual files offers more flexibility – Can specify target sizes, truncation options or empty individual files DBCC SHRINKFILE ( { logical_file_name | file_id } { [ , EMPTYFILE ] | [[ , target_size ][ , {NOTRUNCATE | TRUNCATEONLY }]]} )
Shrinking a Database
• EMPTYFILE – Moves pages to another file in the file group
• target_size – Cannot compress data – Defaults to MB – If not specified, uses the creation size
Shrinking a Database
• Let’s demo!
Suggested Changes After the Upgrade Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Testing Acceptance/sign-off
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade: Patching
• It’s likely that, after your upgrade, there are hotfixes, patches, service packs, etc available • It is often difficult to determine which is needed – How do you know you are experiencing a bug until you test your server?
After the Upgrade: Patching
• Remember that: – Service packs are the most-tested – Cumulative Updates are the next-most-tested – Critical Updates are the least-tested
• You don’t have to install the most recent updates – Unless there is a specific security issue
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade: Compatibility
• The first and most obvious thing to understand is compatibility level • SQL Server allows a database to use one of three compatibility levels – 80 (SQL Server 2000) – 90 (SQL Server 2005) – 100 (SQL Server 2008)
After the Upgrade: Compatibility
• Changing a database’s compatibility level provides partial backward compatibility – Setting a database’s compatibility level to 80 makes that database behave as though it was on a SQL Server 2000 instance – Allows you to upgrade to SQL Server 2008 without making breaking application changes
After the Upgrade: Compatibility
• When you upgrade a database/instance, the previous version remains as the compatibility level – If you upgrade “Northwind” from SQL Server 2000 to 2008, the Northwind database remains in SQL Server 2000 compatibility level • All other databases on SQL Server 2008 are in SQL Server 2008 compatibility level
After the Upgrade: Compatibility
• Let’s demo!
Suggested Changes After the Upgrade Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• If you upgraded from SQL Server 2005 – There are no database-level configuration changes needed
• If you upgraded from SQL Server 2000, you might want to make changes
After the Upgrade
ALTER DATABASE ‘AdventureWorks’ SET ANSI_NULL_DEFAULT ON , ANSI_NULLS ON , ANSI_PADDING ON , ANSI_WARNINGS ON , ARITHABORT ON , AUTO_CREATE_STATISTICS ON , AUTO_UPDATE_STATISTICS ON , NUMERIC_ROUNDABORT ON , PAGE_VERIFY CHECKSUM
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• What should you test? – Have the developers unit test any applications using the database • Not always possible with 3rd party apps
– Reports should be verified to ensure that they run – SQL Server jobs should be run to ensure they still “work” as expected – DTS and/or SSIS packages
After the Upgrade
• Known trouble areas – SQL Server jobs • If jobs point to specific files/folders, verify that – SQL Server service(s) have access – The file/folder paths have not changed or, if they have, change job definition
– DTS to SSIS – Database compatibility
After the Upgrade
• After your upgrade is complete, there are still critical steps 1. 2. 3. 4. 5. 6.
Patching the new SQL Server installation Verifying/changing configuration settings Testing Implement any changes that testing uncovered Verifying configuration settings Acceptance/sign-off
After the Upgrade
• Once the DBA has completed his/her testing, recruit other departments – Developers test “plumbing” – Quality Assurance (“QA”) tests application compatibility – System administrators test backup routines
The SQL Server 2008 Sample Databases Presented by Scott Whigham
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
What We’re Going to Cover
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
The SQL Server 2008 Sample Databases
• SQL Server 2008 does not ship with any sample databases – SQL Server 6.x shipped with the pubs database – SQL Server 7.0 and 2000 shipped with pubs and Northwind – SQL Server 2005 shipped with AdventureWorks and AdventureWorksDW
The SQL Server 2008 Sample Databases
• There are sample databases for SQL Server 2008 – You must be smart enough to figure out • • • •
What they are called How to download them How to install them Which to use for what purpose
The SQL Server 2008 Sample Databases
• Let’s answer each question individually : – What they are called • The sample databases are called collectively “AdventureWorks” • They are the sample databases for the fictitious AdventureWorks Cycling Company
What is AdventureWorks
• The AdventureWorks Cycling Company – Sells through internet and brick & mortar channels – Covers commonly used patterns for working with products, sales, contact management and more
The SQL Server 2008 Sample Databases
• AdventureWorks stores roughly three years worth of sales data in: – An OLTP database – An OLAP database
The SQL Server 2008 Sample Databases
• How do you get the sample databases? – The files are available at http://www.codeplex.com/MSFTDBProdSamples – Shameless marketing from MSFT
The SQL Server 2008 Sample Databases
• How do you get them onto your SQL Server? – One of three ways: • Install from .msi • Restore from backup • Run the scripts
– Good luck!
The SQL Server 2008 Sample Databases
• How do you know which to use for what purpose? – There are three different types of databases: • OLTP, OLAP and a “Light” version of the OLTP
The SQL Server 2008 Sample Databases
• This is where it gets confusing… – Each database has several names depending on where you look – There are as many as three different “versions” of each database in the default installation – There are many different files you can download once you actually figure out where to look for it
The SQL Server 2008 Sample Databases
• A look at the default install using .msi installation:
The SQL Server 2008 Sample Databases
• On the CodePlex website, they make mention of three databases in the AdventureWorks suite: – AdventureWorks OLTP – AdventureWorks OLAP – AdventureWorks LT (Light
The SQL Server 2008 Sample Databases
• AdventureWorks OLTP actually goes by many names – AdventureWorks OLTP in documentation/website – AdventureWorks OLTP and AdventureWorks 2008 OLTP in the downloaded .zip file – AdventureWorks and AdventureWorks2008 in installation
The SQL Server 2008 Sample Databases
• AdventureWorks OLAP also goes by many names – AdventureWorks OLAP in documentation/website – AdventureWorks 2008 Data Warehouse in the downloaded .zip file – AdventureWorksDW and AdventureWorksDW2008 in installation
The SQL Server 2008 Sample Databases
• AdventureWorks LT (Light) is the fewertables version of AdventureWorks OLTP – Meant to be simpler to learn and train people on
The SQL Server 2008 Sample Databases
• Which should I use when? – “I’m a total beginner to SQL Server” • AdventureWorksLT
– “I want to learn about the new datatypes and/or new features in SQL Server 2008” • AdventureWorks2008
– “I want to learn about data warehousing” • AdventureWorksDW
– “I want to learn about the new features in SQL Server 2008 data warehousing” • AdventureWorks2008DW
• Creating Databases • Adventure Works • Configuring Database Options
Management
• Overview of Chapter • How Databases Work in SQL Server • The System Databases • Types of Databases
Creation and Demo
Overview
In the next video…
• Indexes and Partitioning • Managing Database Files • What to Do After Upgrading
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
What We’re Going to Cover
• Encryption overview • Linked servers • SIDs and User accounts
Chapter Overview
• This chapter focuses on – How to create and manage logins – How to design and manage database security – Understanding Transparent Data Encryption – Creating and managing Linked Servers
Chapter Overview
• SQL Server Security is not that difficult – Understanding the main security model is easy – Remembering what each term does is easy – Understanding what to do when is easy once explained
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
What We’re Going to Cover
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Security
• A primary principle of security is to “reduce the attack surface area” • Another main focus is to only allow the least amount of privileges needed to do a task • This chapter teaches how to do both
SQL Server Security
• There are many levels of security in SQL Server – Physical machine – Operating System – Network – Instance – Database
SQL Server Security
• Physical Machine – Who has access to the actual machine? • Hard drives in many servers can easily be removed
– Recommendations to reduce attack surface area: • Server in a locked room with access limited to systems admins only
SQL Server Security
• Operating System – Should you run other software on this machine? • Internet Information Server (IIS), Exchange, SharePoint
– Which users have logins to the server? • If they can login to the server, can they make a file copy of SQL Server databases?
– Recommendations to reduce attack surface area: • The more critical/sensitive/large the database(s), the more likely you need a dedicated machine • Only allow system administrators and DBAs to login to the operating system
SQL Server Security
• Network – By changing default network settings you can reduce the surface area • Changing default instance TCP port from 1433 to another value
– Recommendations to reduce attack surface area: • Place your web SQL Server behind a firewall to prevent remote access (if possible) • Consider allowing local-only connections for applications installed on the server
SQL Server Security
• Instance-level – Defines who can login to the SQL Server instance and what permissions they have instance-wide – sa has instance-wide administrator privileges – Anyone who can login to the SQL Server is called a “login”
SQL Server Security
• Database-level – Defines which logins are able to use this database and what permissions each has in that database – By default, database permissions are granular • Permissions in “AdventureWorks” do not mean a user has those same permissions in another database
SQL Server Security
• Let’s look in-depth at instance-level and database-level security next!
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video…
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Security
• Something you need to know before we begin: – Permissions are GRANT, DENY and REVOKE • REVOKE is like an eraser • DENY cannot be overturned • One GRANT is all it takes
SQL Server Security
• Other basic terms to know are: – Logins – Users – Roles – Principals
• Let’s do a quick definition of each!
SQL Server Security
• New term: “Login” – To use a SQL Server, you must have a login – The “sa” login has full rights to the instance – Having a login does not necessarily give you access to a database
SQL Server Security
• New term: “User” – To use a database, you must have a user account • For brevity, just called a “user”
– Just having a user account does not mean you can access a particular table
SQL Server Security
• New term: “Role” – Logins or users can be grouped into a “Role” and then permissions granted to the role • Called “Groups” in Windows and other systems
– There are server roles (instance-level) and database roles – SQL Server combines login/user/role level permissions when evaluating requests
SQL Server Security
• New term: “Principal” – “Principals are entities that can request SQL Server resources”1 • A resource is anything from logging in to manipulating data
– A principal is an abstraction layer that encompasses users, roles, and logins • There are server principals (logins, server roles) and database principals (users, database roles) 1 – Source: SQL Server Books Online
SQL Server Security
• Why the abstraction layer? – Instance-wide permissions can be granted: • To an individual login • To a role that the login belongs to
– Remember that SQL Server combines the login, user and role permissions to determine whether a user is GRANT or DENY • A single DENY prevents access
SQL Server Security
• Why the abstraction layer?
(cont.)
– A “Login” can be created from: • Windows Group – All members of the group are granted/denied access
• Windows User account – A single Windows User account is granted/denied access
• SQL Server Authentication login – The DBA creates a login/password and grants/denies access
SQL Server Security
• Why the abstraction layer?
(cont.)
– By using the generic term “principal” when talking about permissions, it encompasses roles as well as logins of all types – Easier to say “Make sure to DENY all principals” than to say “Make sure you DENY John, Mark, Jane, the CustomerSupport role, and the public role”
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video…
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• The SQL Server DBA is like the door person at an exclusive, invitation-only party – To enter requires your name being on “the list” – If your name isn’t on the list, you can’t get in!
• “The List”, in SQL Server terms, is the list of logins
SQL Server Logins
• A login can be created from: – Windows User account – Windows Group – SQL Server login
• Logins can be granted or denied access
SQL Server Logins
• Example scenario: – The Windows Group “SQLDevs” is granted access – Chad is a member of SQLDevs – Chad’s latest code check-in caused major data problems – What are our options to prevent Chad from logging in?
SQL Server Logins
• Solution #1: Remove Chad’s Windows User account from the Windows group – PROs: • Problem solved quickly and easily
– CONs: • The Windows Group may have other necessary permissions outside of SQL Server • SQL Server DBA may not have rights to Active Directory
SQL Server Logins
• Solution #2: 1. Add Chad’s Windows User account as a login 2. Deny Chad’s Windows User login access 3. Now all other SQLDevs except Chad can login
• PROs – Does not affect Chad’s other Windows permissions outside of SQL Server – SQL Server DBA can implement regardless of Active Directory access
• CONs – Requires an extra step
SQL Server Logins
• Let’s demo!
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• SQL Server authentication logins are used when a user/application does not have a Windows account – A 3rd party application uses a custom login
• The “sa” account is a SQL Server login
SQL Server Logins
• To use SQL Server logins requires Mixed Mode – Changing authentication mode requires restarting SQL Server
SQL Server Logins
• SQL Server logins can use password policies of the computer/domain
SQL Server Logins
• Administrative Tools -> Local Security Policy
SQL Server Logins
• There are other logins as well for specific situations – Certificate-based logins and logins mapped to an asymmetric key • Used to sign routines for extra/elevated security
– Logins mapped to a credential • Credentials are simply a fancy database name for a Windows user/password
SQL Server Logins
• All logins have – A default database – Default language for error messages – Server role membership(s) – User mapping – Securables – Status
• Let’s look at each in the next video!
SQL Server Logins
• But first – demo!
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server Logins
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server Security
• The default database is the database the user is placed in upon login unless the user requests a different database • The default language specifies the language of error messages and functions
SQL Server Security
• Server roles allow for instance-wide grouping of permissions – Inflexible • Permissions cannot be changed • Cannot create your own roles
– To use, put logins in and take them out
SQL Server Security
• sysadmin role has all permissions – “sa” login is a member of sysadmin
• dbcreator sounds good but…. – Members can DROP any database on the server – (Documentation does not list that little goodie…)
SQL Server Security
• Be careful - each server role has the ability to add other logins to it! – If I add Chad to the “dbcreator” role he can add others to it
• Suggestion: Grant permissions in the Securables tab (upcoming)
SQL Server Logins
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server Security
• A login must be made a user of a database upon creation – Unless it is a member of sysadmin server role
• User accounts coming up in a future video…
In the next video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
SQL Server 2008 Security
Presented by Scott Whigham
In this video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
Securables
• First thing: SQL Server uses the GDR for permissions – GRANT – DENY – REVOKE (i.e. The Eraser)
Securables
• Securables are another abstraction layer – Securables are simply things that you define permissions to • Tables, views, etc
• You define permissions on securables to principals
Securables
• Securables have “scope” – Server • Database, Login
– Database • User, role, schema
– Schema • Table, view, procedure, function
Securables
Securables
• For each of these, there are three basic permissions – CONTROL – ALTER – VIEW DEFINITION • Each level can have individual permissions as well – Logins can have IMPERSONATE permission – TABLES have SELECT permission
Securables
• CONTROL permission effectively gives ownership rights – Includes the ability to give others rights as well -- Give the login Chad sysadmin-level rights:
GRANT CONTROL SERVER to Chad
Securables
• ALTER permission gives ability to modify – ALTER ANY <entity> • Permission applied for all entities of that entity type
– ALTER < entity> • Permission applied for only the given <securable>
Securables -- Give the login Terry the ability to change -Chrissy’s password
GRANT ALTER ON LOGIN::Chrissy TO Terry -- Give Chrissy the option to change -- anyone’s password GRANT ALTER ANY LOGIN TO Chrissy -- Yikes!!!!!
Securables
• VIEW DEFINITION permission allows for viewing: – Source code if given to stored procedures, functions, triggers, views – Login names/info if given at Login level – Table metadata if given at table level
Securables -- Give the login Terry the ability to view -information about Chrissy’s login GRANT VIEW DEFINITION ON LOGIN::Chrissy TO TERRY
-- Give Chrissy the option to view the source -code for a view vw_Customer GRANT VIEW DEFINITION ON vw_Customer TO Chrissy
Securables
• Each permission can also be granted using the WITH GRANT option – If I give Chad the VIEW DEFINITION permission on a stored procedure, he can then give that to Chrissy
Alternative to db_creator role
• Remember our problem with db_creator? – Any member can now ALTER or DROP any user database
• It is recommended that you do not use db_creator
Alternative to db_creator role
• To give logins the ability to create a database without giving them “extra” permissions, grant them the server permission “Create any database”
Alternative to db_creator role
Alternative to db_creator role
Alternative to db_creator role
Securables
• Let’s demo the server-scoped securables
SQL Server 2008 Security
Presented by Scott Whigham
In this video…
• All logins have – A default database – Default language – Server role membership(s) – User mapping – Securables – Status
Status
• Logins can be: – Disabled or Enabled – Grant or Deny – Locked out!
A most confusing screenshot…
Account Status
• Logins that are “Disabled” or have “Deny” cannot login – No functional difference
• If you “Deny”: DENY CONNECT SQL to [login]
• If you “Disable”: ALTER LOGIN [login] DISABLE
Status
• Disabling or denying logins: – Allows you to deny access to logins that you may later need to restore access • Consultants, staff on vacation or out of the office
– Allows you to deny access to creators of objects (databases, tables, etc) yet still let others use those objects • Alternative would be to transfer object ownership to other login and then drop the login
Locked out!
• A SQL Server authentication login can be locked out due to policies
Locked out!
• There are two ways to reset a locked-out account: – By resetting the password – By resetting the lockout time
• It is recommended that you reset lockedout accounts by resetting the password – Resetting the lockout time will clear any password history associated with the account!
Locked out!
• To reset a locked-out account by resetting the password, use ALTER LOGIN ALTER LOGIN [Scott] WITH PASSWORD=‘e%3I' UNLOCK
Locked out!
• To reset a locked-out account without resetting the password, use ALTER LOGIN … WITH CHECK_POLICY ALTER LOGIN [Scott] WITH CHECK_POLICY = OFF GO ALTER LOGIN [Scott] WITH CHECK_POLICY = ON GO -- Account may now login using existing password
SQL Server 2008 Security
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Accounts Added by SQL Server
• When you install SQL Server, there are several accounts added by default – SQL Server authentication accounts – Windows authentication accounts – Certificate-based logins
SQL Server accounts added
• sa – Full permissions – Can be disabled – Only available in Mixed mode – Recommendation: DON’T USE IT • Create Windows authentication logins for your sysadmins instead
Certificate-based accounts added
• Logins that are surrounded by the double hash signs are Certificate-based SQL Server logins – – – –
##MS_SQLReplicationSigningCertificate## ##MS_PolicyEventProcessingLogin## ##MS_PolicySigningCertificate## ##MS_PolicyTsqlExecutionLogin## (and several more)
• These get installed depending on which features you have installed
Windows-based accounts added
• SQL Server 2008 also adds several Windows-based logins – NT AUTHORITY\SYSTEM – an internal, default Windows group that is granted sysadmin privilege • Do not modify this account’s privileges • Used by Microsoft Update, service packs, hotfixes, and the SQL Writer Service (and more) • For more information, view KB article 932881
Windows-based accounts added
• SQL Server 2008 creates a local Windows group for each service that you install • These groups are subsequently granted SQL Server logins and given sysadmin privilege
Windows-based accounts added
• For a default install of SQL Server, there are two critical Windows Groups installed: – SQLServerMSSQLUser$<machine_name>$
• Remember: Instance IDs for a default instance are MSSQLSERVER
Windows-based accounts added
• For the server named LEARNITFIRST with a default instance of SQL 2008: – SQLServerMSSQLUser$LEARNITFIRST$MSSQLSERVER – SQLServerSQLAgentUser$LEARNITFIRST$MSSQLSERVER
• For a server named COLTRANE with a named instance of SQL Server with an instance ID of “Development”: – SQLServerMSSQLUser$COLTRANE$DEVELOPMENT – SQLServerSQLAgentUser$COLTRANE$DEVELOPMENT
Windows-based accounts added
• SQLServerMSSQLUser group: – Any user in this group can login to SQL Server as a sysadmin – Also has the full privileges of the SQL Server service account – No need to directly edit this group; SQL Server will do the right thing when you change service accounts through the SQL Server Configuration Manager
Windows-based accounts added
• In SQL Server, you will see these new accounts listed as: – NT SERVICE\
Account Provisioning
• Any accounts that you provision to be system administrators during installation will become sysadmins – Make sure to provision your Windows account during installation!
Account Provisioning
Changes in security
• Unlike previous editions, operating system and domain administrators are not sysadmins in SQL Server by default – BUILTIN\Administrators is no longer added by default – Can be added manually if necessary
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Users
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Database-Level Security
• A login allows access to the SQL Server • A user account is required to access a database • Remember: logins and users are actually “principals”
Database-Level Security
• You, the DBA, can grant user accounts: – When creating the login – After the login has been created – Using SSMS’s GUI tools – Using T-SQL’s CREATE USER command
Database-Level Security
• There are four built-in user accounts – dbo • Full privileges; the person who creates the database is the dbo by default unless changed • Sysadmins are dbo of every database
– Guest • Disabled by default
– INFORMATION_SCHEMA • Internal
– Sys • Internal
Database-Level Security
• A login has no user accounts created for it by default – If you enable Guest in a database, this means that any login can use that database even without a user account – Guest is disabled by default; It is not a standard practice to enable Guest
Database-Level Security
• A login may have zero or one user account per database – A login without a user account in a database cannot access that database unless the Guest account is enabled
Database-Level Security
• User names do not have to match login names – Let’s look at the login ScottWhigham: Database
User mapping
master
guest
model
guest
msdb
ScottWhigham
tempdb
guest
AdventureWorks
<none>
ScottsDatabase
dbo
WebsiteMarketing Scott
Database-Level Security
• A login may have zero or one user account per database – A login without a user account in a database cannot access that database
Database-Level Security
• Imagine your SQL Server as a 100-floor skyscraper – Huge foyer complete with menacing looking security guards – Unlike most skyscrapers, you must show your key card (i.e. your credentials) to enter the foyer Image by wallyg at flikr.com
Database-Level Security
• Think of the “secure” foyer as the master database – Anyone who has access to any floor of the skyscraper is allowed in through the turnstiles and into the foyer – Certain staff are only allowed into the foyer
• In SQL Server, this is called a login
Database-Level Security
• The elevators of our skyscraper require a key card – This key card determines which floor(s) you have access to – You may only go to floors that your key card specifically allows Image by SYangPhoto at flikr.com
Database-Level Security
• In SQL Server parlance, the “floor” is really a database – Logins may not use a database unless they have a user account – A user account in one database has no effect on whether a login may access another database • Example: your key card lets you go to the 32nd floor. Would you also expect that key card to allow you permission to the 23rd floor? No!
Database-Level Security
• Let’s demo!
SQL Server 2008 Security: Schemas
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles and Schemas
• The word schema has multiple definitions in SQL Server: – The structure of a database including tables, columns, and relationships – An XML schema collection – A container
Roles and Schemas
• The word schema has multiple definitions in SQL Server: – The structure of a database including tables, columns, and relationships – An XML schema collection – A container
Roles and Schemas
• DBAs and Developers place objects in schemas for – Organization of objects – Security of objects
HumanResources schema
• Organization: • HumanResources schema in AdventureWorks contains all objects containing information about the Human Resources department
Schemas and name resolution
• When you request an object, SQL Server allows up to four “levels”: SELECT * FROM Server.Database.Schema.Object SELECT * FROM Jupiter.LearnItFirst.dbo.Customer
Schemas and name resolution
• If you are on the same server, you can skip the server name -- Logged on to Jupiter server: SELECT * FROM LearnItFirst.dbo.Customer
Schemas and name resolution
• If you are already in the correct database, you can skip the database -- Logged on to Jupiter server and using LearnItFirst:
SELECT * FROM dbo.Customer
Schemas and name resolution
• How would you query a table in the WebMarketing database while you were in the LearnItFirst database? _________________________________________
Schemas and name resolution
• How would you query a table in the WebMarketing database while you were in the LearnItFirst database? SELECT * FROM WebMarketing.dbo.Log
Schemas and name resolution
• If the default schema for your user account is the same as your desired object’s schema, you can omit the schema: -- Logged on to Jupiter server, in LearnItFirst db, and default schema is “dbo”: SELECT * FROM Customer -- Warning: not best practice
Schemas and name resolution
• We’ll cover more on schemas and name resolution later in the chapter
Schemas
• Let’s demo!
SQL Server 2008 Security: Roles
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles (covered already) – Database-level roles – Application Roles
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles (covered already) – Database-level roles – Application Roles
Database-level Roles
• Database-level roles allow DBAs to group users together and assign permissions to the role – Far superior to doing user-level permissions – Fixed database roles = the built-in roles – Flexible database roles = the ones you create
Database-level Roles
• You can use the built-in database roles as well as creating your own • Some commonly used built-in roles are: Role
Description
db_owner
Full rights to database
public
Any user is a member of public; sets base permissions for all connections
db_ddladmin
Create objects of any type; can alter or drop objects they created
db_datawriter
Can modify the data any table
db_datareader
Can read the data from any table
db_securityadmin
Manage role memberships and permissions
Database-level Roles
• Example of using roles: – Your database has 10 users – Your security assessment shows that there are really three types of permissions needed: • Full control (Micah and Thomas) • Development work (Sam and Nica) • Read-only access to all tables but modify to dbo.Customer table (all others)
Database-level Roles User
Permission
Micah, Thomas
Full control
Sam, Nica
Developers
All others
Read all tables; modify dbo.Customer
• Option 1: Assign each permission to each individual user – Pros: easy to understand when creating – Cons: repetitive, difficult to maintain, silly
Database-level Roles User
Permission
Micah, Thomas
Full control
Sam, Nica
Developers
All others
Read all tables; modify dbo.Customer
• Option 2: Create roles for each group – Pros: easy to understand when creating, easy to assign permissions, easy to add/remove – Cons: ?
Database-level Roles
Login
User
Role
Micah
Micah
public, db_owner
Thomas
Thomas
public, db_owner
Sam
Sam
public, db_ddladmin, Developers
Nica
Nica
public, db_ddladmin, Developers
Five other logins
Five users, one for each login
public, db_datareader, CustomerManagers
Roles and Schemas
• Notice that roles can be given the same permissions that users can
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Roles
Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Roles
• Roles are easy: same thing as a group! • There are three types of roles: – Fixed Server Roles – Database-level roles – Application Roles
Roles
• Application roles allow you to tie a specific application to certain permissions in the database – Step 1: User launches application – Step 2: Application logs into SQL Server – Step 3: Application uses a database – Step 4: App sets or invokes the application role • The application has now lost the user perms and now has the app role perms
Roles
• To use app roles, you still must have: – Login – User
• You can create app roles in SSMS or TSQL
Roles
Roles
• For an application to use the app role, it must know the password • The stored procedure sp_setapprole is how you invoke or set the app role • sp_unsetapprole can be used to restore the original security contexts
Roles
Roles
Roles
• Application roles are for specific situations – For stateful apps, not stateless apps like websites – Useful for kiosk-style apps in which users will not have their own logins – Useful for apps that bypass SQL Server users and create their own table-based security system – Most people will not ever encounter app roles
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
The GDR
• As we saw earlier, permissions are given using the GDR – GRANT – DENY – REVOKE
The SQL Server Security Model
• Deny takes precedence • User cannot perform an action unless they have been granted that permission • Users inherit permissions from roles • It only takes one DENY to prevent!
The SQL Server Security Model
• Micah is a user of the WebsiteMarketing db – public has DENY SELECT on dbo.Log – Micah belongs to db_datareader – Micah’s user account: • GRANT SELECT on dbo.Log • GRANT UPDATE on dbo.Customer
• What are Micah’s effective permissions?
The SQL Server Security Model
• Micah’s effective permissions are: – Read all tables except dbo.Log – Update dbo.Customer
• Micah is a user of the WebsiteMarketing db – public has DENY SELECT on dbo.Log – Micah belongs to db_datareader – Micah’s user account: • GRANT SELECT on dbo.Log • GRANT UPDATE on dbo.Customer
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
Server scoped permissions
• Server scoped permissions – Apply to the connection and are always “on” -- Give the login Chad sysadmin-level rights:
GRANT CONTROL SERVER to Chad -- Give Chrissy the option to change -- anyone’s password GRANT ALTER ANY LOGIN TO Chrissy
Server scoped permissions
Permission
Description
Administer bulk operations
Required to perform bulk operations
Alter any login
Change anyone’s password, et al
Control server
Sysadmin privileges
Create any database
Allows non-sysadmins to create databases
Shutdown
Uhhhhhhhhh
View any database
Can see metadata about every database; PUBLIC has this permission by default
View any definition
Discussed on next slide
View server state
Discussed on next slide
• Server roles often include these •
Adding a login to sysadmin grants “Control server”
Server scoped permissions
• In SQL Server 7.0 and SQL Server 2000, users were allowed to view the metadata about every object in the database -- Return metadata about every object in SQL 2000:
SELECT * FROM sysobjects
Server scoped permissions
• Starting with SQL Server 2005, you must enable permissions: -- Returns only objects that user has perms defined for:
SELECT * FROM sys.objects
Server scoped permissions
• If you need backward compatibility, then you need to GRANT server scoped permissions -- So that any user can view objects in db:
GRANT VIEW ANY DEFINITION to public -- Give users permissions to see health of server
GRANT VIEW SERVER STATE to public
Server scoped permissions
• Please! Do not issue these statements on a production server unless you know that need them! – These changes in security reflect changes to the SQL Server security model that were put in place for a reason
Roles
• When you use SSMS’ GUI to manage permissions, behind the scenes it will execute GDR statements to perform the requested action
Roles
• The easiest way to learn the Transact-SQL statements that DBAs use is to use the Script button to view the statements that SSMS will execute
Roles
• (We’ll do a full Transact SQL review later)
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: Database Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
Database scoped permissions
• Database scoped permissions – Apply to objects in the database only – Can assign permissions to: • Users • Roles • Schemas
Admin-related permissions
Permission
Description
Alter any application role
Change passwords
Alter any role
Create/alter/drop app roles, modify privileges
Alter any schema
Create/alter/drop schemas, modify perms, change object ownership
Alter any user
Create/alter/drop users, rename user; change default schema
Backup database
Can perform FULL and DIFFERENTIAL backups
Backup log
Can perform LOG backups
Control
Same permission as dbo
View database state
Required to view dynamic management views (DMVs) related to database state
Developer-related permissions
Permission
Description
Create assembly
Load new assemblies into database
Create function
Can create functions in any schema unless specifically denied
Create procedure
Can create stored procedures in any schema unless specifically denied
Create table
Can create tables in any schema unless specifically denied
Create view
Can create views in any schema unless specifically denied
Select
Same as placing in db_datareader
Insert, Update, Delete
Same as db_datawriter
Database scoped permissions
• Many user-level and flexible role permissions can be handled with the fixed database roles – “DENY Insert, Update, Delete…” is equivalent to adding user/role to db_denydatawriter role – Giving a user create function, view, proc, and table is the same as db_ddladmin
Database scoped permissions
• Roles can be nested – Step 1: Create DbAdmins flexible database role – Step 2: Make DbAdmins members of db_ddladmin
• The DbAdmins now have the same permissions as db_ddladmin – This may not make much sense yet…
Transact SQL
• Give the user Chrissy the ability to create functions -- Always make sure you are in the right db!
USE[LearnItFirst.com] GO GRANT CREATE FUNCTION TO [Chrissy]
Transact SQL
• Remove Chrissy’s ability to create functions REVOKE CREATE FUNCTION FROM [Chrissy] -- Chrissy now has no permissions
Transact SQL
• Prevent Chrissy from ever being able to create functions DENY CREATE FUNCTION TO [Chrissy]
Exercise
• What is the best way to give a user the ability to read all tables in the WebsiteMarketing db? _________________________________________ • Can you think of at least two other alternatives? _________________________________________
Exercise
• What is the best way to give all users the ability to read all tables in the WebsiteMarketing db? – Add user to db_datareader
• Can you think of at least two other alternatives? – Grant SELECT TABLE as database permission – Manually GRANT SELECT on each table in database – Grant SELECT on each table public role
Up next…
• Enough lecture – let’s demo!
SQL Server 2008 Security: Database Permissions Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Securables and Principals
• You will use the GDR to manage permissions at every level • Permissions have scope – Server – Database – Schema
What is a schema again?
• Schemas are containers – For convenience – Addictive
• Schemas are used in name resolution SELECT * FROM Server.Database.Schema.Object
Schema scoped permissions
• Schema-scoped permissions – Apply to objects in the schema only – Can assign permissions to: • Tables, views, procedures, functions and much more
• A schema is a database-level securable – Permissions on a schema in one database have no effect on a user’s permissions in another database
Schema level permissions
• If you have a schema-level permission, you have that permission for all objects contained within that schema Permission
Meaning
CONTROL
User has owner-like privileges on all objects in schema
ALTER
User may alter all objects in schema
INSERT, UPDATE, DELETE, SELECT
User may perform these operations (specified individually) on all objects in schema
VIEW DEFINITION
User may view definition (i.e. metadata) on all objects in schema
Roles and Schemas
• DBAs can secure objects at the schema level • DBAs can GRANT/DENY permissions on all objects in schema to users
Up next…
• Impersonation and EXECUTE AS • Demos
SQL Server 2008 Security: Impersonation and EXECUTE AS Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Impersonation and EXECUTE AS
• Before we get into deep demo mode, you need to understand impersonation at a high level – Impersonation allows you to change the execution context – This means you can execute statements as another login or another user without physically logging in as that principal
Impersonation and EXECUTE AS
• We use the EXECUTE AS command to perform impersonation – You can impersonate a login or a user – Requires the IMPERSONATE permission
• We use REVERT to switch back to our login
Impersonation and EXECUTE AS
• Sysadmins have impersonate permission for all logins and users – This means that the ‘sa’ can execute statements as any login or user
• Normal users cannot perform impersonation by default – You must grant them IMPERSONATE permission on a specific login or user
Granting login impersonation
-- We are logged in as „sa‟ GRANT IMPERSONATE ON Login::[LearnItFirst] TO [Chrissy] -- Chrissy is a login GO
Granting user impersonation
-- We are logged in as „sa‟ USE [LearnItFirst.com] GO GRANT IMPERSONATE ON User::[LIF] TO [Chrissy] -- Chrissy is a user GO
EXECUTE AS syntax { EXEC | EXECUTE ] AS
– NO REVERT – once activated, cannot be undone using REVERT
Impersonation and EXECUTE AS
• There are two types of context switching – Server-level (impersonating a login) • EXECUTE AS LOGIN = ‘NewLogin’
– Database-level (impersonating a user) • EXECUTE AS USER = ‘NewUser’
Impersonation and EXECUTE AS
• Server-level impersonation: – Once you switch, you are no longer you; you are the login you are now impersonating – Use REVERT to return to your original context – Can impersonate while impersonating! -- Useful T-SQL functions for server-level impersonation: SELECT SUSER_SNAME() AS CurrentLogin, ORIGINAL_LOGIN() AS OriginalLogin
Impersonation and EXECUTE AS
• Database-level impersonation: – Once you switch, you are no longer you; you are the user you are now impersonating – Use REVERT to return to your original context – Can also impersonate while impersonating
• One “gotcha”: once triggered, you cannot access objects from databases that do not have a guest account
Impersonation and EXECUTE AS -- Useful T-SQL functions for database-level impersonation: SELECT SUSER_SNAME() AS CurrentLogin, USER_NAME() AS CurrentUser
Impersonation and EXECUTE AS
• After we discuss object ownership, we will see there is another level of impersonation
Up next…
• Let’s do a few demos showing permissions, roles, users, and context switching!
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Ownership
• In SQL Server, we talk about database ownership, object ownership, and schema ownership – The owner has full rights
Database Ownership
• Database ownership: – The database owner is a login – The dbo user is the database owner – By placing users in the db_owner role, they have similar permissions • However, the dbo can DENY privileges to the members of the db_owner role
Cross database ownership chaining
• “Cross database ownership chaining” (‘CDOC’) allows database owners of one database to act as owners of other databases without having specific user access
Cross database ownership chaining
• “Cross database ownership chaining” is both a server setting and a database setting – Set it at the server-level to enable for entire instance – Set it at the database-level to allow specific databases to participate
Cross database ownership chaining
• “Cross database ownership chaining” is not recommended – Disabled by default – “… should not be enabled unless it is specifically required.” – Source: Books Online
Schema Ownership
• Schema ownership is extremely important – Remember that all objects in a database are contained within a schema – The schema owner has full rights on all objects in that schema
Object Ownership
• Object ownership: – You can grant CONTROL on certain objects within a schema instead of granting permission on the entire schema
Object Ownership
• Transferring ownership is easy – Can transfer schema ownership – Can transfer object ownership
• Can use Transact-SQL (recommended) or the SSMS GUI screens (lots of clicks!)
Object Ownership
• Example #1 – Terry owns the “Sales” schema – Terry leaves the company and we hire Chad to replace him – We transfer ownership of the “Sales” schema to Chad
Object Ownership
• Example #2 – Chrissy owns a table in the “Sales” schema named “Prospects” – Chad owns the schema – We hire Candy to replace Chrissy – We grant CONTROL on “Sales.Prospects” to Candy – We remove Chrissy!
In the next videos…
• Examples and scenarios • Ownership chains • How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• Let’s walk through a few examples on object ownership first – Remember that the owner has full rights
Example #1
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Can Chad SELECT from dbo.Merchant? – ___________________
• Can Chad execute dbo.AddMerchant? – ___________________
• Why is this? – ___________________
Example #1
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Can Chad SELECT from dbo.Merchant? – Yes!
• Can Chad execute dbo.AddMerchant? – Yes!
• Why is this? – He is the dbo and has full rights to the database
Example #2
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #2
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: <none>
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – No
• Why or why not? – Her effective permission does not contain a GRANT
Example #3
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #3
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy is a user with no permissions • Can Chrissy SELECT dbo.Merchant? – Yes
• Why or why not? – Her effective permission contains a GRANT with no DENY
Example #4
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: member of db_datareader • Chrissy: DENY SELECT ON SCHEMA::[dbo] • Can Chrissy SELECT dbo.Merchant? – ___________________
• Why or why not? – ___________________
Example #4
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: member of db_datareader • Chrissy: DENY SELECT ON SCHEMA::[dbo] • Can Chrissy SELECT dbo.Merchant? – No
• Why or why not? – Her effective permission contains a DENY
Object Ownership
• Clear cut examples like the previous ones make things seem easy • What happens when there are multiple objects and multiple permissions?
Example #5
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: DENY SELECT ON dbo.Merchant • Chrissy: GRANT SELECT ON dbo.ViewMerchants • Can Chrissy SELECT dbo.ViewMerchants? – ___________________
Example #5
dbo schema (owned by dbo, the user): • Merchant (table) • AddMerchant (proc) • ViewMerchants (view) WebsiteMarketing dbo: Chad
Permissions: • public: SELECT on dbo.Merchant
• Chrissy: DENY SELECT ON dbo.Merchant • Chrissy: GRANT SELECT ON dbo.ViewMerchants • Can Chrissy SELECT dbo.ViewMerchants? – Yes, and she can see all of dbo.Merchant!
Object Ownership
• When multiple objects access each other sequentially, this is called a chain
Object Ownership
• When the owner of all objects in the ownership chain is the same, permission is only checked at the entry point of the chain – Example: if the dbo owns all objects in the chain
Ownership Chain Demo
Object Ownership
• This makes life easy for the DBA: you only need to concern yourself with whether your user/role has permissions at the entry point – Most real-world database work is done on objects all owned by the same user
In the next videos…
• Ownership chains • How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• When there are different owners in the chain, SQL Server checks the permission at each change in ownership – This is called ownership chaining – It’s a hard life dealing with databases with multiple schema owners that also contain objects controlled by multiple users
Ownership Chain Demo
Example #1
• The dbo creates a view, Admin.GetMerchants, that references dbo.Merchant
• The dbo grants Chrissy SELECT on the view • When Chrissy selects from the view, is ownership chaining in effect? – No, the same owner owns all objects!
Object Ownership
• It is not the schema name that causes ownership chaining – It is the fact that the schemas are owned by different users
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Can Chrissy create this view despite not having the SELECT permission?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Since Chrissy cannot select but can create the view, can she select from her own view and get the data in that way?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Since Chrissy cannot select but can create the view, can she select from her own view and get the data in that way? • No! She must be given SELECT privilege on dbo.Merchant
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view?
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view? • No! But for a different reason…
Object Ownership
• Remember: SQL Server checks permissions at each change in ownership • Terry must have: – SELECT permission on Admin.GetMerchants – SELECT permission on dbo.Merchant
• But there is another way……………………….
In the next videos…
• How impersonation can affect permissions • How to transfer ownership of databases and schemas
SQL Server 2008 Security: Ownership Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Object Ownership
• This is hard stuff… Follow me here: – Chrissy: • Owns Admin schema • Has CREATE VIEW privilege • Creates a view, Admin.GetMerchants, based on dbo.Merchant
– Chrissy GRANTs Terry SELECT permission on her view. Terry has no permissions on dbo.Merchant. Can he SELECT from the view?
Object Ownership
• Remember: SQL Server checks permissions at each change in ownership • Terry must have: – SELECT permission on Admin.GetMerchants – SELECT permission on dbo.Merchant
• But there is another way……………………….
Impersonation in Modules
• The creator of the module (i.e. procedure, view, function, trigger) can specify the execution context – Just because you can doesn’t mean you should…
Example #1: • Chrissy: – Owns Admin schema – Has CREATE VIEW privilege – Creates a view, Admin.GetMerchants, based on dbo.Merchant
CREATE VIEW Admin.GetMerchants AS SELECT * FROM dbo.Merchant GO
Impersonation in Modules
• This example actually runs under the execution context EXECUTE AS CALLER: – EXECUTE AS CALLER says that SQL Server should check permissions at each change in ownership (the default)
Impersonation in Modules
• There are actually four options for switching execution context in modules: – EXECUTE AS CALLER (the default) – EXECUTE AS ‘user_name’ – EXECUTE AS SELF – EXECUTE AS OWNER
Impersonation in Modules
• EXECUTE AS ‘user_name’ – All access to other tables and modules in this module is done via the security context of ‘user_name’ – Creator of module must have IMPERSONATE ‘user_name’
Impersonation in Modules
• EXECUTE AS SELF – All access to other tables and modules in this module is done via the security context of the creator of the module
Impersonation in Modules
• EXECUTE AS OWNER – All access to other tables and modules in this module is done via the security context of the current owner of the module
Example #1: •
Chrissy: – – – –
Owns Admin schema Has CREATE VIEW privilege Creates a view, Admin.GetMerchants, based on dbo.Merchant Has IMPERSONATE „Terry‟
CREATE VIEW Admin.GetMerchants EXECUTE AS ‘Terry’ AS SELECT * FROM dbo.Merchant GO -- Terry must have SELECT on -- dbo.Merchant for any user -- (except dbo) to get data
How to transfer objects…
• The final piece here is knowing how to transfer objects • You can transfer: – Databases to new logins – Schemas and modules to users or roles
To Transfer a Database
• DBAs often create databases and then transfer them to another login – We are stingy about giving out GRANT CREATE DATABASE permissions…
To Transfer a Database
• To transfer a database, use the sp_changedbowner stored procedure – The login must not be an existing user in the database prior to transfer - why? -- Make sure you are in the right DB! USE [LearnItFirst.com] GO EXEC sp_changedbowner „Chrissy‟ -- The login „Chrissy‟ is now the dbo!
To Transfer an Object
• Transferring object ownership is really transferring the object to a new schema or changing the schema’s owner – Generally you want to change the schema’s owner in the real world
To Transfer an Object
• To transfer ownership of a schema: – Generally you want to change the schema’s owner in the real world
To Transfer an Object
• To transfer ownership of a schema in Transact-SQL: ALTER AUTHORIZATION ON SCHEMA::[Customer] TO [Terry]
To Transfer an Object
• If you absolutely, positively have to change the schema of an object (table, view, etc), it is easiest to do so in T-SQL – You can do it in SSMS’ GUI but sheesh! ALTER SCHEMA [NewSchema] TRANSFER „OldSchema.MyTable‟
SQL Server 2008 Security: Encryption Overview Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Encryption Overview
• SQL Server 2008 has native encryption built in – Connections can be encrypted – Source code can be encrypted – Data can be encrypted
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
• Let’s talk about each!
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encrypting Connections to SQL Server
• SQL Server can use 40 or 128-bit Secure Sockets Layer (“SSL”) to encrypt network packets – Slower performance but higher security – Requires a certificate to be installed on the machine that hosts the SQL Server – Best practice is to use a trusted certificate • Self-signed certificates can be thwarted
Encrypting Connections to SQL Server
– Step 1: Install certificate – Step 2: In the SQL Server Configuration Manager, right click on Protocols for
Encrypting Connections to SQL Server
– Step 3: Choose your certificate in the Certificates tab
Encrypting Connections to SQL Server
– Step 4: Choose whether or not encryption is required • The Force Encryption option: – If “No”, encryption is optional – If “Yes”, clients who are not encrypted will not be allowed to connect
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encrypting Source Code
• Source code can also be encrypted – However, whether it can be decrypted or not is another matter – Always have a separate copy of your encrypted source code
CREATE PROC MyProc WITH ENCRYPTION AS PRINT „You cannot see the source code now!‟ GO
Encrypting Data
• When we talk about encryption in SQL Server, we are generally discussing the encryption of actual data • There are many ways to encrypt data inside your SQL Server – Pass in data that is already encrypted – Have SQL Server handle the encryption for you
Encrypting Data
• In the next video, we’ll cover the types and techniques of native SQL Server encryption
SQL Server 2008 Security: Encryption Overview Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Encryption Overview
• DBAs need to know about: – How to encrypt connections – How to enable encryption – How to backup and restore the encryption mechanisms (i.e. the encryption ‘keys’)
Encryption
• Encryption can be done at the databaselevel or at the column-level (or cell-level) • Be aware that encryption will result in poorer performing queries
Encryption
• If you don’t have Enterprise Edition… – You must perform column-level encryption manually using functions like: • • • •
EncryptByPassphrase() EncryptByCert() EncryptByAsymKey() EncryptByKey()
– Column-level encryption is covered in detail in out SQL Server 2008 Transact-SQL programming class • Course 161
Encryption
• If you do have Enterprise Edition, then you can use Transparent Data Encryption (“TDE”) – Entire database is encrypted – Requires no coding changes by developers – Encryption/decryption is done automatically and cannot be bypassed – Performance hit of “3-5%” – Source: Microsoft article, “Database Encryption in SQL Server 2008” via http://learnitfirst.com/go.aspx?id=J4EK
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
• Let’s look at each step
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 1: Create a master key in the master database – A master key is a symmetric key used to protect the private keys of certificates and asymmetric keys – There is no “default” master key so you must either create it or use an existing key if already present
Encryption
• Step 1: Create a master key in the master database (cont.) USE master GO CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password„ -- Encrypted using Triple DES
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
• Let’s look at each step
Encryption
• Step 2: Create a certificate based on the master key – Required to create Step 3’s database encryption key
USE master GO CREATE CERTIFICATE LearnItFirstCert WITH SUBJECT = „Transparent Data Encryption Cert‟
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 3: Create a database encryption key – Based on the certificate from Step 2 – Can use AES, DES, Triple DES, RC4, and more USE [LearnItFirst.com] -- The db to protect GO CREATE DATABASE ENCRYPTION KEY WITH ALGORITHM = AES_256 ENCRYPTION BY SERVER CERTIFICATE LearnItFirstCert -- Cert from Step 2
Encryption
• To enable TDE: – Step 1: Create a master key in the master database – Step 2: Create a certificate based on the master key – Step 3: Create a database encryption key – Step 4: Turn it on!
Encryption
• Step 4: Turn it on! USE [LearnItFirst.com] -- The db to protect GO ALTER DATABASE [LearnItFirst.com] SET ENCRYPTION ON GO
TDE
• A few tips for working with TDE: – Backup master and your DB before beginning – Configure on multiple test servers and databases until you are very comfortable before rolling into production – TDE requires time to do the encryption so schedule over a weekend or during planned downtime – Once you enable TDE, tempdb will be encrypted as well • This lowers the performance of SQL Server for non-TDE databases
TDE
• Backup and Restore – Take backups of your service master key, your master key and certificate(s) at the same time you turn on TDE – Store these backups in a separate location from your database backup – Your database cannot be attached or restored without these!
TDE
• Backing up TDE databases requires more… – The service master key will be covered later… BACKUP SERVICE MASTER KEY TO FILE = „F:\SMK_Timestamp.bak„ ENCRYPTION BY PASSWORD = „p@55w0rD!'
TDE
• Backing up TDE databases requires more… BACKUP MASTER KEY TO FILE = „F:\MasterKey.bak„ ENCRYPTION BY PASSWORD = „n3Wp@55!‟
TDE
• Backing up TDE databases requires more… BACKUP CERTIFICATE LearnItFirstCert TO FILE = „F:\LearnItFirstCert.bak‟
TDE
• Restoring TDE databases is exciting! – Make sure you take your heart medicine first… – Step 1: Restore Service Master Key (if needed) – Step 2: Restore Master Key (if needed) – Step 3: Re-create the certificate (if needed) – Step 4: Restore the database
TDE
• We will cover restoring service master keys in chapter 6 • Once all keys are restored and the certificate is created, just restore the database – No need to go through any hoops if restoring to the same server (all keys and certs are present!)
SQL Server 2008 Security: Linked Servers and Distributed Queries Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Distributed Queries Overview
• You are able to query additional data sources using Transact-SQL: – Linked servers are pre-configured for granular security and repeated use – OPENQUERY(), OPENROWSET() and OPENDATASOURCE() allow you to specify data source at run-time
• These are called distributed queries
Distributed Queries Overview
• Distributed queries often involve multiple data source types – You can JOIN a SQL Server table to an Oracle table, for example
Linked Servers
• Linked Servers: – Allow you to write easy Transact-SQL statements against multiple data sources without having to enter login credentials with each query – Login credentials are stored with the linked server’s definition – Used whenever you need to repeatedly access an external data source from within SQL Server and do not want to have to enter credentials each time
Linked Servers
• Linked Servers are configured by the DBA – In SSMS, go to Server Objects
Linked Servers
• Let’s view a demo!
SQL Server 2008 Security: Linked Servers and Distributed Queries Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
Pass-through queries
• Pass-through queries are queries whose parsing is done completely by the remote data source – Are written in the language of the underlying data source – The SQL Server query parser does not even parse the statement
OPENQUERY()
• OPENQUERY() allows us to issue passthrough queries against linked servers – Anyone can execute – Permission is checked in the linked server – Can issue any SELECT, INSERT, UPDATE and DELETE
OPENQUERY()
-- If we have a linked server named -- JUPITER to a SQL Server 2008 -- instance SELECT * FROM OPENQUERY(Jupiter, „SELECT FirstName, LastName FROM dbo.Customer‟)
OPENQUERY()
-- OPENQUERY is not necessary here -- This is not a pass-through query: SELECT FirstName, LastName FROM Jupiter.TheDb.dbo.Customer -- Notice the four-part address -- Required for linked servers
OPENQUERY()
• OPENQUERY is generally used with nonSQL Server-based linked servers – Like Microsoft Access or Excel, Oracle, DB2 – For SQL Server-based linked servers, we generally just use “regular old four-part queries” SELECT FirstName, LastName FROM Jupiter.TheDb.dbo.Customer
OPENDATASOURCE() and OPENROWSET()
• OPENDATASOURCE() and OPENROWSET() are functions that allow ad-hoc queries to remote data sources – No linked servers – Require the server-level option “Ad Hoc Distributed Queries” to be enabled EXEC sp_configure „Ad Hoc Distributed Queries‟, 1 RECONFIGURE WITH OVERRIDE GO
OPENROWSET()
• OPENROWSET() is extremely flexible – Can bulk load data from text files – Can load data from Oracle, Access, et al -- Example query to MS Access database SELECT * FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0', 'C:\Program Files\Microsoft Office\OFFICE11\SAMPLES\Northwind.mdb';'admin ';'', Orders) AS x
OPENROWSET()
• Example showing OPENROWSET() -- Example query to a .txt file SELECT * FROM OPENROWSET(BULK 'c:\test\values.txt„)
OPENDATASOURCE()
• OPENDATASOURCE() allows four-part addressing without using a linked server – Not that common
OPENDATASOURCE () -- Example query to MS Access database SELECT * FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0', 'C:\Program Files\Microsoft Office\OFFICE11\SAMPLES\Northwind.mdb';'admin ';'', Orders)...[Customers] -- Notice the “...”
-- Example query to SQL Server instance SELECT * FROM OPENDATASOURCE('SQLNCLI', 'Data Source=London\Payroll;Integrated Security=SSPI') .AdventureWorks.HumanResources.Employee
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In the next video
• Encryption overview • Linked servers • SIDs and User accounts
SQL Server 2008 Security: SIDs and Users - What You Need to Know Presented by Scott Whigham
• Database principals • Schemas • Roles • Assigning Database permissions
More….
• Chapter Overview • Reducing the attack surface area • An Overview of SQL Server Security Model • Logins and Permissions
Database-level Security
Server-level Security
In this video
• Encryption overview • Linked servers • SIDs and User accounts
What is a SID?
• SID is an acronym for Security IDentifier • When you create a login, it is given a SID: – Windows-based login SIDs are the same as the Windows account/group SID – SQL Server login SIDs are unique values for that instance
Example
• SIDs are important: – What happens if we want to restore a database on another server? Let’s look at a scenario…
Example
• Your production server has a SQL Server authentication login, “BigLogin” and a database, “OurDb” – BigLogin can access the OurDb database using BigLogin’s user account, “BigUser” – Your application is configured to use the BigLogin login to access SQL Server
Example
• You want to create an offsite copy of the database on another server – You take a backup of OurDb and restore it onto your new SQL Server – You configure your application to use this new server
• Will your application be able to connect?
Example
• No! – You have not created the BigLogin login on the new server
• Lesson learned… So you go create the BigLogin SQL Server authentication login on the new server • Will your application be able to connect?
Example
• Maybe! – It depends on whether or not you have set the default database for the connection string to be OurDb • If you set it, you cannot log in • If you did not set it (unlikely), then you can log in
• Right now, the login on the new server cannot access OurDb!
Example
• Here’s the key take away from this video: – When you create a SQL Server authentication login, it creates it with a random SID by default
Example CREATE LOGIN Scott WITH PASSWORD='p@55w0rD!' GO SELECT sid FROM sys.server_principals WHERE name = 'Scott' GO DROP LOGIN Scott GO CREATE LOGIN Scott WITH PASSWORD='p@55w0rD!' GO SELECT sid FROM sys.server_principals WHERE name = 'Scott„
Example
• When you create a SQL Server authentication login, it creates it with a random SID by default
Example
• So back to our scenario… – When you created the new login, it created it with a random SID – Main point: The user account in the OurDb database on the new server is not associated with your new login!
• Why not?
Example
• Because the SIDs for the login BigLogin are different on each server! • Enough for now… We’ll revisit this in the next chapter and show examples and how to resolve!
Example
• Speaking of the next chapter… • Up next: Chapter 6: Backup and Recovery
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Chapter Overview
• Lack of backups can break a company • As the DBA, it is your responsibility to create backup and recovery plans and test those plans – Perform these tests early and often for any critical project
Chapter Overview
• This chapter will teach you how to: – Develop backup plans – Understand the risks associated with any single plan – Take and restore backups
Chapter Overview
• This chapter will not teach you how to: – Implement Database Mirroring, Log Shipping, Replication, or Snapshots – Discuss creating or managing jobs and logs
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
Next up…
• Restoring User Databases • Restoring System Databases
Scenario Setup
• Before we get too deep, we need a real world example to use • For the scenarios in this chapter, you are the database administrator for LearnItFirst.com
Scenario Setup
• LIFWeb: Manages online website – Small-to-medium size (100GB) – Real-time data and transactions – Few bulk transactions (indexing only) – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFWeb: Manages online website – Real-time data and transactions – Few bulk transactions (indexing only) – Backup and restore goals: • Be able to restore quickly • Ensure no lost transactions • Be able to recover to “point of failure” and “point in time”
Scenario Setup
• LIFUtility: Internal support – Very small (100MB) – Static data (numbers tables, functions, procs) – No bulk transactions – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFUtility: Internal support – Very small (100MB) – Static data (numbers tables, functions, procs) – No bulk transactions – Backup and restore goals: • Backup/restore full database only • Point in time recovery is not important
Scenario Setup
• LIFReporting: OLAP database to feed SQL Server Analysis Services – Large (400GB) – Mainly bulk transactions loaded on weekends • indexing, text files, XML
– Not Mission Critical – Backup and restore goals: • ___________________________ • ___________________________
Scenario Setup
• LIFReporting: OLAP database to feed SQL Server Analysis Services – Large (400GB) – Mainly bulk transactions loaded on weekends • indexing, text files, XML
– Not Mission Critical – Backup and restore goals: • Minimize backup size • No need for point in time or point of failure recovery
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
Coming up…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Types of SQL Server Backups
• The first thing to know is what you can back up – Entire database (all files - data and log) • Called a “Full” backup
– Log only • Called a “Transaction Log” backup
– Only the data that has changed since the last Full backup • Called a “Differential” backup
– Filegroups
Dispelling Myths
• Dispelling myths – You do not need to take a database offline to back it up – Backups will include open transactions – All backups include the transaction log – You cannot just backup a table by default • “All or nothing” • One exception: if that table is on a separate filegroup
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
What We’re Going to Cover
• Restoring User Databases • Restoring System Databases
Recovery Models
• The next thing to know is what Recovery Models are • A database’s recovery model determines – What you can backup (and subsequently restore) – How the backup process works – How often the transaction log is cleared out
Recovery Models
• There are three types of recovery models – Simple – Full – Bulk Logged
• Each has specific scenarios for use • Can switch between them instantly
Recovery Models
Simple Can backup transaction log? Transaction log is “cleared out” automatically? Transaction log grows until manually cleared out? Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
Bulk
Full
Simple Recovery Model
• The Simple recovery model: – Automatically clears committed transactions from the log • By default, this happens about 1x per minute • Prevents the log from filling up and auto-growing
– Cannot backup the transaction log directly
Simple Recovery Model
• The Simple recovery model is appropriate when: – You do not need “Point in time” recovery – You want to minimize disk space – The default for the master, msdb, and tempdb – Perfect for static, test and development databases
Scenario
• Which of our databases is appropriate for using the Simple recovery model?
Scenario
• Which of our databases is appropriate for using the Simple recovery model? – LIFUtility because it is static and we do not need point in time recovery
Full Recovery Model
• The Full recovery model: – Allows you to restore to a specific point in time or to the point of failure • “I need to restore everything up until 12:15:35 - that’s when we know that Scott ran that nasty UPDATE statement that messed everything up!”
– For this reason, most production databases are Full recovery databases • Least risk of losing work!
Full Recovery Model
• The reason that you can restore to a point in time is that you can restore the log – You can only restore to point in time or point of failure from a transaction log backup, not from a Full or Differential
Scenario
• Which of our databases is appropriate for using the Full recovery model?
Scenario
• Which of our databases is appropriate for using the Full recovery model? – LIFWeb because it changes constantly and we do not want to lose any work
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Logged Recovery Model
• The Bulk Logged recovery model: – Is identical to Full recovery model except when dealing with bulk operations
Recovery Models
Simple Can backup transaction log? Transaction log is “cleared out” automatically? Transaction log grows until manually cleared out? Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
Bulk
Full
Bulk Operations in SQL Server 2008
• Certain statements and tools are considered bulk operations – CREATE INDEX and ALTER INDEX – SELECT INTO – BULK INSERT – OPENROWSET() – bcp
Bulk Operations in SQL Server 2008
• Bulk operations can be – Fully logged - transaction is logged first and then written to the data file – Minimally logged - minimal information is written to the log first and then entire transaction is written directly to data file
Bulk Operations in SQL Server 2008
• In the Full recovery model, ALL transactions are fully logged – This means that your bulk imports/loads/operations are incurring potentially unnecessary overhead
Bulk Operations in SQL Server 2008
• In the Bulk Logged recovery model, bulk operations are minimally logged – No performance overhead associated with logging means faster performance
• So if Bulk Logged is faster for bulk operations, why don’t we always use it?
Bulk Logged Recovery Model
• The Bulk Logged recovery model, while optimized for bulk operations, has a serious “gotcha” Simple
Bulk
Backs up in-progress transactions? Backs up bulk transactions? Completely logs bulk transactions? Supports point in time recovery?
* *
Full
The “Gotcha”
• You cannot backup the log if: 1. You are using Bulk Logged recovery model 2. … and the data file is missing 3. … and bulk operations have occurred since the last backup
Bulk Logged Recovery Model
• Let me recap: – Bulk Logged and Full recovery models are identical provided no bulk operations have occurred since the last backup – Bulk Logged recovery model may prevent you from backing up your log
Bulk Logged Recovery Model
• Why? You need to know the architecture of the transaction log! – Let’s do that in the next video…
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video…
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
The Transaction Log, Part 1
• SQL Server uses “write ahead logging”
The Transaction Log, Part 1
• SQL Server allocates space an extent at a time – An extent is a collection of eight 8kb pages
The Transaction Log, Part 1
• When you have minimally-logged transactions, the only thing written to the log is the extents that were affected – Detailed row information is not written
The Transaction Log, Part 1
• Demo of a minimally logged transaction
Bulk Logged Recovery Model
• Name two advantages of using the Bulk Logged recovery model – _________________________ – _________________________
Bulk Logged Recovery Model
• Name two advantages of using the Bulk Logged recovery model – Faster bulk operations/loads – Minimize log space – Same protection as Full recovery model unless bulk operations have occurred
The Transaction Log, Part 1
• However… you do have to pay the price for the upload at some point – Detailed row information is not written during writes to the log thus minimizing overhead and log space – Detailed row information is written at log backup! • Expect your transaction log backups after a bulk operation to be large and time-consuming
Scenario
• Which of our databases is appropriate for using the Bulk Logged recovery model?
Scenario
• Which of our databases is appropriate for using the Bulk Logged recovery model? – LIFReporting because is loaded primarily using bulk operations – However, we likely will use Full recovery model most of the time and switch to Bulk Logged when using bulk operations • Remember that we can switch between them as needed
Switching Between Recovery Models
• You can switch between recovery models as needed – It is common to use FULL as your production model and switch to BULK LOGGED only during bulk operations – If you switch to SIMPLE, backup the log first – If you switch from SIMPLE, requires FULL or DIFF to actually take effect
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In the next video… Demo!
• Restoring User Databases • Restoring System Databases
SQL Server Backup and Recovery
Presented by Scott Whigham
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Scenarios
• Let’s walk through a few backup strategies using each recovery model • One caveat: we are not discussing the “how to” or the “why” of the restore yet
The Basics of Restoring Databases
• However you need to know a few basics: – You must always start with a FULL backup – You may only restore a single DIFFERENTIAL • Remember that a DIFFERENTIAL contains all changes since the last FULL backup • You do not need to restore any log backups taken before your DIFFERENTIAL
The Basics of Restoring Databases
• Basics of restoring LOG backups: – Logs must be restored in sequence – You cannot skip logs during the restore – You cannot skip transactions during a restore – It is possible to restore only part of the last log file you restore • Achieves “point in time” and/or “point of failure” recovery
The Basics of Restoring Databases
• Basics of restores: – The final backup you restore will, by default, take the database through the RECOVERY process and bring the database ONLINE • You decide which is the final backup file in your Transact-SQL code
– Any open transactions in the database will be rolled back during the RECOVERY process
The Basics of Restoring Databases
• One final wrinkle: – In the FULL and BULK LOGGED recovery model, you can back up the transaction log even if the data file is corrupt/suspect/missing • The only exception is when using BULK LOGGED recovery model and a bulk operation has occurred since the last backup
– This is how to do “point of failure” recovery!
The Basics of Restoring Databases
• Reminder: we will discuss all of this in detail later in the chapter
The Basics of Restoring Databases
• Example restore: – FULL backup: from Monday: 0800 – LOG backups: Monday: 0900, 1000, 1100 – DIFF backup: Monday 1200 – LOG backups: Monday: 1300, 1400, 1500
The Basics of Restoring Databases
• Example restore (cont.): – At 1430, user “Steve” notifies you that a recently applied application hotfix has “messed up the whole database” • You: “Okay, Steve - when was the hotfix applied?” • Steve: “We’re not exactly sure - sometime around 12:30”
– You and your team decide to restore up through 1220 (i.e. “point in time” restore)
The Basics of Restoring Databases
• Example restore (cont.): – What is your strategy to restore up through 1220? • Step 1: _____________________________ • Step 2: _____________________________ • Step 3: _____________________________
The Basics of Restoring Databases
• Example restore (cont.): – What is your strategy to restore up through 1220? • Step 1: Restore the most recent FULL • Step 2: Since the DIFF occurred before the stop time of 1220, restore the most recent DIFFERENTIAL • Step 3: Restore the 1300 LOG backup but tell SQL Server to STOP AT 1220
The Basics of Restoring Databases
• Example restore (cont.): – Since your last transaction in the restored database occurred at 1220, what happened to all transactions after 1220? ___________________ – What happened to any open transactions that were open as of 1220? _____________________
The Basics of Restoring Databases
• Example restore (cont.): – Since your last transaction in the restored database occurred at 1220, what happened to all transactions after 1220? Never occurred so “lost” – What happened to any open transactions that were open as of 1220? Rolled back in RECOVERY
Confused?
• It’s okay if you did not fully understand the scenario • We will thoroughly review restores including how to STOP AT a specific time later in this chapter
SQL Server Backup and Recovery
Presented by Scott Whigham
73
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Determining your backup strategy
• Let’s walk through backup strategies and restore scenarios • How do we determine the optimal backup strategy? What factors do we have to consider? ________________________________ ________________________________ ________________________________ ________________________________ ________________________________ ________________________________
Determining your backup strategy
• Factors to consider: 1. 2. 3. 4. 5. 6. 7.
Size of database Time it takes for each backup type to complete How much “drag” backups place on the system How much disk space our backups consume Where to put the backups? How much “loss” is acceptable How much downtime is acceptable
Factors to consider
1. Size of database – Large databases are problematic: • Backup time can be hours and hours – Therefore the restore time can be hours and hours
• Backup file sizes are huge – In SQL Server 2008 Enterprise Edition, backup files can be compressed (more on this later)
• Since the backups are so massive, they have an impact on system performance
– Smaller and mid-size databases do not have as much to worry about
Factors to consider
2. Time it takes for each backup type to complete – Some sites have downtime - they work business hours and therefore full system backups can occur during “off” hours with no impact – Many sites require maintenance windows to perform full system backups • Large database backups often do not “fit” your window!
Factors to consider
3. How much “drag” backups place on the system – A rule of thumb for most scenarios/databases: • FULL backups often have the most performance impact • DIFFERENTIAL backups have the next-most performance impact • LOG files, if taken often, will have the least impact
– This does not cover all situations
Factors to consider
4. How much disk space our backups consume – By default, SQL Server will do no compression on backups • Large databases means large backups • A SQL Server backup only contains the data, not the “empty” pages – If you have a 100GB database with only 10GB of data, your FULL backup will be ~10GB
– SQL Server 2008 Enterprise has the option to compress your backups (more on this later)
Factors to consider
5. Where to put the backups – You can backup to: • Disk - local and remote • Tape - local only
– Disks are fast and tapes are slow – Often the strategy is: • • • •
Backup to local or network disk Copy disk copy to tape Move/rotate tape offsite Delete disk backup according to a schedule
Factors to consider
5. Where to put the backups (cont.) – Backing up across a network often adds overhead • If time is a problem, backup local and do a network file copy later
– Do not store backups on same disks as data or log files • For performance reasons • For recovery reasons (more on this later)
Factors to consider
6. How much “loss” is acceptable – Strange factor, eh? – All backup strategies want to minimize data loss • Impossible to completely prevent the possibility that data loss can occur • Saying that “Data loss is impossible” is akin to saying, “My network is so secure no could ever hack into it”
– One technique for minimizing data loss is taking frequent backups • Some sites take LOG backups every 10 minutes
Factors to consider
6. How much “loss” is acceptable (cont.) – Backups are not the only strategy in play – High Availability solutions such as Database Mirroring, Log Shipping, and Replication are options we will discuss in a future chapter
Factors to consider
7. How much downtime is acceptable – All backup strategies want to minimize downtime • Just like with data loss, it is impossible to completely prevent the possibility that downtime can occur
– High availability solutions (including clustering) offer protection against downtime
Factors to consider
7.
How much downtime is acceptable (cont.) – If you do have to completely restore a database, “It takes as long as it takes” • A 1TB FULL backup is going to take a long time to restore…
– There are strategies that minimize restore time: • Frequent DIFFERENTIAL backups minimize the number of LOG backups that have to be restored, for example • Certain software packages minimize backup and restore times • SQL Server 2008 Enterprise Edition includes a feature called Fast Recovery
Factors to consider
7. How much downtime is acceptable (cont.) – SQL Server 2008 Enterprise Edition includes a feature called Fast Recovery • This allows users to access the database before the restore has rolled back uncommitted transactions (i.e. before the Undo phase of the restore has completed)
Determining your backup strategy
• The challenge is: “How do we create a backup strategy that protects us from data loss while also ensuring minimal downtime during a restore?”
Determining your backup strategy
• For a great article on specifically how the restore process works, visit http://msdn.microsoft.com/library/ms191455.aspx
• Article: “Understanding How Restore and Backups Work in SQL Server”
SQL Server Backup and Recovery
Presented by Scott Whigham
90
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Scenario Reminder
• Remember our three databases: – LIFWeb: Manages online website • Small-to-medium size (100GB) • Full recovery model
– LIFUtility: Internal support • Very small (100MB) • Simple recovery model
– LIFReporting: OLAP database to feed SQL Server Analysis Services • Large (400GB) • Full recovery model and switch to Bulk Logged when using bulk operations
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• If a problem occurs on Friday at 1200, can you back up the log so that you can achieve “point of failure” recovery? Explain your answer – ______________________________________________
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• If a problem occurs on Friday at 1200, can you back up the log so that you can achieve “point of failure” recovery? Explain your answer – No - Simple recovery model does not allow log backups
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• Since you cannot backup the log, what is your restore sequence? – ______________________________________________
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 Database: LIFUtility Recovery model: SIMPLE
• Since you cannot backup the log, what is your restore sequence? – Restore the FULL backup from Saturday – Any data added or modified after Saturday night’s backup will be lost
Backup Strategy
• Are there any situations in which this is acceptable? – Read-only databases – Databases whose updates occur on Saturday mornings only – Test databases – “Junk” databases
Backup Strategy
• What are some strategies to employ if that much potential data loss is not acceptable? – More frequent backups – Changing the recovery model – Adding high availability options
Backup Strategy
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
Example #3
Backup Strategy:
Database: LIFUtility Recovery model: SIMPLE
• Full backups on Saturdays at 2300 • Differential backup Wednesday at 2300
• How does adding in differential backups each night change your restore sequence? – ______________________________________________
Example #3
Backup Strategy:
Database: LIFUtility Recovery model: SIMPLE
• Full backups on Saturdays at 2300 • Differential backup Wednesday at 2300
• How does adding in differential backups each night change your restore sequence? – Step 1: Restore the FULL from Saturday – Step 2: Restore Wednesday night’s DIFFERENTIAL – Any data added or modified after Wednesday night’s backup will be lost
In the next video…
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
SQL Server Backup and Recovery
Presented by Scott Whigham
103
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Backup Strategy
• What are some other strategies to employ if that much potential data loss is not acceptable? (cont.) – More frequent backups – Changing the recovery model – Adding high availability options
Simple Recovery Model
• Simple recovery model has a huge “hole”: – You cannot perform a “point of failure” restore – Why? • You cannot backup the transaction log • You cannot restore only part of a FULL or DIFFERENTIAL • You can restore only part of a LOG
Crash Procedure for Simple Recovery Model
• Your restore sequence in Simple recovery model is: • Step 1: Restore your most recent FULL backup • Step 2: Restore your most recent DIFFERENTIAL (if available) • Any changes after this will be lost
Simple Recovery Model
• Let’s take a look at what the Pros and Cons are when we change from Simple to Full recovery model
Example #1
Backup Strategy:
Database: LIFUtility Recovery model: FULL
• Full backups on Saturdays at 2300 • Differential backups each day at 2300
• In this scenario, a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. How does changing the recovery model change your restore sequence? – ______________________________________________
Full Recovery Model
• Remember that the full recovery model allows you to backup your transaction log – This is critical to remember during a crisis – Do not make the mistake of “forging ahead” with a restore before taking a backup!
Full Recovery Model
• Also remember that you can restore only part of your last transaction log backup – We can stop at 1129 to ensure minimal data loss!
Example #1
Backup Strategy:
Database: LIFUtility Recovery model: FULL
• • • • • •
• Full backups on Saturdays at 2300 • Differential backups each day at 2300
Step 1: Backup the current log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore the log but tell it to stop at 1129 All changes after 1129 will be lost Any open transactions as of 1129 will be rolled back
Backup Strategy
• Question #1: – What would happen if we accidently forgot to take the log backup in Step 1? • We would lose all changes since the DIFF
Backup Strategy
• Question #2: – What would happen if the disk(s) holding the data file crashed during the backup of the log? • Remember that we can still backup the log because changes are written to the log first, and then to the data file – This is a primary function of the transaction log
Backup Strategy
• Question #3: – What would happen if the disk(s) holding the transaction log crashed during the backup of the log? • We would lose all changes since the DIFF
Be Careful!
• Question 3 highlights a real problem: – If you do not backup the log frequently, you are at risk of losing all data since the last backup
• How do we mitigate this problem? – More frequent backups – Redundant disks
In the next video…
• We’ll cover more strategies and issues for the FULL recovery model
SQL Server Backup and Recovery
Presented by Scott Whigham
118
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Full Recovery Model
• Let’s take a look at a final example – This is a more likely scenario for a database with active changes (like LIFWeb)
Example #1
Backup Strategy:
Database: LIFWeb Recovery model: FULL
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
• Same scenario: a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. What is your restore sequence? – ______________________________________________
Example #1
Backup Strategy:
Database: LIFWeb Recovery model: FULL
• • • •
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore all log backups that have occurred since Thursday’s DIFF and stop at 1129 in the log backup from Step 1
Reminders
• You only restore the most recent DIFFERENTIAL – This is critical to understand for performance reasons • Restoring 300+ log files takes a long time whereas restore a single DIFF backup can be quite fast
Reminders (cont.)
• You will likely have a large number of log files – Do not lose them - you cannot skip logs during a restore! – Be cautious about who (or what) you allow to take log backups and manage the files – Keep redundant copies • Backup to disk and then copy to tape
Remember our strategy:
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 • Log backups each hour
Database: LIFWeb Recovery model: FULL
Day
Type
Filename
Saturday
FULL
LIFWeb_Full_20100306.bak
Sunday
- LOG
LIFWeb_Log_20100307_0000.bak
Sunday
- LOG
LIFWeb_Log_20100307_0100.bak
Sunday
- LOG
LIFWeb_Log_20100307_0200.bak
(log backups each hour = 23 log backup files per day)
Sunday
- DIFF
LIFWeb_Diff_20100307.bak
Differentials
• It is possible to only use FULL and LOG backups – If you lose your DIFF backups, you can simply use the LOGs instead
Differentials
• Each DIFF contains all changes since the last FULL backup – Prepare that your DIFFs will take longer and longer (and be larger and larger) as you get farther away from the FULL backup • Saturday: FULL • Sunday: Diff (100MB) • Monday: Diff (200MB)
Differentials
• Example of a 1GB/day delta: Day
Type
Size in in GB
Saturday
FULL
300
Sunday
DIFF
1
Monday
DIFF
2
Tuesday
DIFF
3
Wednesday
FULL
304
Thursday
DIFF
1
Friday
DIFF
2
Crash Procedure for Full Recovery Model
• Restore sequence for Full recovery model: • • • •
Step 1: Backup your current transaction log Step 2: Restore your most recent FULL backup Step 3: Restore your most recent DIFFERENTIAL (if available) Step 4: Restore all transaction log backups taken since the last FULL or DIFFERENTIAL in sequence
– If you follow these steps, you will restore to “point of failure”
In the next video…
• How changing to BULK LOGGED recovery model affects your risk
SQL Server Backup and Recovery
Presented by Scott Whigham
131
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Logged Recovery Model
• Okay - the final one: Bulk Logged – This is a more likely scenario for a database with bulk operations like LIFReporting – Remember that BULK LOGGED is identical to FULL except when bulk operations occur
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same scenario: a “mistake” is identified at 1200 on Friday and tracked back to a hotfix installation at 1130 on Friday. What is your restore sequence? – ______________________________________________
Example #1
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same as with the FULL: – – – –
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore log backup from Step 1 but stop at 1129 – “Point in time” restore
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• New scenario: At 1200 on Friday, the disk(s) storing your transaction log crash. What is your recovery sequence? – ______________________________________________
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Step 1: Restore Saturday’s FULL • Step 2: Restore Thursday’s DIFF • Step 3: All changes since the last backup will be lost
Example #2
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• New scenario: At 1200 on Friday, the disk(s) storing your data file crash. What is your recovery sequence? – ______________________________________________
Aha!
• You actually do not have enough information to answer! – What question do you need to know the answer to before you can come up with a sequence? • “Have any bulk operations taken place since the last backup?”
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Let’s be more specific: At 1200 on Friday, the disk(s) storing your data file crash. There have been no bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Same as with the FULL: – – – – –
Step 1: Backup the log Step 2: Restore Saturday’s FULL Step 3: Restore Thursday’s DIFF Step 4: Restore all log backups in sequence “Point of failure” restore
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• Let’s be more specific: At 1200 on Friday, the disk(s) storing your data file crash. There have been bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #3
Backup Strategy: • Full backups on Saturdays at 2300 • Differentials each day at 2300 Database: LIFReporting Recovery model: BULK LOGGED
• We cannot back up the log – – – –
Step 1: Restore Saturday’s FULL Step 2: Restore Thursday’s DIFF All changes since the last backup are lost “Point of last backup” restore
Example #4
Backup Strategy:
Database: LIFReporting Recovery model: BULK LOGGED
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Hourly log backups
• Same as before: At 1200 on Friday, the disk(s) storing your data file crash. There have been bulk operations since the last DIFFERENTIAL backup. What is your recovery sequence? – ______________________________________________
Example #4
Backup Strategy:
Database: LIFReporting Recovery model: BULK LOGGED
• Full backups on Saturdays at 2300 • Differentials each day at 2300 • Hourly log backups
• We still cannot back up the log: – Step 1: Restore Saturday’s FULL – Step 2: Restore Thursday’s DIFF – Step 3: Restore all log backups that have occurred since last DIFF backup – All changes since the last backup are lost – “Point of last backup” restore • However, we’ve “lost less”
In the next video…
• We’ll cover a practical way to manage bulk operations that will minimize your risk
SQL Server Backup and Recovery
Presented by Scott Whigham
147
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Bulk Import How-to
• When you want to perform long-running bulk imports, you want to maximize performance – Example: you have a text file with rows of data that previously were stored in DB2 – You can use BULK INSERT, OPENROWSET(), SSIS, or bcp to load these rows into a SQL Server table
Bulk Import How-to
• There are two techniques to maximize performance: – Drop nonclustered indexes when doing bulk imports • Speeds up the import • Must rebuild after import
– Use BULK LOGGED recovery model
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• We backup the log first so that we have a “safe” restore point – Assuming we are loading data from a text file, if the import fails, we can always restore back to this log and re-do the import with no data loss
Bulk Import How-to
• If desired, we could switch Steps 1 and 2 around with no negative effects – Remember that we can switch between FULL and BULK LOGGED as necessary
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Use the DROP INDEX statement – Don’t forget to script out your index creation statements before you drop your indexes…
• Review BULK INSERT, OPENROWS, Integration Services, and, as a last resort, bcp.exe for options on importing rows
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup your log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• These two steps could also be switched around if you prefer • It is recommended that perform your log backup prior to adding indexes – Minimizes the amount of restore time in the event a catastrophe occurs during index creation
Bulk Import How-to
• Field Guide to Performing Bulk Imports: Assumes your database is normally in FULL recovery model
1. 2. 3. 4. 5. 6. 7. 8.
Backup the log Switch to BULK LOGGED recovery model Drop your nonclustered indexes Perform the import Backup the log Add your indexes back in Switch back to FULL recovery model Backup the log (optional)
Bulk Import How-to
• Now that you have completed your bulk ops, switch back to FULL for normal, everyday operations – FULL recovery model provides the most complete coverage against data loss
Bulk Import How-to
• Question: if you forget to switch over to BULK LOGGED recovery model, can you still perform bulk operations? – Somewhat of a trick question… – Let’s break it into two parts…
Bulk Import How-to
• If you load a text file using BULK INSERT into a SQL Server database that uses the FULL recovery model, will the operation be fully logged or minimally logged? – Fully logged! – SQL 2000/7.0 allowed DBAs to prevent these operations but not SQL 2005/2008
Bulk Import How-to
• So if this is a fully logged operation, is it technically correct to still call it a “bulk operation”? – Yes, these are still bulk operations yet they are not minimally logged
Bulk Import How-to
• Question: What tool(s) will you use to perform the steps in this guide? – A SQL Server job is perfect – Integration Services is ideal as well – You can even schedule a SQL Server job to execute your Integration Services package!
SQL Server Backup and Recovery
Presented by Scott Whigham
165
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Checkpoints
• To understand the transaction log, you need to understand what a checkpoint is – Writes all dirty pages for the current database to disk. – Writes “maintenance” information in the event a recovery needs to occur – Checkpoints happen about once per minute • Configured via sp_configure and “recovery interval”
The Transaction Log
• We talked earlier about how the Simple recovery model “clears out the log” – The transaction log is “cleared out” at each checkpoint • This is why we cannot take “meaningful” log backups
– But what exactly gets removed from the log? How does SQL Server know what to keep and what to remove?
The Transaction Log
• The log has three basic parts: – Active log – Inactive log – Free space
The Transaction Log
• The active log (or “active portion of the transaction log”) is: – From the beginning of the oldest “open” transaction – From the beginning of any transactions marked for replication that have yet to be delivered to the distribution database
The Transaction Log
• The inactive log (or “inactive portion of the transaction log”) is: – From the beginning of the logical log up until the beginning of the active log
Visualizing the Transaction Log
• Visualization:
In the next video…
• We’ll learn more about the architecture of the transaction log
The Transaction Log
• Let’s take this backup strategy and work through it – FULL: Saturdays at 2300 – DIFF: Weekdays at 2300
SQL Server Backup and Recovery
Presented by Scott Whigham
175
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
The Transaction Log
• You can remove the inactive portion of the log – This is “clearing out the log” – Technically called truncating the log
• You cannot remove any part of the active portion
The Transaction Log
• The Simple recovery model automatically truncates the log on checkpoint • Why do we take log backups? – So that we can “replay” the events on the server in the exact order as they happened – About 1x/minute the inactive portion of the log is cleared out – Makes the log useless for recovery
The Transaction Log
• FULL and BULK LOGGED do not truncate the log – Do not become one of my examples to talk about – Your transaction log can grow to fill up the disk!
The Transaction Log
• By default, when you perform a transaction log backup, the inactive portion of the log becomes free space • FULL and DIFFERENTIAL backups do not clear out the log
The Transaction Log
• When you take a log backup, SQL Server: – Copies the entire log to the backup file – Marks the inactive portion of the log as “free space” (i.e. it can/will be overwritten)
• This is why your logs must be restored “in sequence”
The Transaction Log
• It is possible to backup the log without removing the inactive portion • Used in two instances: 1. When you want to take a log backup without affecting your “normal” backup routines •
Remember that you must restore “all logs since the last DIFFERENTIAL in sequence”
The Transaction Log
• Used in two instances: 2. When the data file is unavailable • •
What if the disk(s) holding the data file (.mdf) crash? We can still backup the log provided we do not truncate the log
In the next video…
• Let’s demo what happens when you never backup your log!
SQL Server Backup and Recovery
Presented by Scott Whigham
185
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Securing Your Backups
• You need to secure your backup files and tapes • Anyone can take a backup file and restore it on another server
Securing Your Backups
• Previous versions of SQL Server allowed passwords on backups – Deprecated in SQL Server 2005 due to being a weak protection mechanism BACKUP DATABASE master … WITH PASSWORD = ‘$hjg#907563’ -- required to restore
Securing Your Backups
• Best practice: backup your databases to disk and secure those files with proper Windows file permissions – Technically called “Access Control Lists” (ACLs) – Set the permissions on the root folder the backups are stored in
Securing Your Backups
• Best practice: secure your tapes – Again, if someone can get to a backup file, they can restore it…
Misc Bits
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
Securing Your Backups
• Remember that SQL Server does no compression on backups by default • For years, we’ve relied on 3rd party tools – WinZip, WinRAR, 7-Zip are unsupported
Securing Your Backups
• SQL Server 2008 debuts a feature called Backup Compression – Can only create compressed backups in Developer and Enterprise Edition – All editions can restore a compressed database
Securing Your Backups
• Compression actually increases backup speed – CPU intensive – If too many resources are consumed by backup, consider using Resource Governor to limit CPU usage by Backup Compression
In the next video…
• And now for the random bits here and there about backups – Securing your backups – Backup Compression – NO_WAIT, ROLLBACK AFTER, and ROLLBACK IMMEDIATE – “Emergency! My transaction log has filled up!” – Taking a backup of the log without emptying the log
SQL Server Backup and Recovery
Presented by Scott Whigham
198
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Reminders about the restore process
• Remember the basics: – You must always start with a FULL backup – You may only restore a single DIFFERENTIAL • Remember that a DIFFERENTIAL contains all changes since the last FULL backup • You do not need to restore any log backups taken before your DIFFERENTIAL • You can, however, skip the DIFFERENTIAL and restore the logs instead
The Basics of Restoring Databases
• Basics of restoring LOG backups: – Logs must be restored in sequence – You cannot skip logs during the restore – You cannot skip transactions during a restore – It is possible to restore only part of the last log file you restore • Achieves “point in time” and/or “point of failure” recovery
The Basics of Restoring Databases
• Basics of restores: – The final backup you restore will, by default, take the database through the RECOVERY process and bring the database ONLINE • You decide which is the final backup file in your Transact-SQL code
– Any open transactions in the database will be rolled back during the RECOVERY process
If you are lucky…
• Hopefully your backups are in the same place that SQL Server created them – When you take a backup, that information is stored in that instance’s msdb database • Database, time, name, user, and location • Used by SSMS to suggest restore sequence!
msdb
• msdb contains several tables for backups: – dbo.backupset - one row for each backup set • A backup set contains the backup from a single backup operation
– dbo.backupfile - one row for each backup file • A backup set contains the backup from a single backup operation
– There are others as well
SSMS is your friend
• If, during a restore, all backup files are in the same location they were when they were created: – You can breathe easy - SSMS will automatically create the restore sequence (correctly!) – You then can choose “point in time”, “point of failure”, or to restore up to a specific backup
SSMS is your friend
• It is highly recommended that you at least use SSMS to generate your restore sequence – SSMS reads the msdb tables to come up with: • Most recent FULL • Most recent DIFF (if available) • Each LOG in sequence
SSMS is your friend
• If you do use SSMS to recover, SSMS defaults to: – “Point of last backup”
• You can change this with ease
Transact-SQL is a pain…
• Now… If you want to restore using Transact-SQL, prepare for a lot of typing – Transact-SQL requires a separate RESTORE statement for each backup – Let’s learn more in the next video!
SQL Server Backup and Recovery
Presented by Scott Whigham
209
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Deeper into the restore
• Now… If you want to restore using Transact-SQL, prepare for a lot of typing – Transact-SQL requires a separate RESTORE statement for each backup – To get it “right the first time” requires a deeper understanding of the restore options and process
The RECOVERY process
• Remember the concept of “RECOVERY” – When a database goes through RECOVERY: • • • •
Committed transactions are rolled forward Open transactions are rolled back Database is brought ONLINE Only done on the last backup restored
The RECOVERY process
• When you have to restore multiple backups, you cannot go through RECOVERY until the last backup – Example restore sequence: • FULL: from Saturday • DIFF: From 2300 night before • LOG: 12 log backups to restore
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 1: Restore the FULL – Do you want to go through RECOVERY now? • No, otherwise open transactions that might be completed in a later backup to be restored would be rolled back
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 2: Restore the DIFF – Do you want to go through RECOVERY now? • No, for the same reason…
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 3: Restore 11 logs (but not the last log) – Do you want to go through RECOVERY during any of these logs? • No, for the same reason…
The RECOVERY process •
Example restore sequence: – – –
FULL: from Saturday DIFF: From 2300 night before LOG: 12 log backups to restore
• Step 4: Restore the last log – Do you want to go through RECOVERY? • Yes! There are no more backups to be applied so we want any uncommitted transactions rolled back!
Restore options
• When restoring a backup file, you have three options: – RECOVERY (the default) – NORECOVERY • Don’t rollback or roll forward
– STANDBY • • • •
Allows read-only access to committed transactions Still able to continue with backup Requires an external file to store rolled back transactions This file must be available for subsequent restores
In the next video…
• We’ll go through restoring databases in SSMS and exploring the options…
SQL Server Backup and Recovery
Presented by Scott Whigham
220
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Point-in-time Restores are tricky
• Scenario: – “Mike” calls you and explains that, sometime around 11:30AM, a “bad” transaction was issued by a developer • Mission critical - must undo this transaction or restore to point prior to the transaction • Gotta do it now!
Point-in-time Restores are tricky
• Question: how can you read the transaction log to view the transactions? – Short answer: you can’t, by default – Well, you can but in a generally not-easy-to-reador-remember way
Point-in-time Restores are tricky
• There are third party applications that allow you to view the SQL Server log easily – Not cheap – Can be massive time savers – Lumigent Log Explorer, ApexSQL Recover among others
Point-in-time Restores are tricky
• If you don’t have access to the third party application, what are your options? – If you had Profiler running at the time, review! – Can review default traces – Can restore then “look and see if ‘bad’ transactions were gone
Enough lecture - let’s demo!
• Let’s go see this in action…
SQL Server Backup and Recovery
Presented by Scott Whigham
227
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Restoring Your Entire Server
• The Scenario: – The disk system that held all of your data files (.mdf) has just crashed and cannot be recovered – The sysadmins have just completed installing new drives and building the array – Now it’s your turn to get that SQL Server back up and running
Restoring Your Entire Server
• You are right in the middle of a catastrophe – You can’t make up for the past now • So you’d better have been backing things up!
– Right now, you need a plan…
Restoring Your Entire Server
• Since you cannot backup, your only option for a restore is “point of last backup” – Or is it??? We’ll just have to see about that later…
Restoring Your Entire Server
• You are awesome – Thankfully you have recent backups of all the right things • • • •
System Databases User Databases Service Master Key Certificates
Restoring Your Entire Server
• You are double awesome – You also have documentation: • • • •
Server collation Install locations of files Service pack/hotfix level of server Output of sqldiag.exe – sqldiag.exe is a general purpose diagnostic utility that makes keeping track of many servers easy
Restoring Your Entire Server
• We use the SQL Server setup.exe to perform a rebuild • The process is: 1. If there are system database files already in the target folder, delete them 2. Create new system database files from /Binn/Templates/ files •
Located in the folder that your program files are located
Restoring Your Entire Server
• The rebuild process is done via setup.exe: – NOTE: INSTANCENAME=MSSQLSERVER for default instance
setup.exe /QUIET /ACTION=REBUILDDATABASE /INSTANCENAME=instance_name /SQLSYSADMINACCOUNTS= accounts [/SAPWD=password] [/SQLCOLLATION=collation_name]
Restoring Your Entire Server
• Once you rebuild master, you will have an “empty” server – Remember: you are creating new master, model, msdb, and tempdb databases from the templates – The template master database does not know about your databases
Restoring Your Entire Server
• You will not see your: – Databases – Logins – Jobs – Server Objects (i.e. linked servers, backup devices)
• Server settings will not be your settings • model database is back to the default
In the next video…
• We’ll take a look at how to rebuild master
SQL Server Backup and Recovery
Presented by Scott Whigham
239
• Backup Strategies • Backing Up Your Databases • Backing Up Your SQL Server
Restoring Databases
• Overview of Chapter • Scenario Setup • Types of SQL Server Backups • Recovery Models
Backup Your Databases
Overview
In this video…
• Restoring User Databases • Restoring System Databases
Restoring Your Entire Server
• The Scenario: – The disk system that held all of your data files (.mdf) has just crashed and cannot be recovered – The sysadmins have just completed installing new drives and building the array – Now it’s your turn to get that SQL Server back up and running
Restoring Your Entire Server
• To get your SQL Server back: 1. Reinstall service pack and any hotfixes 2. Restore master •
Once complete, you databases, logins, server objects and server settings will be restored
3. Restore msdb •
Restores your jobs, database mail, and more
4. Restore model (optional) 5. Backup the tail of the log for any user databases affected 6. Restore any user databases affected
Restoring Your Entire Server
• Reminders: – No one can be in your database when you restore – You are overwriting a database during the restore • Conceptually think of it in three phases: 1. 2. 3.
Delete original database files Create new versions of database files Restore the backup data into the new files
Restoring Your Entire Server
• To restore master, your server must be in single user mode • Cannot restore other databases while in single user mode
Restoring Your Entire Server
• A word of caution: – Remember that, if you configure SSMS to open both Object Explorer and a query window, that is two connections
Restoring Your Entire Server
• Another word of caution: – Remember that, if your SQL Server Agent is configured to auto start with SQL Server, it logs in before you • When in doubt, stop the SQL Server Agent service
Restoring Your Entire Server
• To put your server in single user mode: 1. Stop the SQL Server Agent 2. Add the -m switch to your SQL Server service’s Startup Parameters via Configuration Manager 3. Restart the service 4. Restore master 1. SQL Server will automatically shut down
5. Remove the -m switch 6. Restart your SQL Server
Restoring Your Entire Server
Restoring Your Entire Server
• About restoring msdb: – SQL Server Agent logs in before you do – SQL Server Agent loves msdb – Make sure you stop the SQL Server Agent before trying to restore msdb
In the next video…
• We’ll take a look at how to rebuild master
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
1
p. 1
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 2
Chapter Introduction
• Chapter 8 covers several unrelated topics – All essential to being a DBA – None large enough to warrant a full chapter
p. 3
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 4
Moving and Copying Databases
• There are several techniques for moving and copying databases – BACKUP and RESTORE – DETACH and ATTACH – SQL Server Integration Services – Copy Database Wizard
p. 5
Moving and Copying Databases
• Depending on which technique you us, you can copy and move: – On the same server – Onto multiple servers – Multiple databases at the same time
p. 6
Moving and Copying Databases
• In the next few videos, we’ll see examples of how to move and copy databases using the following techniques: – BACKUP and RESTORE – DETACH and ATTACH – SQL Server Integration Services – Copy Database Wizard
p. 7
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
8
p. 8
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 9
Orphaned Users
• Way back in Chapter 5 (security) we talked about how user accounts are related to logins via the security identifier (SID) – Not by name
p. 10
Orphaned Users
• User accounts that previously had a login but now have no associated login are called orphaned users • Orphans can be created: – By deleting logins – By copying/moving database to new instance
p. 11
Orphaned Users
1. When you delete a login, the associated users are not deleted
p. 12
Orphaned Users
2. When you copy/move a database to a different instance, the user accounts may be orphaned –
–
If the destination instance does not have logins for the user account SIDs, the user accounts will be orphaned If the destination has logins with the expected SIDs, the user accounts will be mapped
p. 13
Orphaned Users
• Windows authentication logins: – If your user accounts are built on Windows user/group authentication logins, just create the Windows authentication login in the SQL Server • Remember from Chapter 5 that the SID is the actual Windows SID therefore the login/user will be mapped automatically
p. 14
Orphaned Users
• SQL Server authentication logins: – For SQL Server authentication login-based user accounts, simply creating the login with the expected name won’t work • CREATE LOGIN … generates a random SID
– Later on, we’ll cover how to fix!
p. 15
Orphaned Users
• You will encounter this problem often – The user accounts are stored in the database but logins are not – Any time you copy/move a database, you have to deal with this issue
p. 16
In the next video…
• We’ll talk about how to identify orphaned users and strategies for resolving
p. 17
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
18
p. 18
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 19
How to Identify Orphaned Users
• There are several ways to identify orphaned users 1. sp_change_users_login stored proc
2. Query the sys.database_principals view to locate users with no matching sid in sys.server_principals 3. Use SSMS’s graphical tools
p. 20
Orphaned Users
• There are two solutions to this problem: 1. Use sp_change_users_login stored procedure 2. Script out the login complete with SID from the source server and run script on destination server
p. 21
Let’s demo!
• Let’s take a look at using sp_change_users_login and scripting
techniques!
p. 22
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
23
p. 23
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 24
CLR Assemblies
• SQL Server 2008 allows you to load .NET DLLs (called assemblies) into your databases – Assemblies can be written in any language that implements that .NET Common Language Runtime (CLR) • C#, Visual Basic, etc • Often called “managed code” or “managed assemblies”
p. 25
CLR Assemblies
• An assembly loaded into SQL Server can contain one or more: – Stored procedures – User-defined functions – Triggers – and more
p. 26
CLR Assemblies
• Why would we need to use CLR assemblies? – Mostly because certain operations are more efficient in .NET than Transact-SQL • Examples: string manipulation, regular expressions
– Secondly because .NET can do things Transact-SQL cannot do • Example: create table results containing file and folder info
p. 27
CLR Assemblies
• What does this have to do with the DBA? – As the “gate keeper”, you must decide whether to allow and, if you do allow, to what extent you allow CLR integration – Remember the Principal of Least Privilege! • http://learnitfirst.com/go.aspx?id=LPSX
p. 28
CLR Assemblies
• CLR integration is disabled by default – Enabled using the sp_configure stored proc
p. 29
CLR Assemblies
-- To view setting: EXEC sp_configure 'clr enabled' -- run_value = 1: enabled
-- To enable: EXEC sp_configure 'clr enabled', 1 GO RECONFIGURE WITH OVERRIDE
p. 30
In the next video…
• We’ll talk about permission sets and how to configure security for CLR objects
p. 31
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
32
p. 32
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 33
CLR Assemblies
• Once you enable SQL CLR, you must decide which permissions to allow • When you install an assembly, you must determine it’s Host Policy Level – SAFE – EXTERNAL_ACCESS – UNSAFE
p. 34
SAFE
• SAFE is the default and most secure – Cannot go outside the SQL Server process – Can connect to SQL Server in-process
p. 35
EXTERNAL_ACCESS
• EXTERNAL_ACCESS is “SAFE plus go out of process” – Can do all that SAFE can do – Can also go outside the SQL Server process to access external resources • • • •
Read files and folders Read environmental variables Read registry more…
p. 36
EXTERNAL_ACCESS
• UNSAFE can do anything – Can take over the world! – Only sysadmins can assign – Has same permissions as EXTERNAL_ACCESS plus: • Can call unmanaged code (i.e. C++ DLLs) • Allows assemblies to perform “illegal operations against the SQL Server process space…” – Source: Books Online
p. 37
How do I know which to use?
• As a DBA, you cannot know every nuance of every language that can write managed code assemblies • Must coordinate with developer to understand security needs
p. 38
How do I know which to use?
• As a general guideline: – Use SAFE for assemblies that need only to do computations, manipulations, and table access – Use EXTERNAL_ACCESS for when assemblies need to go outside SQL Server space – Use UNSAFE when working with unmanaged code
p. 39
How do I know which to use?
• Each permission set has different requirements for creation: – SAFE: requires CREATE ASSEMBLY permission in the database – EXTERNAL_ACCESS: SAFE plus EXTERNAL ACCESS ASSEMBLY server permission – UNSAFE: sysadmins only!
p. 40
How do I know which to use?
• Create assembly in the [LearnItFirst.com] database USE [LearnItFirst.com] GO CREATE ASSEMBLY LIFRegEx FROM „C:\MyAssembly.DLL‟ WITH PERMISSION_SET = SAFE p. 41
How do I know which to use?
• Can change permission set with ALTER ASSEMBLY ALTER ASSEMBLY LIFRegEx WITH PERMISSION_SET = EXTERNAL_ACCESS
p. 42
The DBA’s Role
• With respect to the CLR, the DBA is expected, at a minimum, to be able to: – To be able to explain what the CLR is – To be able to make suggestions related to when to use the CLR for certain operations – To be able to stop developers from using CLR unnecessarily – To understand the various permission sets
p. 43
The DBA’s Role
• To reiterate: you are not expected to know every single language – Don’t be intimidated by the CLR – If a manager tells you that, “You have to know C# to be a good DBA” during a job interview, you do not want to work for that person! • ….. Well, how much does it pay again?
p. 44
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
45
p. 45
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 46
The Resource Governor
• The Resource Governor: – New in SQL 2008 – Allows you to limit “resources” based on certain criteria • CPU, memory, number of requests, more
– Only available in Enterprise and Developer editions
img by eliburd @ flickr.com
p. 47
Example Usage
• Examples uses of the Resource Governor – Can prevent expensive queries from consuming more than an allowed amount of memory or CPU – Limit the amount of CPU that backup compression can consume – Ensure that specific applications cannot issue queries or workloads that are unhealthy to the rest of the server – Web host wants to provide guaranteed levels of service to multiple clients
p. 48
Example Usage
• The basics of how it works: 1. DBA creates a resource pool 2. DBA assigns the resource pool one or more workload groups 3. DBA writes a classifier function that binds incoming sessions to the appropriate workload group
p. 49
The Resource Governor
• Source: Using the Resource Governor, white paper, Microsoft 2009
p. 50
Resource Pools
• Resource Pools are the highest layer of abstraction – Can define a MIN and MAX CPU – Can define a MIN and MAX memory – Contain n workload groups
p. 51
Workload Groups
• Workload groups allow more granular settings
p. 52
Workload Groups
• Settings available for workload groups: – 0 means unlimited for all – Maximum Requests • Max number of simultaneous requests
– CPU Time (sec) • Max amount of CPU time that a request can use
– Memory Grant % • Max amount of memory a single request can take from the pool
– Grant Time-out (sec) • Max time that a query can wait for a resource before the query fails
– Degree of Parallelism • Max degree of parallelism for parallel requests. Range is 0 to 64
p. 53
Classifications
• Classification is the process of determining the workload group for a given session – A user-defined scalar function that is registered using the ALTER RESOURCE GOVERNOR statement • Called the classifier function
– Evaluated for every new session except Dedicated Administrator Connections
p. 54
The Classifier Function
• The classifier function: – There can be only one – If it returns NULL or a non-existent group, the session is bound to the default workload group – Sessions are bound to the workload group for life
p. 55
Putting It All Together
• After enabling the Resource Governor and putting a classification function in place: 1. User logs in 2. The login authentication is verified 3. Upon successful authentication, any LOGON triggers are executed 4. Finally, classification occurs
p. 56
Words of Wisdom
• A final word of caution – Be ready… • If you mess up your classifier function, you’ve messed up the entire server • Only Dedicated Administrator Connection (DAC) sessions can bypass classification • If a problem arises, log in as DAC and remove classifier
p. 57
Let’s play!
• In the next video, let’s play with the Resource Governor!
p. 58
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
59
p. 59
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 60
Policy-Based Management
• Windows administrators have long been familiar with policy-based management – “All user accounts must change their password every 42 days and it must contain letters, numbers, and special characters”
p. 61
Policy-Based Management
• DBAs have always had standards and business rules – “All stored procedures must begin with usp_” – “No database may have AutoShrink turned on”
• However, what we haven’t had is a way to enforce them automatically
p. 62
Policy-Based Management
• SQL Server 2008 introduces Policy-Based Management (“PBM”) – Allows DBA to define business rules (i.e. policies) that can either be: • Automatically enforced by SQL Server (i.e. prevent noncompliant modifications) • Reported to the DBA as non-compliant
p. 63
Policy-Based Management
• Nearly all DBAs have a daily task list: – Verify backups – Look for anomalies (low disk space, unrecognized files or folders, etc) – Check the error logs – Check the job logs – and many more tasks…
p. 65
Policy-Based Management
• One such task is to verify that new servers are compliant – Example: “Have any new servers been installed?” • If “Yes”, are they up to our standards? – Evaluation or Developer Edition only unless installed by DBAs (to ensure licensing compliance) – Minimum build number – MYDOMAIN\SQLDBAs Windows group is sysadmin – Authentication mode is Windows-only (a.k.a. Integrated)
p. 66
Policy-Based Management
• Another task is ensuring no changes have occurred that result in databases/objects being non-compliant – Are all production databases • Online • Using correct recovery model
– Are there any new [tables/views/functions/procs]? • Do they meet naming conventions?
p. 67
Policy-Based Management
• Manual enforcement of these types of policies is error prone and a time sink – Spending just 10 minutes a day on manually enforcing policies adds up to more than 40 hours at the end of a year • An entire work week!
– How much time did you have to spend during the year fixing a problem that occurred because of non-compliance?
p. 68
In the next video…
• We’ll take a look at the terms and concepts behind SQL Server 2008’s PBM
p. 69
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
70
p. 70
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 71
Policy-Based Management
• Understanding SQL Server’s policy-based administration is – Easy because we are all familiar with the basic ideas behind policy-based management – Difficult due to having to learn new abstractionlayer-type terms
p. 72
Policy-Based Management
• First, let’s talk about what you can police: – Nearly anything that has properties
• “Okay, so what items in SQL Server have properties?” – Everything
p. 73
Policy-Based Management
• “Are you saying that I can create a policy on anything I want?” – Pretty much!
• “What can’t I police?” – Anything you would normally do in a constraint or trigger – Anything you would normally do with Resource Governor
p. 74
Policy-Based Management
• Let’s now talk about the terms and concepts of Policy-Based Management (PBM) in SQL Server 2008
p. 75
Policy-Based Management
• It only takes two steps to define your policies: – Step 1: Create a condition – Step 2: Create the policy built on the condition
p. 76
Policy-Based Management
• Conditions define what you want to police – “I want to ensure that all stored procedures in our WebMarketing database start with usp_” – “No database may have AutoShrink turned on”
• This may seem like the policy but…
p. 77
Policy-Based Management
• Policies define the action you want to occur when a set of conditions is met – “A developer should receive an error if they try to create a stored procedure in our WebMarketing database that does not start with usp_”
p. 78
Policy-Based Management
• A (weak?) analogy: – Conditions are like SELECT statements •
Rows that are returned from running the query are compliant
SELECT * FROM sys.procedures WHERE name LIKE 'usp_%'
p. 79
Policy-Based Management
• So far, so good, right? – Nothing too difficult yet… – Let’s make things a bit more complicated in the next video!
p. 80
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
81
p. 81
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 82
Policy-Based Management
• In truth, PBM is quite easy • The tricky parts are: – Understanding terms like facet, property, and target – Understanding which facets allow which evaluation modes – Writing conditions and always “being positive”
p. 83
Policy-Based Management
• Facets are confusing at first – Simple mode: facets are just another word for things you already know by other names • Table, database, server, login, etc
– Advanced mode: facets are an abstraction layer that models behavior or characteristics for targets. • A target is a table, a database, a server, a login, etc
p. 84
Policy-Based Management
• Each facet contains at least one property • Properties have values • To retrieve or query a property’s value, you write an expression
p. 85
Policy-Based Management
• Example facet: The Stored Procedure facet has several properties: – Name – CreateDate – Schema
p. 86
Policy-Based Management
• To continue our previous analogy in which a condition was similar to a SELECT statement – Think of the facet as what occurs after the FROM – Think of the property as the column
p. 87
Policy-Based Management
• Consider this query: SELECT * FROM sys.procedures WHERE name LIKE 'usp_%„
• In this example, ‘sys.procedures’ would represent the Stored Procedure facet and ‘name’ would represent the facet’s Name property p. 88
Policy-Based Management
• We write our conditions against facets – Our desired condition: “All stored procedures in the WebMarketing database must begin with usp_”
• Which two facets will we use? 1. _________________________ 2. _________________________
p. 89
Policy-Based Management
• We write our conditions against facets – Our desired policy: “All stored procedures in the WebMarketing database must begin with usp_”
• Which two facets will we use? 1. Stored Procedure 2. Database
p. 90
Policy-Based Management
• Explanation: – “Stored procedure” is the facet and we must find all stored procedures whose Name property starts with usp_ – “Database” is the facet and we must find only that database whose name is WebMarketing
p. 91
In the next video…
• We’ll learn more about expressions and the mindset of writing conditions and policies
p. 92
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
93
p. 93
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 94
Policy-Based Management
• We write conditions using expressions – Not the same as a Transact-SQL expression – Same syntax as SSIS and SSRS expressions – You need to know expression syntax to be a good DBA
p. 95
Policy-Based Management
• Expression syntax is not difficult – Operators are same as C#: ==, !=, etc – Functions are similar to C# and T-SQL: • GETDATE() returns current system date/time • ISNULL() returns true/false to indicate if an expression is null
– Conditions are similar to Transact-SQL • LIKE and NOT LIKE, IN and NOT IN
– Double-quotes around strings
p. 96
Policy-Based Management
• Expression syntax is used in the graphical tools only – If you want to script out your conditions, you must write XML!
p. 97
Policy-Based Management
• Misc info about conditions: 1. Cannot change the facet of a condition after creation • Can change the expression but not the facet
2. A single condition may only operate on one facet • Can combine conditions in a policy to create complex scenarios
3. Conditions can be used by multiple policies • Design your conditions so that they can be combined with other conditions (i.e. create reusable conditions)
p. 98
Policy-Based Management
• On a production server using PBM, you will likely have hundreds of conditions and policies – It is important to have a naming convention – Opt for longer, descriptive names • Conditions have an optional Description for more information • Policies have Descriptions and “Additional help links” allowing you to create web page links in your errors
p. 99
Policy-Based Management
• Policies also have Categories (a.k.a. Policy Groups) – Can run the entire category against a database/server instead of one-policy-at-a-time – Can mandate that new databases participate
p. 100
In the next video…
• We’ll cover Execution Modes
p. 101
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
102
p. 102
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 103
Evaluations
• Anytime a policy is evaluated and finds non-compliance, it logs the offender – In the SQL Server error Log – In the Windows Application log – In the msdb database
• Be prepared for msdb growth!
p. 104
Execution Modes
• Policies also have Execution Modes which determine – When the policy will be evaluated – The response to non-compliance
p. 105
Execution Modes
• There are four execution modes: – On demand – On schedule – On change: prevent – On change: log only
• Not all facets support all execution modes
p. 106
Execution Modes
• On demand: policies are not evaluated unless manually done so by admin – Useful for audits and non-critical policies – Supported by all facets – Actions taken: • Raises error #34052 • Logs infraction
p. 107
Execution Modes
• On schedule: a SQL Server job is created that periodically evaluates the policy – Useful when evaluation performance impacts system – Supported by all database-engine facets – Job does not error out if policy violations found • Writes errors to the logs and evaluation history
– Actions taken: • Raises error #34052 • Logs infraction
p. 108
Execution Modes
• On change: log: logs non-compliance only – Useful when you want a history of policy – Supported by few facets – Actions taken: • Logs infraction
p. 109
Execution Modes
• On change: prevent: non-compliance is not allowed – Useful to prevent critical changes – Expensive – Supported by fewest facets – Actions taken: • Rolls back user transaction • Raises error #34050 or #34051 • Logs infraction
p. 110
Summary
Facet
On change: log
On change: prevent
Application Role
a
a
Asymmetric Key
a
a
Database Option
a
Database Role
a
a
Login Options
a
a
Resource Pool
a
a
Schema
a
a
Server Configuration
a
Stored Procedure, Functions, Views
a
a
Table Options
a
a
Workload Group
a
a p. 111
In the next video…
• We’ll cover whether or not policies replace permissions!
p. 112
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
113
p. 113
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 114
Writing Policies and Conditions
• Consider this security requirement: “All databases must be owned by a sysadmin” – Should this be done via policy or other means? • Answer: It depends on what action you want!
p. 115
Writing Policies and Conditions
• If you want to prevent someone who is not a sysadmin from adding a database, then this is a permission • If you want to report on the databases that are currently not owned by sysadmins, then you want a policy
p. 116
Writing Policies and Conditions
• “Do policies take the place of permissions in SQL Server 2008?” – No! – Policies and permissions actually have nothing to do with one another
p. 117
Writing Policies and Conditions
• Generally speaking, actions that can be prevented by permission are not preventable using On change: prevent – Example: preventing someone from creating databases, tables, procs, etc
p. 118
Writing Policies and Conditions
• Prevention policies are “things you have permission to do but, for some reason, management doesn’t want you doing them” – Such as creating stored procs in the WebMarketing database that do not start with usp_
p. 119
Writing Policies and Conditions
• Remember our policy: “We want all stored procedures in WebMarketing to start with usp_” – Is this a permission or a policy? • A policy however the user still needs the CREATE PROC permission in the database!
p. 120
In the next video…
• We’ll discuss how to write effective conditions and policies
p. 121
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
122
p. 122
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 123
Writing Policies and Conditions
• Lastly, writing effective policies requires practice and patience – Writing conditions can be a confusing mess • It is easy to create two policies that contradict each other
– Expect to spend a few hours “getting it right” for your first several policies and conditions • You will mess up!
p. 124
Writing Policies and Conditions
• Example policy requirement: “All databases must be owned by the sa” – This requires one condition since there is only one facet (Database facet)
p. 125
Writing Policies and Conditions
• How would you write the condition? 1. To return databases owned by the sa 2. To return databases not owned by the sa
p. 126
Writing Policies and Conditions
• How would you write the condition? – To return databases owned by the sa – To return databases not owned by the sa
• Your conditions define the ideal, not the exception • Since we want only sa-owned databases, our condition is where @Name=“sa” p. 127
Writing Policies and Conditions
• Example policy req’t: “We want all stored procedures in WebMarketing to start with usp_” – This requires two conditions since there are two facets: • •
Stored Procedure facet Database facet
p. 128
Writing Policies and Conditions
• We have the choice of writing our Stored Procedure condition so that it returns: – Stored procs that start with usp_ – Stored procs that do not start with usp_
• Which do we choose?
p. 129
Writing Policies and Conditions
• We have the choice of writing our Stored Procedure condition so that it returns: – Stored procs that start with usp_ – Stored procs that do not start with usp_
• Remember – we want our conditions to define what we do want, not what we don’t want
p. 130
Writing Policies and Conditions
• To understand why it is this way, consider our options for the second facet (Database facet): – Database named WebMarketing – Databases not named WebMarketing
• Clearly we choose the first option
p. 131
Writing Policies and Conditions
• The final step is to create a policy on these two conditions
p. 132
In the next video…
• Enough talk - let’s demo!
p. 133
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
134
p. 134
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 135
PowerShell!
• PowerShell support for SQL Server 2008 is included in SSMS • PowerShell is a very extensible and powerful scripting shell that allows admins to script (“automate”) administration functions
p. 136
PowerShell!
• If you know DOS, you can easily get started with PowerShell (“PS”) – Commands like dir, del, cd perform the same function in PS
• PowerShell can do everything you can do at the command line and more
p. 137
PowerShell!
• So why talk about PowerShell in a SQL admin course? – SQL DBAs can automate SQL Server via PowerShell scripts! – This allows for uniformity across the enterprise • Anyone who knows PowerShell can write scripts • Allows integration between systems - can load Windows Event Logs into SQL Server tables for analysis via PS
p. 138
PowerShell!
• You can manage your SQL Server by – Issuing PS commands directly into the PS shell (powershell.exe) – Executing PS files that contain PS scripts • C:\Scripts\DeployNewServer.ps1
– Creating SQL Server jobs that execute PowerShell steps
p. 139
PowerShell!
• Full coverage of PS is beyond the scope of this course however there are several concepts you will need to know • Let’s start with the basics…
p. 140
PowerShell!
• PowerShell and the OS: – Included in Windows 2008 and Windows 7 – Download for Windows 2003, Vista, XP, etc
• Versions of PowerShell – 2.0 - Windows 2008 R2 and Windows 7 – 1.0 - Windows 2008, Windows 2003, Vista, XP
• http://www.microsoft/powershell
p. 141
PowerShell!
• SQL Server 2008 ships with the sqlps.exe, a minishell – Can drop to command line and just run sqlps to manage servers – Integrated into SSMS
p. 142
PowerShell!
• The sqlps minishell incorporates several cmdlets for managing SQL Server – Pronounced “commandlets”
• cmdlets are .NET classes invoked by PowerShell at runtime • Common cmdlets included in sqlps: – Invoke-Sqlcmd – Invoke-PolicyEvaluation
p. 143
In the next video…
• PowerShell is easier to demo than talk about!
p. 144
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
145
p. 145
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 146
Unstructured Data
• Let’s compare data storage/access of 1990 to 2010 for a health insurance company • 1990: – Storage: DB2 – Tables to store an insured family: 25 – Type of data stored: row data only – Export formats: CSV, text
p. 147
Unstructured Data
• 2010: – Storage: SQL Server 2008 – Tables to store an insured family: 50 – Type of data stored: row data, xml, images, audio – Export formats: CSV, text, xml, images, audio
p. 148
Unstructured Data
• Today’s companies have to deal not just with row data but “unstructured data” as well – Legal firms need transactional support for MS Word, Excel documents – Health industry needs transactional support for imaging – Music industry needs transactional support for audio
p. 149
Unstructured Data
• Unstructured data and relational databases have a history – Most common technique: store unstructured data (“files”) in file system; store pointers to file locations in the database • Pros: fast, easy to understand, easy to implement, easy to separate storage • Cons: No transactional support, backup/restore complicated, requires separate permissions, links can become stale
p. 150
Unstructured Data
• Files are stored in a folder in the file system that requires additional permissions
p. 151
Unstructured Data
• Beginning in SQL Server 2005, we could store unstructured data in the form of files in a new data type, VARBINARY(MAX) – Replaced the older image data type – Pros: transactional support, simplify backup/restore, native SQL Server security, never stale, can separate storage using filegroups – Cons: speed, data must be converted in/out to binary format, limited to 2GB files, explosion of database growth, buffer pool memory used to access data
p. 152
Unstructured Data
• Files are stored in a VARBINARY(MAX) column as a Binary Large Object (BLOB)
p. 153
Unstructured Data
• SQL Server 2008 introduces FILESTREAM access – Sort of a blend of file system pointers and VARBINARY(MAX) techniques – Data is stored in the NTFS filesystem • Improved speed, separation of storage
– All data access is through SQL Server • Single point of security, full transactional support, fulltext indexing, no stale links
p. 154
Unstructured Data
• FILESTREAM is not for everything BLOB… – For small BLOBs, a standard VARBINARY(MAX) will offer better performance • Consider “small” to mean around 1MB in file size
– No backup compression – No Transparent Data Encryption (use Encrypted Files System instead) – Not supported in database mirroring
p. 155
Unstructured Data
• More about FILESTREAM: – Works in all editions – Allows full ROLLBACK support – Supported on clusters and in log shipping – No limit on file size • VARBINARY(MAX) is limited to 2GB • VARBINARY(MAX) FILESTREAM has no such limit
p. 156
In the next video…
• We’ll cover more DBA-related details about FILESTREAM
p. 157
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
158
p. 158
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 159
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 160
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 161
FILESTREAM
• FILESTREAM requires both SQL Server DBA and the Windows admin – The SQL Server Service must be configured for FILESTREAM access first using the SQL Server Configuration Manager
p. 162
FILESTREAM
• Inside SQL Server, FILESTREAM is an instance-level option – Disabled by default; sysadmins only can change
p. 163
FILESTREAM
• To enable FILESTREAM using TransactSQL: EXEC sp_configure ‘filestream access level’ -- 0: disabled (default) -- 1: Transact-SQL access only -- 2: Full access enabled (a.k.a. -T-SQL and streaming access -via Windows APIs) -NOTE: Local disks only for now…
p. 164
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 165
FILESTREAM
• Once you enable FILESTREAM at the instance level, you must now: 1. Create a new filegroup specifically for FILESTREAM 2. Add files to the filegroup
• You can do this with CREATE DATABASE or ALTER DATABASE
p. 166
Enable FILESTREAM on a Database
p. 167
Enable FILESTREAM on a Database
p. 168
Enable FILESTREAM on a Database
p. 169
FILESTREAM
• To create the filegroup at creation: CREATE DATABASE BigInsurance ON PRIMARY (NAME = BigInsurance, FILENAME = „D:\BigInsurance.mdf'), FILEGROUP BIDocs CONTAINS FILESTREAM( NAME = BigInsuranceDocs1, FILENAME = ‘F:\BigInsuranceFiles') LOG ON ( NAME = BigInsurance_log, FILENAME = „E:\BigInsurance.ldf')
p. 170
FILESTREAM
• To add FILESTREAM to an existing database: ALTER DATABASE [LearnItFirst.com] ADD FILEGROUP FilestreamFG CONTAINS FILESTREAM GO ALTER DATABASE [LearnItFirst.com] ADD FILE(Name=„File1‟, FileName=„F:\File1‟) TO FILEGROUP = FilestreamFG
p. 171
FILESTREAM
• To add FILESTREAM to an existing database: ALTER DATABASE [LearnItFirst.com] ADD FILEGROUP FilestreamFG CONTAINS FILESTREAM GO ALTER DATABASE [LearnItFirst.com] ADD FILE(Name=„File1‟, FileName=„F:\File1‟) TO FILEGROUP = FilestreamFG
p. 172
In the next video…
• We’ll take a look at the other requirements to use FILESTREAM
p. 173
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
174
p. 174
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 175
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 176
FILESTREAM
• You can add FILESTREAM support to new tables or existing tables – Use ALTER TABLE to add to an existing table
USE [LearnItFirst.com] GO CREATE TABLE dbo.ClaimImage ( ClaimId INT NOT NULL PRIMARY KEY , TheImage VARBINARY(MAX) FILESTREAM NOT NULL , RowId UNIQUEIDENTIFIER NOT NULL ROWGUIDCOL UNIQUE DEFAULT NEW_ID() ) -- Tables with FILESTREAM data must have a -- non-null UNIQUEIDENTIFIER column
p. 177
FILESTREAM
• To use FILESTREAM for a database, follow these five steps: 1. Enable FILESTREAM for the instance 2. Add a FILESTREAM-enabled filegroup to the database 3. Add one or more files to the new filegroup 4. Create your column and assign it the FILESTREAM attribute 5. Load your data! p. 178
FILESTREAM
• Writing/reading binary data is for developers – Requires accessing Win32 APIs – .NET is popular but can write Transact-SQL directly
INSERT dbo.ClaimImage (ClaimId, TheImage) VALUES (1, 0x01)
p. 179
FILESTREAM
• DBAs needs to know about security and backup/recovery – Security permissions are done inside SQL Server, not on the file system – The physical files/folders are secured by SQL Server service account – Backup/restore will include all files
p. 180
FILESTREAM Best Practices
• Best Practices for FILESTREAM – Do not make your ROWGUIDCOL a clustered index – For best performance, FILESTREAM files should be on separate physical volumes when possible – NEWSEQUENTIALID() will perform better than NEWID() when replicating databases – Read Paul Randall’s column: • http://learnitfirst.com/go.aspx?id=WBLX
– Have your network admins (or whoever configures the physical Windows Server 2008/2003 machine) read the top part of this article: • http://msdn.microsoft.com/library/dd206979.aspx
p. 181
FILESTREAM
• Final thoughts… – Transact-SQL’s DELETE statement causes the physical file to be deleted • Physical file deletion is not performed until a CHECKPOINT operation
– Do not attempt to manage files via the file system • Cannot change names, attributes or delete files
– There are other options… • Search for “SQL Server 2008 remote BLOB storage”
p. 182
Chapter 8: Miscellaneous Administration Presented by Scott Whigham
183
p. 183
• Chapter Introduction • Moving and Copying Databases • Dealing with Orphaned Users • CLR and .NET Assemblies
Management
Overview
What We’re Going to Cover
• Resource Governor • Policy-Based Management • Powershell • FILESTREAM • Snapshots
p. 184
Database Snapshots
• A database snapshot is a read-only “copy” of a database that “stops” at the point the snapshot is taken – Example: If you take a snapshot of the LearnItFirst database at 0800, the snapshot would have the entire database yet would not reflect any changes that occurred after 0800
p. 185
Database Snapshots
• About database snapshots: – Must exist on same instance as source – Available for Enterprise & Developer editions only – Debuted in SQL Server 2005 – Multiple snapshots can exist of same source db • Some sites create a snapshot for each day of the week
p. 186
Database Snapshots
-- At 0800, we execute: CREATE DATABASE LIFSnapshot_0800 ON(NAME=„LearnItFirst‟ ,FILENAME=„D:\LIF_0800.ss‟) AS SNAPSHOT OF LearnItFirst -- Logical file name must be -- same as source. Also note -- that we do not define a log p. 187
Database Snapshots
USE LIFSnapshot_0800 GO -- Returns data up through 0800: SELECT * FROM dbo.MyTable GO -- Changes after 0800 are unseen
p. 188
Database Snapshots
• Reads against the snapshot: – Will use the source database if the data requested has not changed since snapshot creation • Acquire SHARED locks at source to prevent blocking
– Will return the snapshot copy if the data has changed • This provides a “at creation” level of consistency to data
p. 189
Database Snapshots
• Database snapshots are generally used for three purposes: – Reporting/History: reads against a snapshot are consistent to the point-in-time of creation – Restore point: can restore a database using the snapshot; transactions after the snapshot was taken are lost – Testing: can test effect of changes and revert back to snapshot in the event of disaster/error
p. 190
Database Snapshots
• Example restore: -- Only allowed if: -- (1) there is only one snapshot -- (2) all source db files exist
RESTORE DATABASE LearnItFirst FROM DATABASE_SNAPSHOT = „LIFSnapshot_0800‟
p. 191
Database Snapshots
• Database snapshots use sparse files to minimize disk space – Feature of NTFS that allows NTFS to allocate disk space on an “as needed” basis – When you create a snapshot, database pages are not copied to the snapshot – As pages become dirty, the “original” data page is copied to database snapshot • When that happens, the sparse file’s size stays the same yet the “Size on disk” increases
p. 192
Database Snapshots
• A 1.1GB database with a “fresh” snapshot
p. 193
Database Snapshots
• After performing UPDATEs to source:
p. 194
Database Snapshots
• Snapshots get stale – Have to plan to delete old snapshots – If they stick around, the can become larger than source!
p. 195
Database Snapshots
• Final thoughts… – Pros: • Great way to provide users with a point-in-time view of database • Snapshots are quick and easy to create thanks to sparse files
– Cons: • No way to create using SSMS graphical tools; T-SQL only • Cannot change user permissions for snapshot vs. source – A snapshot is a read-only copy of the source
• Added overhead to source database whenever changes occur – Transaction is not complete until snapshot is up to date
p. 196
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
1
p. 1
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 2
Chapter Introduction
• Chapter 9 is about performance and exception monitoring your server • We’ll start with the basics (using Profiler) and end with new features in SQL Server 2008 (Data Collector)
p. 3
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 4
Tools of Performance Monitoring
• There are many built-in tools to monitor performance and health: – Activity Monitor and sp_who • Covered in Chapter 5!
– SQL Server Profiler • Detailed view of statements issued against instance
– Dynamic Management Views (DMVs) • “Live” view of server health and activity
– Performance counters and objects (“PerfMon”) • “Live” view of server health and activity
– Data Collector • Historical view of server health and activity
p. 5
Tools of Performance Monitoring
“But Mr. Scott, how do I know when to use each one?” Profiler
DMVs
Perf Mon
Data Coll.
Requires manual load
Requires manual load
Requires manual load
Detail
Depends…
Summary
Summary
Low
None
Negligible
Low-Hi
100-300
200-400
200-300
300-500
Provides “live”, real-time reporting? Can do trend analysis and historical reporting? Can view SQL issued? Provides detail-level view or summary? Can be used to diagnose performance problems? Level of impact on server Difficulty level
p. 6
In the next video…
• Let’s play with Profiler!
p. 7
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
8
p. 8
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 9
SQL Server Profiler
• SQL Server Profiler is a tool for monitoring real-time queries issued against your server – “Old Faithful” - been around for years – Requires sysadmin or ALTER TRACE server permission
p. 10
SQL Server Profiler
p. 11
SQL Server Profiler
• DBAs use traces to monitor statements sent to SQL Server – We’ve already seen how SQL Server uses “default traces” in Chapter 7
• Running traces will have an impact on server performance – How much? It depends on how much you trace
p. 12
SQL Server Profiler
• SQL Server Profiler is the GUI for defining, viewing, and replaying traces – It does not have to be running to create a trace – Many DBAs create their traces via Profiler and then script out to jobs
p. 13
SQL Server Profiler
• Running Profiler on a “live” system adds to an already stressful situation – Particularly if run interactively on the server
• If you must run Profiler against a “live” server, run it remotely • Adds network overhead but this is almost never an issue today
p. 14
SQL Server Profiler
• Trace data can be stored: – In a .trc file (fastest) – In a SQL Server table
• .trc files can be configured to rollover to a new file after a certain threshold • Can load .trc file contents into SQL Server table after trace for maximum flexibility!
p. 15
SQL Server Profiler
• After trace is complete, you can export results
p. 16
SQL Server Profiler
• Tips on working with Profiler – Traces can have filters to help minimize the noise • The better you filter, the less of an impact you put on the server
– Being able to write queries against the trace results can help you identify many common trends and problems • This requires you load the trace data into a table!
p. 17
In the next video…
• Let’s play with Profiler!
p. 18
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
19
p. 19
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 20
Dynamic Management Views
• DBAs use Dynamic Management Views (“DMVs”) and Dynamic Management Functions (“DMFs”) to peek under the covers – In the sys schema and begin with dm_ – Used to get up-to-the-second performance information during critical time – Often need to JOIN many together to come up with desired results p. 21
Tools of Performance Monitoring
Reminder… Profiler
DMVs
Perf Mon
Data Coll.
Requires manual load
Requires manual load
Requires manual load
Detail
Depends…
Summary
Summary
Low
None
Negligible
Low-Hi
100-300
300-500
200-300
300-500
Provides “live”, real-time reporting? Can do trend analysis and historical reporting? Can view SQL issued? Provides detail-level view or summary? Can be used to diagnose performance problems? Level of impact on server Difficulty level
p. 22
Dynamic Management Views
• There are too many DMxs to cover each one so we’ll cover concepts and core DMxs – Over 130 DMVs and DMFs in SQL Server 2008!
• It is important for DBAs to maintain a script library
p. 23
Dynamic Management Views
• The easiest way to get to know the DMVs/DMFs is to categorize them – Below are the more popular categories for performance monitoring Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 24
Dynamic Management Views
• Intellisense can help you greatly!
p. 25
Dynamic Management Views
• Object Explorer can also help
p. 26
DMV and DMF Query
• You can also query system views for full list SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_%‘ ORDER BY name p. 27
DMV and DMF Query Results
p. 28
Database Category
• Database category: – 16+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_db%'
p. 29
In the next video…
• We’ll take a look at a few categories and examples
p. 30
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
31
p. 31
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 32
Dynamic Management Views
• Reminder: the more popular categories for performance monitoring Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 33
Database Category
• Database category: – 16+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_db%‘ ORDER BY name p. 34
Database Category
• From a performance standpoint, you will spend a lot of time in the Database category – Primarily looking at table, index, and partition information • Size, fragmentation • Missing indexes • Index use
p. 35
Database Category
• Common index-related DMVs in the Database Category – sys.dm_db_missing_index_details returns information about indexes that could potentially help speed up the system – sys.dm_db_index_usage_stats can tell you whether an index is being used and, if so, in what way
p. 36
Database Category
• Common index-related DMFs in the Database Category – sys.dm_db_index_physical_stats is one way to identify fragmentation (see Chapter 8!) – sys.dm_db_index_operational_stats is similar to the DMV sys.dm_db_index_usage_stats yet only includes actual execution events instead of plan generation events
p. 37
In the next video…
• We’ll take a look at writing queries using the DMVs and DMFs in the Database category
p. 38
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
39
p. 39
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 40
Dynamic Management Views
• Let’s look at the other categories now… Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 41
Execution/Session Category
• Execution/Session category: – 20+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_exec%'
p. 42
Execution/Session Category
• The Execution/Session category will help you spy on people – That’s what you really want to do, isn’t it? – Can also view specific query information • Plans, text, stats, caching information
p. 43
Execution/Session Category
• There is at least one DMF in this category that you definitely want to know – sys.dm_exec_sql_text allows you to see the query text a user submits • By itself, it is not that useful but combined with other Execution/Session DMVs it is very powerful • Accepts one parameter - the memory location of the query plan – Populated via a column found in other sys.dm_exec_ DMVs
p. 44
Execution/Session Category
• A common DMV in this category is: – sys.dm_exec_query_stats allows you to see
metrics about queries submitted to server • Reads, writes, waits, execution count, etc • plan_handle column is the column to pass to sys.dm_exec_sql_text to return SQL
p. 45
Return 50 most-run queries
SELECT TOP(50) sql.text , qs.last_execution_time , qs.execution_count , qs.last_elapsed_/1000000.0 as ElapsedTimeInSecondstime FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS sql ORDER BY qs.execution_count DESC
p. 46
Return 50 most-run queries
p. 47
Execution/Session Category
• Other common DMVs in this category are – sys.dm_exec_procedure_stats same as query_stats yet only returns stored procedure
info – sys.dm_exec_cached_plans gives you info on
the cached query plans • Is it compiled or adhoc? • How many times has it been used?
p. 48
Execution/Session Category
• Spying on users is easy! – sys.dm_exec_sessions is one row for each
authenticated session
-- Returns all non-internal sessions SELECT * FROM sys.dm_exec_sessions WHERE session_id>50
p. 49
In the next video…
• We’ll write several queries and see the Execution/Session category in action
p. 50
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
51
p. 51
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 52
Dynamic Management Views
• Let’s look at the IO and OS categories now to help diagnose disk pressure Category
DMx prefix
Database
sys.dm_db_
Execution/Sessions
sys.dm_exec_
IO
sys.dm_io_
OS
sys.dm_os_
Transactions
sys.dm_tran_
p. 53
IO Category
• IO category: – 4+ DMVs and DMFs
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_io_%‘ ORDER BY name p. 54
IO Category
• The most common item in this category is – sys.dm_io_virtual_file_stats() returns information about usage of the database files • Reads, writes, IO waits, size • Accepts two parameters: – database_id - use NULL for all databases – file_id - the file_id from sys.database_files; use NULL for all files
p. 55
IO Category
• sys.dm_io_virtual_file_stats
returns several columns: Column
Description
sample_ms
Milliseconds since server was restarted
num_of_reads
Physical reads of objects in database file
num_of_bytes_read
Large textual-based columns will affect this greatly
io_stall_read_ms
Total time a user process waited for IO to this file
num_of_writes, num_of_bytes_written, io_stall_write_ms
Database write activity
io_stall
Total amount of waits reading /writing to this file
p. 56
IO Category
• Here are some things to monitor with dm_io_virtual_file_stats() – Look at total reads+writes to find “most active” – Look at io_stall to identify files that might benefit from new disks • All databases have stalls; the important metric is “What percentage of the uptime of the server am I having stalls?” • Compare read/write stalls on log and data file to identify which could benefit from new/faster disks
p. 57
IO DMF -- View IO stats for all DBs on instance: SELECT DB_NAME(mf.database_id) AS DB , mf.physical_name, mf.type_desc , vfs.* FROM sys.dm_io_virtual_file_stats(null,null)vfs JOIN sys.master_files mf ON mf.database_id = vfs.database_id AND mf.file_id = vfs.file_id ORDER BY DB_NAME(mf.database_id)
p. 58
IO DMF
p. 59
IO DMF -- Aggregate total IO since last restart: SELECT DB_NAME(vfs.database_id) AS database_name, CAST(SUM(vfs.num_of_bytes_read + vfs.num_of_bytes_written) / 1048576. AS DECIMAL(12, 2)) AS IO_MB FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs GROUP BY vfs.database_id ORDER BY IO_MB DESC
p. 60
OS Category
• The OS category is one of the largest – 36+ DMVs (no DMFs!)
SELECT name, type_desc FROM sys.system_objects WHERE schema_id=4 AND name LIKE 'dm_os_%‘ ORDER BY name p. 61
OS Category
• You can use this category to find waiting tasks: – sys.dm_os_wait_stats - aggregated info on dozens and dozens of wait types – sys.dm_os_waiting_tasks - detail info on waits for a specific resource • Hint: use dm_os_wait_stats to identify the areas of contention and then dm_os_waiting_tasks to view task information
p. 62
OS Category
• These two DMVs are accumulative – Their statistics are the totals since SQL Server service has been started
• To track effectively over time, it is necessary to reset periodically – Common to store in a table for historical perspective – DBCC SQLPERF(‘
p. 63
OS Category -- Current wait info: SELECT wait_type, wait_time_ms / 1000. as wait_time_s FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC -- Clear out stats: DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR) -- Run this at a later time to view waits since CLEAR SELECT wait_type, wait_time_ms / 1000. as wait_time_s FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC
p. 64
OS Category
p. 65
OS Category
• To identify disk pressure, you can look in sys.dm_os_wait_stats for specific wait types – Order by wait_time_ms DESC to identify most often waits – If anyone of these are in the top two positions, you have disk I/O pressure: • • • • •
ASYNCH_IO_COMPLETION IO_COMPLETION LOGMGR WRITELOG PAGEIOLATCH_x
p. 66
In the next video…
• Let’s look further at the OS category and others
p. 67
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
68
p. 68
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 69
OS Category
• The OS category has lots of info – sys.dm_os_sys_info - info such as number of CPUs, start time of service – sys.dm_os_tasks - lists each running task – sys.dm_os_loaded_modules - lists each loaded DLL that SQL Server is using • • • •
Cryptography Manager Kerberos Security Package Microsoft SQL Server Native Client 10.0 SQL Server Windows 95 Lite SCM
p. 70
OS Category
• Many OS category DMVs can help diagnose memory pressure – Performance Counters can help too – We’ll cover Perf Counters in an upcoming section of this chapter
p. 71
OS Category
• The main areas for memory pressure are: – Database page cache (used to cache table/index) – Query Workspace memory (for Hash and Sort ops) – Plan cache – There are others…
p. 72
OS Category
-- Find out what has the most entries -- in memory SELECT TOP (20)[Name], COUNT(*) AS 'Count' FROM sys.dm_os_memory_cache_entries GROUP BY [Name] ORDER BY COUNT(*) DESC;
p. 73
OS Category
p. 74
OS Category -- Top 10 consumers of Buffer Pool SELECT TOP 10 [type] , SUM(single_pages_kb) AS memory_in_kb FROM sys.dm_os_memory_clerks GROUP BY type ORDER BY SUM(single_pages_kb) DESC;
p. 75
OS Category
p. 76
OS Category
• One area that many servers have trouble with is the TokenAndPermUserStore cache – An internal security system that generates security tokens for sessions – Particularly troublesome area for DBs with lots of adhoc SQL – The problem can be that the TokenAndPermUserStore cache takes up so much memory and is so frequently accessed that when there are a high number of items in the cache, access takes a long time
p. 77
OS Category
1. Queries for diagnosing TokenAndPermUserStore cache
issues: -- What is the size? The larger, the slower! SELECT SUM(single_pages_kb + multi_pages_kb) AS ‘CurrentSizeOfTokenCache(kb)’ FROM sys.dm_os_memory_clerks WHERE name = 'TokenAndPermUserStore'
p. 78
OS Category
2. Queries for diagnosing TokenAndPermUserStore cache
issues: -- How many entries in the -- TokenAndPermUserStore? SELECT TOP (20)[Name], COUNT(*) AS 'Count' FROM sys.dm_os_memory_cache_entries GROUP BY [Name] ORDER BY COUNT(*) DESC;
p. 79
Scary!
p. 80
OS Category
• There are multiple ways to prevent TokenAndPermUserStore issues 1. Minimize ad-hoc SQL 2. Log in as a sysadmin (no tokens need to be generated) 3. Limit the size of the cache via sp_configure using • • •
access check bucket count access check cache quota http://support.microsoft.com/kb/955644
p. 81
OS Category
• There is one way to fix a “live” TokenAndPermUserStore issue – Free the system cache -- WARNING: Do not use on production without -a full understanding! -- After running, there will not be an entry -in sys.dm_os_memory_clerks for TAPUS!
DBCC FREESYSTEMCACHE('TokenAndPermUserStore')
p. 82
In the next video…
• Let’s look at blocking and locking…
p. 92
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
93
p. 93
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 94
Performance Monitor
• To maximize coverage, you want to use both DMVs/DMFs along with Performance Monitor – a.k.a. PerfMon and System Monitor
• PerfMon gathers stats about SQL Server as well as the rest of the environment – Disk subsystems, non-SQL resource usage
p. 95
Performance Monitor
• PerfMon uses objects and counters – Objects are for categorization – Objects contain one or more counters
• When you install SQL Server, it installs a set of instance-specific objects
p. 96
Performance Monitor
• You can view the SQL Server objects and counters using the DMV sys.dm_os_performance_counters – Careful - more than 1,100 rows
p. 97
Performance Monitor
• A few things to know about
sys.dm_os_performance_counters – Only displays SQL Server instance’s counters – Any counter that has “/sec” in the name is cumulative since time SQL Server service was started • Transactions/sec • Bulk copy rows/sec
– To accurately read these values, compare intervals • Compare 0800 reading to 0900 reading, for example
p. 98
Performance Monitor
• Example query to return all deprecated features usage – Many feature usages are actually SQL Server internals and tools! SELECT * FROM sys.dm_os_performance_counters pc WHERE object_name LIKE '%Deprecated Features%' AND cntr_value > 0 ORDER BY counter_name p. 99
Performance Monitor
p. 100
Performance Monitor
• Querying for specific counters allows adhoc views of point-in-time information – But no graphs, charts, pretty colors
• For trend analysis, you have a few choices: – Store the query results in a table for later review – Use PerfMon to store logs for later analysis
p. 101
Performance Monitor
• Some DBAs will store certain values in a table for later querying or for historical perspective – Watch the rest of the chapter prior to attempting this! -- Create a new table (”PerfCounters”) -- and store current counters
SELECT * INTO PerfCounters FROM sys.dm_os_performance_counters
p. 102
Performance Monitor
• All DBAs will use PerfMon
p. 103
In the next video…
• Let’s take a look at specific objects and counters
p. 104
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
105
p. 105
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 106
Performance Monitor
• There are a few specific areas you will want to monitor – Sometimes you can find that “Aha!” moment using ad-hoc monitoring – Sometimes you can only find that “Aha!” moment using trend analysis
p. 107
Performance Monitor
• A few topics before we go further: – PerfMon, counters with “/sec” in the name are not cumulative • Makes it easy to do ad-hoc monitoring
– PerfMon’s GUI defaults to taking measurements every one second • Too often for trend analysis • 30-seconds is generally “good enough” for ad-hoc troubleshooting
p. 108
Performance Monitor
• A few areas DBAs want to monitor: – Disk usage – CPU usage – Memory usage
• Let’s look at each!
p. 109
Performance Monitor
• A few areas DBAs want to monitor: – Disk usage – CPU usage – Memory usage
p. 110
Disk Usage
• Disk usage monitoring – Disk I/O is commonly a performance stealer – While SQL Server manages the “when” and “how often” of disk I/O, it is the operating system that performs I/O – When monitoring disk activity, you need to monitor both SQL Server’s disk activity and the disk activity of the server as a whole
p. 111
Disk Usage
• Disk I/O cannot be isolated as a metric – Example: You are monitoring disk I/O and notice that you have waits for disk reads/writes. Does this always mean that you need faster/more disks? • No! Could be that you are paging constantly and memory is the real bottleneck
p. 112
Disk Usage
• Important system objects and counters: Object
Counter
When to worry
PhysicalDisk
% Disk Time
Consistently > 90%
PhysicalDisk
Current Disk Queue Length
Particularly helpful when “% Disk Time” is high; allows to see how many I/O operations are waiting. Start worrying when this value is more than 1.5x the number of spindles
PhysicalDisk
Avg. Disk Queue Length
In many configs, a disk has 1 spindle. Start worrying when this value is more than 1.5x the number of spindles
Memory
Page faults/sec
“Some” page faults are normal; look for consistently high numbers
p. 113
Disk Usage
• PhysicalDisk: % Disk Time – Tells us how “busy” your array is – With this counter, you want to worry about sustained activity, not bursts – A % Disk Time of more than 75% for several continuous minutes occurring several times a day can likely be resolved: • By adding more disks to the array • By using faster drives
p. 114
Disk Usage
• PhysicalDisk: % Disk Time (cont.) – When you do notice this is high, start looking at the PhsyicalDisk: Current Disk Queue Length • How many disks are in my array? – Tells you how many spindles you have
• How many I/O tasks are waiting?
– If the number of waiting tasks is 1.5 or higher per spindle, you need better I/O performance!
p. 115
Disk Usage
• PhysicalDisk: Avg. Disk Queue Length – Monitor this over time • Business hours • 24 hour period • Peak times
– If you find that it exceeds more than 1.5 waiting tasks per spindle for continuous periods, increase disk performance
p. 116
Disk Usage
• Important SQL Server objects & counters to help diagnose I/O trouble – Object: • SQL Server: Buffer Manager
– Counters: • Page reads/sec • Page writes/sec
p. 117
Disk Usage
• Page reads/sec and Page writes/sec measure physical reads, not logical reads – Not logical reads! – Object: • SQL Server: Buffer Manager
– Counters: • Page reads/sec • Page writes/sec
p. 118
One Caveat
• High disk usage is not always a sign that you need more/faster disks – Sometimes poorly optimized queries can masquerade as disk/memory/CPU issues – Proper indexing and better queries can help eliminate slow queries as the culprit
p. 119
A Final Caveat
• PerfMon is great for monitoring disks directly attached to the server – Good but not great for monitoring SANs – Check with your SAN vendor for suggestions on monitoring disk activity
p. 120
In the next video…
• Let’s play with PerfMon and look at Disk, CPU, and memory counters
p. 121
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
122
p. 122
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 123
Data Collector
• While it is fun and educational to learn the various DMVs and DMFs along with the relationships between various system views, it is a huge time sink – Pro tip: download the SQL Server 2008 System Views map – http://learnitfirst.com/go.aspx?id=5GR5
p. 124
Data Collector
• How to be an unsuccessful DBA: – Try to remember all the DMVs, DMFs, and Performance Counters – Perform all of your performance data collection by running 100+ line queries daily • Bonus points if you are the only person who knows where on the network these queries/files are located
– Spend 20 hours a week writing and running DMVbased queries
p. 125
Data Collector
• How to be an successful DBA: – Learn the most important DMVs, DMFs and Performance Counters for your situation • Every job is different and every database is different • OLTP databases need different metrics than OLAP • DBAs of a “single database only” server are often concerned with different things than DBAs of “20 database” servers
– Make your reporting easy to digest and easy to run • No more 20 hours a week manually writing/running DMVbased queries!
p. 126
Data Collector
• Life as a DBA has traditionally been spent doing a lot of manual collection of performance data 1. 2. 3. 4.
Create a tables to store performance data Write the queries to collect the data Create a job to store query results in tables Write queries against tables to determine trends, etc
p. 127
Data Collector
• This was how things were done for more than a decade – As recent as SQL Server 2000… – SQL Server 2005 debuted “Custom Reports” to help us • Allowed integration of SSMS with Reporting Services to create professional quality reports based on our queries • “SQL Server 2005 Performance Dashboard” reports
p. 128
Data Collector
• SQL Server 2008 includes the Data Collector to save you time – Creates a Management Data Warehouse (MDW) that stores critical tuning, planning, and activity history – Used daily by DBAs to do capacity planning, mitigate risks, free up time otherwise spent trying to write complex DMV/DMF queries!
p. 129
Data Collector
• The Data Collector is turned off by default – Enabling it is not a trivial decision… – Performance data collection will impact your system • It’s the Heisenberg uncertainty principle in action! • How much of an impact depends on what you collect and how often
p. 130
Data Collector
• A disclaimer: – Please test, test, test this prior to deploying in a production environment – This is version 1.0 software – Easy to “over collect” • 30GB of data collection in three days!
p. 131
Data Collector
• Q& for Data Collector – “What type of information can DC collect?” • Traces, PerfMon counters, SQL queries
– “Can I use DC to collect SQL 2005/2000 servers?” • No, SQL Server 2008 only
– “Is this another one of those ‘Enterprise Edition only’ features?” • No - works great in Standard and Workgroup Editions too!
p. 132
Data Collector
• Q& for Data Collector (cont.) – “How much of an impact will this have on my server?” • Microsoft says “5% of CPU and 250-350MB” for the built-in collection sets. Custom collection sets that you create may consume more – More on this later…
p. 133
In the next video…
• Let’s learn a bit more about the Data Collector
p. 134
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
135
p. 135
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 136
Data Collector
• The Data Collector (“DC”) is built into SSMS
p. 137
Data Collector
• You can enable the DC via the Configure Management Data Warehouse Wizard
p. 138
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 139
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 140
Data Collector
• The MDW is a relational data warehouse to store all of your metrics for later analysis
p. 141
Data Collector
• When you create the MDW database, you define the name and physical file location • Placing the MDW database on either a dedicated server or dedicated disks will minimize performance impact to monitored servers
p. 142
Data Collector
• The MDW data can be shared with nonsysadmins – All sysadmins have access – Often want network admins to have access to performance data without having access to specific databases
p. 143
Data Collector
• There are three databases roles you can assign:
p. 144
Data Collector
• Once you decide security, you are ready to build the MDW
p. 145
In the next video…
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 146
Chapter 9: Monitoring and Performance Course 157: SQL Server 2008 Database Administration Presented by Scott Whigham
147
p. 147
• Chapter Introduction • Tools for Monitoring • SQL Server Profiler
Monitoring
Overview
What We’re Going to Cover
• Understanding and Using the DMVs • PerfMon • Data Collector
p. 148
Data Collector
• Configuring the DC is a two-step process: 1. Creating the Management Data Warehouse (the “MDW”) 2. Setting up the actual collection of metrics
p. 149
Data Collector
• Back through the wizard…
p. 150
Data Collector
• The Cache directory is local storage on the monitored machine – Remember that this can be run from a central server
p. 151
Data Collector
• Data collector will create SQL Server jobs to perform data collection • The job has two main components: 1. Collect the info 2. Upload the info to the MDW
• Where you collect the info locally is the cache directory – Generally the %temp% folder
p. 152
Data Collector
• Ready, set, go!
p. 153
Data Collector
• What it should look like
p. 154
Data Collector
• Now comes the work… – You can use the System Data Collection Sets – You can create your own collection sets
• There is no graphical way to create your own collection sets – Must write Transact-SQL
p. 155
In the next video…
• We’ll take a look at enabling the Data Collector and viewing the results
p. 156
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
What We’re Going to Cover
• Our final chapter is an overview of the various areas related to high availability – Log Shipping – Database Mirroring – Clustering – Peer-to-peer replication* • Not exactly “high availability” but often used in conjunction with the above
p. 3
High Availability
• High availability is a design that attempts to ensure users always have access to the database – Unplanned unavailability in some systems is not acceptable • Flight systems, emergency response teams, military operations
– Planned unavailability is allowed for maintenance
p. 4
High Availability
• Availability and uptime are not the same thing – A system can be up yet, if the network is down, it is not available
• Downtime could be a combination of things – – – –
Time spent identifying problem Time spent repairing problem Restore time Service or OS restarts
p. 5
High Availability
• To calculate total availability of a system, summarize: – Network downtime – Application downtime – Database downtime – Operating system downtime
p. 6
High Availability
• To increase a system’s availability – Decrease the amount of downtime – Decrease the frequency of downtime
p. 7
High Availability
• Applications and services often feature service level agreements (“SLAs”) that list minimum availabilities: – 90% availability means allowing 36.5 days of unplanned unavailability – 99% availability means allowing 3.65 days of unplanned unavailability – 99.999% (the “five nines”) availability means allowing ~00:05:15 of unplanned unavailability
p. 8
High Availability
• As a SQL Server DBA, your job is to ensure the availability and uptime of your – Server/service – Database(s)
p. 9
High Availability
• In this chapter, you will learn about various built-in techniques to help you achieve high availability – Some are expensive to implement – In all cases there are 3rd party solutions which may offer better alternatives – Often you will combine techniques to increase both uptime and availability • Example: A clustered instance with database mirroring
p. 10
In the next video
• Let’s get started with log shipping!
p. 11
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Log Shipping Best Practices
• There are a few categories of Best Practices as relate to Log Shipping – Backup and Restore – Network – Security
p. 3
Log Shipping Best Practices
• Backup and Restore Best Practices for Log Shipping: 1. Give enough time for your backups to complete •
If you schedule backups every 5 minutes and a backup takes 7 minutes to complete, what happens to the next backup? –
Answer: The next backup will not take place until the next scheduled time
p. 4
Log Shipping Best Practices
• Backup and Restore Best Practices: 2. The more frequently you backup, the larger MSDB becomes •
Check up on sp_delete_backuphistory stored proc for clearing out msdb periodically
3. Use backup compression if you can •
Makes for faster backups, faster network copies, and faster restores - just make sure you have the CPU power to do it!
p. 5
Log Shipping Best Practices
• Backup and Restore Best Practices: 4. The more frequently you backup, the less possible data loss at the target •
SQL Server 2008 allows for as frequent as 10 seconds!
p. 6
Log Shipping Best Practices
• Network Best Practices for Log Shipping 1. Fast networks are good; slow networks are bad
p. 7
Log Shipping Best Practices
• Network Best Practices for Log Shipping 2. If you can’t do native backup compression (available in Enterprise only) and speed of file copy is an issue, consider using 3rd party compression • • •
It does no good to take backups every 5 mins if they take 15 mins to copy 7-Zip is an industry favorite despite it’s being “unsupported” Depending on the type of data, can compress 1:1, 5:1, 20:1 or more
p. 8
Log Shipping Best Practices
• Security Best Practices for Log Shipping 1. Secure all folders that participate in log shipping • •
Having an unauthorized person get to backups is terrible Having a sysadmin who is trying to “clean up disk space” delete critical backup files means you cannot restore
p. 9
Log Shipping Best Practices
• Security Best Practices for Log Shipping 2. Use your log shipping backups in your normal disaster recovery scenario •
•
When you take a log backup for log shipping, that is your log backup for your local disaster recovery plan as well Make sure you copy to the same location as your FULL/DIFF backups!
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Database Mirroring Best Practices
• There are a few categories of Best Practices as relate to Database Mirroring – Architecture – Network – Security is not generally in this list • All data transfer is encrypted if using same domain
p. 3
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: – “Architecture” is an all encompassing term that can include • • • •
Transaction safety levels Editions used Size of database(s) Usage scenario(s)
p. 4
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 1. Transaction safety levels can influence the performance of database mirroring • Safety OFF (i.e. asynchronous mirroring) means that transactions are committed at principal without waiting for confirmation from witness/mirror – If transactions cannot be sent, it forms a queue at principal
• Safety FULL (synchronous mirroring) requires confirmation from mirror that the log has been hardened
p. 5
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 1. Transaction safety levels can influence the performance of database mirroring • Because of how differently applications use a database, it is impossible to just say, “Always use Safety ‘X’ because it performs better” • Only one thing you can do: read the whitepaper! – http://technet.microsoft.com/library/cc917681.aspx
p. 6
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 2. Use Enterprise edition for maximum configurability • Standard edition can only run Safety FULL transaction safety level – Safety FULL will have an impact on transaction throughput at the principal
• Thankfully log compression is enabled in both Standard and Enterprise editions – Log compression is a new feature in SQL 2008
p. 7
Database Mirroring Best Practices
• Architecture Best Practices for Database Mirroring: 3. Get creative when it comes to bulk imports • Database mirroring requires FULL recovery model • You can: – – – – –
Disable mirroring Perform the steps to bulk import Switch back to FULL recovery Perform a log backup and restore the log backup at the target Turn on mirroring again
p. 8
Database Mirroring Best Practices
• Network Best Practices for Database Mirroring: 1. Consider using database mirroring for local or metro-area networks •
The farther away the servers are, the longer it takes to stream the transactions – Also introduces higher potential for communication issues
• Network roundtrip times can be 100x faster on a LAN than on a WAN (Wide Area Network) – 1ms vs. 100ms
p. 9
Database Mirroring Best Practices
• Network Best Practices for Database Mirroring: 2. Faster networks mean promotions and plaudits •
Slow networks mean updated resumes and m-e-e-t-in-g-s to explain why it doesn’t work like the vendor said it would…
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
SQL Clustering Best Practices
• There are a few categories of Best Practices as relate to SQL Clusters – Hardware – Installation – Configuration
p. 3
SQL Clustering Best Practices
• Hardware best practices involve: 1. Do not use the quorum disk for any SQL Serverrelated files 2. Use identical network adapters with identical software/drivers 3. Manually set the speed and duplex mode of any adapters that allow multiple speeds •
Setting to “Auto Select” may result in dropped packets
p. 4
SQL Clustering Best Practices
• Hardware best practices involve (cont.): 4. Since the clustered disks are a single point of failure, ensure they are sufficiently redundant •
RAID 10 is quite popular
p. 5
SQL Clustering Best Practices
• Installation best practices involve: 1. Start the installation from the node that owns the disk resources 2. If you have to install SQL 2008 onto a cluster with SQL 2000 already on it, install SP4 for SQL 2000 on all nodes first, then install SQL 2008 onto the cluster 3. For a 2008 install onto a pre-existing 2005 cluster, install SQL 2005 SP2 first and then SQL 2008 p. 6
SQL Clustering Best Practices
• Configuration best practices involve: 1. On Active/Active clusters, make sure to set max server memory for each instance so that you allow for multiple instances running on one node at the same time 2. Consider using a slipstreamed installation DVD •
Slipstreaming is the process of creating an installation media that already contains a service pack and/or hotfix
p. 7
SQL Clustering Best Practices
• Configuration best practices involve (cont.): 3. Integration Services is not installed as a clustered resource •
Although you could make it a clustered resource, it is better to install it locally to each node
p. 8
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
SQL Server Replication
• Replicating data is the process of copying data and table definitions from database to database – Example: • User enters data at the Los Angeles server • Replication copies the change servers in Sydney, London, and Mumbai • After copying the change, all servers have the same data
p. 3
SQL Server Replication
• Basic terms: – Publisher: the source database – Publication: the tables, views, and stored procedures that the publisher is publishing – Subscriber: the database receiving the changes – Subscription: how the subscriber receives the data – Distributor: the server responsible for delivering the publication to the subscriber
p. 4
SQL Server Replication
Publisher (source)
Distributor
Subscribers
p. 5
SQL Server Replication
• When the data is copied to a database, it is considered synchronized – Synchronization can be instant or scheduled • Instant - as the change is entered at Publisher it is instantly replicated; Subscribers and Publisher always have the same data • Scheduled - allows latency between when data is entered at Publisher and when Subscribers have data
p. 6
SQL Server Replication
• There are three basic types of replication: – Snapshot – Merge – Transactional
• Let’s look at each!
p. 7
Snapshot Replication
• Snapshot replication: – Publisher generates a snapshot - a point-in-time copy of the publication – Distribution then replaces copy of subscribers’ data with the snapshot • Typically one-way replication (data entered only at publisher)
– Analogy: Newspaper industry
p. 8
Snapshot Replication
p. 9
In the next video…
• Let’s continue our tour of the types of replication with Merge and Transactional
p. 10
Chapter 10: High Availability
Presented by Scott Whigham
1
p. 1
• Log Shipping • Database Mirroring
Clustering & Replication
• Overview of Chapter • What is High Availability?
Log Shipping & Mirroring
Overview
What We’re Going to Cover
• Clustering • Replication • Peer to Peer Replication
p. 2
Merge Replication
• Merge replication: – Basic premise: Anyone can change the data – Transactions are replicated • Unlike Snapshot replication which replicates entire snapshot
– Changes are synchronized between all servers
p. 3
Merge Replication
• Merge replication: – Setup flow: 1. Create the publication and generate the snapshot 2. Distribute snapshot to subscribers –
At this point, Publisher and Subscribers have same data
3. Someone changes the data (could be Pub or Sub) 4. Merge Agent handles merging the data so that, in the end, everyone has the same data
p. 4
Merge Replication
• Conflicts can occur in Merge Replication – What happens when two sites change the same data at the same time?
• You can help lessen the chance of a conflict: – You can specify that changes to the same row by different sites is allowed provided that they change different columns – At synchronization, the Merge Agent merges the row and synchronizes all servers so that everyone has the same data
p. 5
Merge Replication
• You can define how conflicts are resolved in several ways: – By setting priority levels for each site • In the event of a conflict between two sites, the highest priority site’s changes “win”
– By setting, “First in wins” or “Last in wins” • Whoever made the change first/last “wins”
– By writing custom business rules
p. 6
Merge Replication
p. 7
Transactional Replication
• Transactional replication: – Transactions are replicated between servers – Transactional can: • Be one-way: only the Publisher can replicate changes • By multi-way: changes can occur at any site
– Transactional may or may not have conflict detection • Depends on topology • Rule of thumb: “Last in wins” unless you configure differently p. 8
Transactional Replication
p. 9
P2P Transactional Replication
• Peer to Peer (P2P Transactional replication: – Allows both scale-out and high availability • Easily scales since each site acts as Publisher and Subscriber (called “Nodes”) • Every site has the same data so clients can connect to any site thus allowing a degree of load balancing • If one site goes down, users just connect to another site
– Debuted in SQL 2005
p. 10
P2P Transactional Replication
p. 11
P2P Transactional Replication
• Conflicts can occur in P2P – What happens when a user connected to the Dallas node updates a row and, at the same time, a user on the Houston node deletes that row? • Default: Last in wins • Can enable “conflict detection” - default setting is to stop distribution when a conflict is found
p. 12
Let’s demo!
• Let’s demo P2P!
p. 13
Related Documents
Sql Server 2008 Database Administration
June 2020 9
Sql Server 2008 Database Administration
June 2020 12
Sql Server 2000 Database Administration
June 2020 10
Sql Server 2005 Database Administration
June 2020 8
Windows Server 2008 Administration
June 2020 16
Sql Server Database Interview Questions
November 2019 9More Documents from ""
Sql Server 2005 Reporting Services
June 2020 21
Working With Databases
June 2020 8
Exchange Server 2003 Administration
June 2020 16
C# Programming With Visual Studio 2005
June 2020 8