Cobol Day 7&8

VSAM

Overview

VSAM (Virtual Storage Access Method ) VSAM is an integral part of MVS. At the end of this course, you, the user, will know VSAM in detail.

 

know the different types of VSAM data sets



know the organization of VSAM data sets

be able to create, delete and alter VSAM data sets, with indexes and alternate indexes

Access Methods 

An access method defines the technique by which data is stored and retrieved. The different types of dataset organizations in MVS are:

    

Physical Sequential Partitioned Indexed Sequential Direct VSAM

What is VSAM?  

VSAM is Virtual Storage Access Method



VSAM operates in Virtual Environment

It is a method used to move data between Disk and Main Storage

VSAM 

VSAM acts as interface between

 Operating System and Application Program

Application

Request

V S A M

Reply

Operating System

VSAM 

Interface between Main Storage and Disk

Main Storage

DATA

V S A M

DATA

Disk

File access methods 

Data (Records) is retrieved

 Sequential (Reading from beginning to end)  Random (Records are read by the value in the key)

 Direct (Records are read based on their physical location/address on disk)

 

VSAM provides all these methods One access method supporting all types of data retrieval

Traditional access methods  

QSAM (Queried Sequential Access Method) BSAM (Basic Sequential Access Method)

 for ‘flat’ files



ISAM (Index Sequential Access Method)

 for Index files



BDAM (Basic Direct Access Method)

 for direct access files

VSAM Dataset Types Entry Sequenced Dataset (ESDS)



ESDS contains records in the order in which they are entered. Records are added to the end of the data set, and can be accessed sequentially.

Key Sequenced Dataset (KSDS)



KSDS contains records in ascending collating sequence. Records can be accessed by a field called a key, or by a relative byte address (RBA - relative position of the record from the beginning of the dataset), or sequentially.

Linear Dataset (LDS)



LDS contains data that has no record boundaries. The manipulation of the data is completely controlled by the user.

Relative Record Dataset (RRDS)



RRDS contains records in the order of relative record number. These records can be accessed only by this number.

VSAM data set organization 

VSAM Data Set can contain three major components

 CLUSTER (Catalog entry)  INDEX  DATA (Actual data)



Data Set is referred by cluster name in JCL

CLUSTER

INDEX DATA

VSAM internals   

CONTROL INTERVAL (CI) VSAM stores Data and Index in Control Intervals (CI) CI is similar to ‘Block’

CI

Record

Record

.......

CONTROL INTERVAL 

CI is the unit of retrieval between DASD and I/O Buffer (Virtual Storage)

DASD

I/O Buffer

CI R1

R2

R3

R1

R2

R3

Control Interval 

CI contains

 Records (or DATA)  Free space (Optional)  Control Information Definition Field (CIDF)  Record Definition field (RDF)

Building blocks of a VSAM Dataset Logical Record



Logical records of VSAM data sets are stored in a different manner from logical records in non-VSAM data sets. VSAM stores records in control intervals.

Control Interval (CI) A control interval consists of the following:

  



Logical records (LR) Free space (FS) Control information fields

LR

LR

LR

LR..FS....FS...RDF

CIDF

Free Space is the space reserved within the CI for inserting new records in a KSDS or for lengthening the existing records.

CONTROL INTERVAL  

R1

CIDF & RDF are VSAM control functions Used by VSAM to access data

R2

R3

Free Space

3 bytes, Length of Record How many successive records have same length

R D F

R D F

C I D F

4 bytes long, One per CI Indicates Free space, where and how much

CONTROL AREA Control Area

 

CIs are grouped into CA



A VSAM data set is actually composed of one or more control areas. The maximum size of a control area is a cylinder, and the minimum size is one track.

The control intervals in a VSAM data set are grouped together into Control Areas.

Control Area (CA) LR LR LR LR.. FS....FS... RDF CIDF LR LR LR LR.. FS....FS... RDF CIDF LR LR LR LR.. FS....FS... RDF CIDF

Cluster  

VSAM datasets are defined as Clusters. A Cluster is a combination of the data component and the index component.



The Index component is applicable only in the case of a KSDS.



The data component holds data records.



The index component holds the index records to access the required information from the data component

Data Component (CA2)

1001 1002

1005

1010

FS

1020 1022

1030

1055

FS

FS

FS

FS

FS

FS

Highest Index Set 2 210

5000

Index Set 1 055

1600

FS

5000

FS

FS

Sequence Set 1010

1055

FS

1350

1600

F S

DATA COMPONENT (CA1)

1300

1305

1310

1350

FS

1400

1410

1415

1600

FS

FS

FS

FS

FS

FS

Control Interval Split CA-1 (CI-1 to CI-3) - Before split





When a data set is first loaded, the key sequence of records and their physical order are the same. However, when records are inserted, control interval split can occur, causing the data control intervals to have a physical order that is different from the key sequence.

1001

1002

1005

1010

FS

1020

1022

1030

1055

FS

FS

FS

FS

FS

CA-1 (CI-1 to CI-3) - After split



For example, try to insert '1004' in CI-1 shown below :

1001

1002

1004

FS



A Control Interval Split occurs and CI-1 is split exactly into two half CI's. CI-3, which is a free control interval is used, and some logical records of CI-1 are moved into CI-3. CI-3 is placed after CI-2 and it is

1020

1022

1030

1055

1005

1010

FS

FS

FS

Control Area Split 

Now, if record ‘1025’ is inserted, there is no free control interval for a CI split to occur. Hence, a CA (control area) split occurs. The CA-1 is split into two half control areas; some of the Control intervals of CA-1 are moved into the newly created CA (CA-2). CA-1 (CI-1 to CI-3) - After split

1001

1002

1004

FS

1020

1022

FS

FS

FS

FS

FS

FS

CA-2 (CI-1 to CI-3)

1025

1030

1055

FS

1005

1010

FS

FS

FS

FS

FS

FS

FS

Types of VSAM data sets    

ESDS

Entry Sequenced Data Set

KSDS

Key Sequenced Data Set

RRDS

Relative Record Data Set

LDS

Linear Data Set

ESDS  

Similar to Sequential File



New Records are added at the end only (chronological order)

 

Supports both Fixed and Variable formats

Sequenced by the order in which data is entered/loaded

Contains only CLUSTER & DATA components

ESDS

(Contd...)

 

Only sequential access in Batch Cobol Programs



Alternate Index is supported in on-line applications (CICS)



NO primary index

Random access is supported in on-line applications (CICS) using Relative Byte Address (RBA)

RBA 

Record location relative to the beginning of the file (Relative Byte Address)

CI

R1 80

R2 40

R3 60

RBA of R1 is 80

Access Method Services (AMS)



Access Method Services is a service program that helps you to allocate and maintain VSAM data sets and catalogs.



IDCAMS is the name of the utility program used to perform the following operations on VSAM data sets.  Creating a Data set  Loading a VSAM data set.  Printing a Data set  Deleting a Data set  Error Detection for KSDS Data set  Creating Generation Data Groups (GDG) etc.

INVOKING ACCESS METHOD

IDCAMS UTLITY

Defining a VSAM Cluster 

The Define Cluster command is used to allocate VSAM data sets. The basic information required for defining a VSAM data set is:

    

Name of the cluster.

   

Record size, and whether it is fixed or variable in length.

Volume(s) on which the data set will be allocated. Type of data set (KSDS, ESDS, RRDS or LDS). Space needed for the data set. For a KSDS, the length of the key and its offset from the beginning of the record. Control interval size. CI and CA Free Space. The following keywords have to be used to define the different types of VSAM data sets:  NONINDEXED for ESDS.  INDEXED for KSDS.  NUMBERED for RRDS.  LINEAR for LDS.

Sample JCL to create an ESDS cluster

//LEM0UXXA JOB MSGCLASS=Q,CLASS=Q,NOTIFY=&SYSUID //DEFKSDS EXEC PGM=IDCAMS //SYSPRINT DD //SYSIN

DD

NONINDEXED - ESDS

SYSOUT=*

*

DEFINE CLUSTER(NAME(LEM0UXX.TEST.ESDS) VOLUMES(LEM001) NONINDEXED TRACKS(2,1) RECORDSIZE(50,50)

-

N1=> AVG RECL

-

N2=> MAX RECL

-

CONTROLINTERVALSIZE(4096) FREESPACE(10,20))

RECORDSIZE(N1,N2)

-

-

-

DATA(NAME(LEM0UXX.TEST.ESDS.DATA)) /*

Freespace(PCT1,PCT2) PCT1=> PCT FREE SPACE IN each CI PCT2=> PCT Of unused CI in CA

COBOL AND ESDS

OVERVIEW

ESDS  

Similar to Sequential File



New Records are added at the end only (chronological order)

 

Supports both Fixed and Variable formats

Sequenced by the order in which data is entered/loaded

Contains only CLUSTER & DATA components

ESDS

(Contd...)

 

Only sequential access in Batch Cobol Programs



Alternate Index is supported in on-line applications (CICS)



NO primary index

Random access is supported in on-line applications (CICS) using Relative Byte Address (RBA)

FILE-CONTROL

Paragraph

  

Format:  

SELECT [OPTIONAL] File-name-1 ASSIGN TO Assignment-name-1 SEQUENTIAL [ ORGANIZATION IS

INDEXED RELATIVE SEQUENTIAL

[ACCESS MODE IS

RANDOM DYNAMIC

  [FILE STATUS IS Data-name-1]        

ORGANIZATION SEQUENTIAL => ESDS INDEXED => KSDS RELATIVE => RRDS ACCESS MODE OPTIONS ESDS : SEQUENTIAL KSDS : SEQUENTIAL or RANDOM or DYNAMIC RRDS: SEQUENTIAL or RANDOM or DYNAMIC

ESDS  

ORGANIZATION IS SEQUENTIAL ACCESS MODE IS SEQUENTIAL

INPUT READ

YES

WRITE

-

OUTPUT -

INPUTOUTPUT

EXTEND

YES

-

YES

YES

DELETE REWRITE

YES

VSAM === ESDS ORGANIZATION IS SEQUENTIAL. ACCESS MODE = SEQUENTIAL OVERVIEW

CREATING ESDS

Writing Data in ESDS

Writing Data in ESDS

Writing Data in ESDS

RUNNING A PROGRAM

View the data from ESDS

REPRO COMMAND

PRINT ESDS EXAMPLE

Overview

PRINT ESDS

PRINT ESDS

PRINT ESDS

KSDS (KEY SEQUENTIAL DATA SET )

Sample JCL to create a KSDS cluster KEYS(N1,N2) N1=> //LEM0UXXA JOB MSGCLASS=Q,CLASS=Q,NOTIFY=&SYSUID

N2=> is starting byte position

//DEFKSDS EXEC PGM=IDCAMS

of Key in Record

//SYSPRINT DD SYSOUT=* //SYSIN

length of the Key(Bytes)

DD *

DEFINE CLUSTER(NAME(LEM0UXX.TEST.KSDS) VOLUMES(LEM001)

-

INDEXED

-

KEYS(9,0)

-

N2=> MAX RECL

INDEXED - KSDS -

CONTROLINTERVALSIZE(4096) FREESPACE(10,20))

RECORDSIZE(N1,N2) N1=> AVG RECL

TRACKS(2,1)

RECORDSIZE(50,50)

-

-

-

DATA(NAME(LEM0UXX.TEST.KSDS.DATA)) INDEX((NAME(LEM0UXX.TEST.KSDS.INDEX))

Freespace(PCT1,PCT2) PCT1=> PCT FREE SPACE IN each CI PCT2=> PCT Of unused CI in CA

SELECT & ASSIGN Syntax

READ statement

WRITE & REWRITE statement

KSDS  

ORGANIZATION IS INDEXED ACCESS MODE IS SEQUENTIAL

INPUT READ

YES

WRITE

-

OUTPUT -

INPUTOUTPUT YES

YES

DELETE

YES

REWRITE

YES

START

YES

YES

INDEX FILE ORGANIZATION IS INDEXED. ACCESS MODE = SEQUENTIAL

OVERVIEW

CREATING KSDS

WRITING A KSDS

WRITING A KSDS

WRITING KSDS

RUNNING A KSDS

Running a KSDS

READING INDEX FILE-KSDS ORGANIZATION IS INDEXED. ACCESS MODE = SEQUENTIAL

OVERVIEW

READING A KSDS

READING A KSDS FILE

EXECUTE A PROGRAM FOR READING DATA FROM KSDS

SPOOL OUTPUT

SPOOL OUTPUT

START & DELETE statement

START :Position a Record

START COMMAND

START COMMAND

DELETE COMMAND

DELETE COMMAND

EXECUTE A PROGRAM AFTER COMPILE

PRINT THE KSDS DATASET

PRINT THE KSDS

KSDS ORGANIZATION IS INDEXED ACCESS MODE IS RANDOM

KSDS  

ORGANIZATION IS INDEXED ACCESS MODE IS RANDOM

INPUT READ

YES

WRITE

-

OUTPUT YES

INPUTOUTPUT YES YES

DELETE

YES

REWRITE

YES

START

KSDS – RANDOM- WRITE

KSDS – Random -WRITE

KSDS –RANDOM MODE READ Operation

Overview

KSDS – RANDOM -READ

KSDS – RANDOM -READ

KSDS ORGANIZATION IS INDEXED ACCESS MODE IS DYNAMIC

KSDS  

ORGANIZATION IS INDEXED ACCESS MODE IS DYNAMIC

INPUT READ

YES

WRITE

-

OUTPUT YES

INPUTOUTPUT YES YES

DELETE

YES

REWRITE

YES

START

YES

YES

KSDS- DYNAMIC MODE WRITE OPERATION

Overview

KSDS –DYNAMIC- WRITE

KSDS –DYNAMIC-WRITE

KSDS –DYNAMIC-READ

KSDS-DYNAMIC-READ

RRDS

Overview

Sample JCL to create an RRDS cluster //LEM0UXXA JOB MSGCLASS=Q,CLASS=Q,NOTIFY=&SYSUID //DEFRRDS EXEC PGM=IDCAMS

NUMBERED - RRDS

//SYSPRINT DD SYSOUT=* //SYSIN

DD *

DEFINE CLUSTER(NAME(LEM0UXX.TEST.RRDS) VOLUMES(LEM001)

TRACKS(2,1) RECORDSIZE(50,50)

N1=> AVG RECL

-

N2=> MAX RECL

-

CONTROLINTERVALSIZE(4096) FREESPACE(10,20))

RECORDSIZE(N1,N2)

-

NUMBERED

-

-

DATA(NAME(LEM0UXX.TEST.RRDS.DATA)) /*

-

Freespace(PCT1,PCT2) PCT1=> PCT FREE SPACE IN each CI PCT2=> PCT Of unused CI in CA

RELATIVE-RANDOM-WRITE

RELATIVE-RANDOM-WRITE

RELATIVE-RANDOM-READ

RELATIVE-RANDOM-READ

RELATIVE-DYNAMIC-READ

RELATIVE-DYNAMIC-READ

RELATIVE-DYNAMIC-READ

ALTERNATE INDEX

Overview

ALTERNATE INDEXES 

Used whenever the data is required to be retrieved on the basis of more than one field

EMPNO

3 4 5 6 

ENAME

SALARY

RAJESH

5000

RAMESH

6000

RANDY

7000

SURESH

8000

e.g.,

Can be defined for both KSDS & ESDS

EMPNO IS

ENAME IS

BASE KEY

ALTERNATE KEY

ALTERNATE INDEXES Reduce Can

data redundancy

have duplicates Easy to define using IDCAMS Allow datasets to be accessed sequentially or randomly Can be updated automatically

Disadvantages  

Performance degradation Complex update logic

Steps for Creating Alternate Index

 

Define AIX using IDCAMS DEFINE AIX



Build AIX & populate it using IDCAMS BLDINDEX

Specify Alternate Index Path using IDCAMS DEFINE PATH

Sample JCL to create an AIX cluster UPGRADE-> VSAM AUTOMATIC Updates the AIX for all ADD,UPDT,DEL of Base cluster

//LEM0UXXA JOB MSGCLASS=Q,CLASS=Q,NOTIFY=&SYSUID //DEFKSDS EXEC PGM=IDCAMS //SYSPRINT DD //SYSIN

DD

UNIQUE /NONUNIQUE

SYSOUT=*

*

KEY DEFINE AIX (NAME(LEM0UXX.TEST.AIX.CLUSTER) – RELATE (LEM0UXX.INFILE.KSDS)

N1->- LENGTH OF AIX key

VOLUMES(LEM001)

-

N2-> KEYS start Loc in BASE CLUSTER

KEY (10, 35)

-

TRACKS(2,1) NONUNIQUEKEY RECORDSIZE(49,49)

RECORDSIZE(N1,N2)

-

N1=> AVG RECL -

N2=> MAX RECL

UPGRADE CONTROLINTERVALSIZE(4096) FREESPACE(10,20))

-

-

DATA(NAME(LEM0UXX.TEST.AIX.DATA)) INDEX(NAME(LEM0UXX.TEST.AIX.INDEX)) /*

( N1,N2)

Freespace(PCT1,PCT2) PCT1=> PCT FREE SPACE IN each CI PCT2=> PCT Of unused CI in CA

UPGRADE/NOUPGRADE

• Syntax : UPGRADE/NOUPGRADE • UPGRADE specifies that records in AIX are to be updated • automatically whenever the base cluster is updated

BUILDING AN INDEX

Overview

BUILD INDEX //JOB CARD //STEP1 EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=*

BUILD INDEX is used

//SYSIN DD * BLDINDEX

To load records in

-

INDATASET(LEM0UXX.KSDS.CLUSTER')

-

AIX after it has been def

OUTDATASET('LEM0UXX.KSDS.AIX.CLUSTER') /*

PATH AND ALTERNATE INDEX

Overview

DEFINE PATH //JOB CARD //STEP1 EXEC PGM = IDCAMS

Path Establishes

//SYSPRINT DD SYSOUT=*

A Bridge Between

//SYSIN

DD *

BASE CLUSTER & AIX

DEFINE PATH ( NAME ( LEM0UXX.KSDS.PATH) – PATHENTRY(LEM0UXX.KSDS.AIX.CLUSTER) – UPDATE ) /* //

ALTERNATE Index Example

Define BASE CLUSTER

LOADING DATA INTO BASE CLUSTER

LOADING DATA INTO BASE CLUSTER

LOADING DATA INTO BASE CLUSTER

EXECUTING A PROGRAM TO LOAD DATA

EXECUTING A PROGRAM TO LOAD DATA

DEFINING AIX

RECORDSIZE(N1,N2) N1= LENGTH OF BASE CLUSTER KEY(3) + LENGTH OF AIX KEY(5) + 5 for CI

Build index

Create path for Bridge

READING RECORDS BY ALTERNATE INDEX

READING RECORD BY ALTERNATE KEY

READING RECORD BY ALTERNATE KEY

EXECUTE A READ PROGRAM BY ALTERNATE KEY

DD1-> BASE CLUSTER DD11-> PATH NAME

DDNAME SHOULD BE IN ORD

SPOOL OUTPUT